Intro to public sector data

How is public sector data kept safe?

With such a wide range of personal information collected, it’s very important that data is held safely and securely. Measures to keep data secure include physical controls such as keeping it in a locked, access-controlled environment and technical security measures, such as strong password protocols with multi-factor authentication, hardware encryption, firewalls and anti-virus software.

Securing data also involves training staff to make sure they are using the correct processes and policies when they are processing data. The Information Commissioner’s Office (ICO) publishes guidance around the technical and organisational controls which organisations must implement to protect data, including those processing public sector data.

How is data made available for research?

When it is ready to be used for research, public sector data is often stored in a Trusted Research Environment such as the National Safe Haven – a secure environment which houses Scottish health data and can only be accessed by approved researchers.

To find out more about Trusted Research Environments, read our explainer: What are Trusted Research Environments?

When data is shared with researchers or other public sector organisations, it is first pseudonymised to ensure that it cannot be traced back to any individuals. Pseudonymisation is a process where information that could directly identify an individual (such as their name) is removed or replaced with a pseudonym such as a random key.

Pseudonymising data is a privacy-friendly way to maximise the potential of public data when improving or developing public services. More information can be found on the ICO’s website.

There is also legislation in place in the UK to protect people’s personal data, including personal public sector data, and these include the Data Protection Act 2018, UK GDPR and the Digital Economy Act 2017.

How do researchers access public sector data?

To access public sector data, researchers must submit an application outlining what data they are requesting access to, how it will be used and how it will be kept safe. Then, the research proposal goes through an information governance (IG) process where it is reviewed to ensure the data will be used safely.

The IG approval panel may include the organisation that controls the data, known as the data controller, and external IG structures such as the NHS Scotland Public Benefit and Privacy Panel for Health and Social Care (HSC-PBPP). If the application is successful, a legal agreement is created between the data controller and the researcher to define how the data can be used and published.

What are the barriers to accessing public sector data for research?

At the moment, public sector data is often locked away in lots of individual systems, across many different organisations. This makes it difficult and time-consuming for researchers to access and use data.

For example, if a researcher wanted to compare health data with homelessness data, they would need to apply to two organisations separately. This means more time is spent applying for data and waiting for access, which can delay research and lead to limited or outdated conclusions.

By making it quicker and simpler to access public sector data - without sacrificing security - Research Data Scotland aims to speed up research, making it easier for decision-makers to base their insights on up-to-date, good quality research built on good quality data.

Find out more about the work we're doing to improve access to data.

What is data linkage?

Data linkage is when an individual’s pseudonymised data can be connected across multiple datasets. This allows researchers to discover relationships between different areas of the public sector and understand how people interact with different services.

Before an individual’s data can be linked, their personal information is removed and replaced with a linkage key – a unique identifier which allows a person’s data to be connected across different datasets without containing information that could be used to identify them.

Open data

Lots of public sector data is sensitive and can be linked to individuals, meaning it needs to be kept secure. However, once it has been anonymised and aggregated to ensure it cannot be traced back to an individual, some public sector data can be safely shared publicly. This anonymised data is known as open data.

Open data is freely accessible to anybody and can help provide broad insights into certain areas. It includes information on topics such as business and economy, crime and justice, education, health, transport and others.

Platforms such as the Scottish Government’s open data platform provide portals for accessing open data from across Scotland.