Skip to content

Intro to public sector data

Want to know more about data but don’t know where to start? We’ve created a short introduction to help you understand what public sector data is and how it’s used in research.

What is public sector data?

Throughout a person’s lifetime, they generate lots of information which is collected by public sector bodies such as the NHS, local authorities, government bodies, schools, social care organisations, and many others.

For example, GP records might contain data about an individual’s medical history and diagnoses. Similarly, local authorities and government bodies may store information about a person’s employment status and housing information. Most public sector organisations will also hold general person-level data such as an individual’s name, date of birth and address.

This information is known as public sector data or administrative data.

This data allows public sector organisations to deliver their services, but it also provides a wealth of information which can be used for the wider public good.

The term ‘public sector data’ encompasses a very broad range of information about people, places and businesses, meaning it is a vital resource in understanding how a society operates and how it can be improved. When researchers are able to access public sector data, they can discover valuable insights which help develop systems and improve lives on a national scale.

How is public sector data kept safe?

With such a wide range of personal information collected, it’s very important that data is held safely and securely. Measures to keep data secure include physical controls such as keeping it in a locked, access-controlled environment and technical security measures, such as strong password protocols with multi-factor authentication, hardware encryption, firewalls and anti-virus software.

Securing data also involves training staff to make sure they are using the correct processes and policies when they are processing data. The Information Commissioner’s Office (ICO) publishes guidance around the technical and organisational controls which organisations must implement to protect data, including those processing public sector data.

How is data made available for research?

When it is ready to be used for research, public sector data is often stored in a Trusted Research Environment such as the National Safe Haven – a secure environment which houses Scottish health data and can only be accessed by approved researchers.

To find out more about Trusted Research Environments, read our explainer: What are Trusted Research Environments?

When data is shared with researchers or other public sector organisations, it is first pseudonymised to ensure that it cannot be traced back to any individuals. Pseudonymisation is a process where information that could directly identify an individual (such as their name) is removed or replaced with a pseudonym such as a random key.

Pseudonymising data is a privacy-friendly way to maximise the potential of public data when improving or developing public services. More information can be found on the ICO’s website.

There is also legislation in place in the UK to protect people’s personal data, including personal public sector data, and these include the Data Protection Act 2018, UK GDPR and the Digital Economy Act 2017.

How do researchers access public sector data?

To access public sector data, researchers must submit an application outlining what data they are requesting access to, how it will be used and how it will be kept safe. Then, the research proposal goes through an information governance (IG) process where it is reviewed to ensure the data will be used safely.

The IG approval panel may include the organisation that controls the data, known as the data controller, and external IG structures such as the NHS Scotland Public Benefit and Privacy Panel for Health and Social Care (HSC-PBPP). If the application is successful, a legal agreement is created between the data controller and the researcher to define how the data can be used and published.

What are the barriers to accessing public sector data for research?

At the moment, public sector data is often locked away in lots of individual systems, across many different organisations. This makes it difficult and time-consuming for researchers to access and use data.

For example, if a researcher wanted to compare health data with homelessness data, they would need to apply to two organisations separately. This means more time is spent applying for data and waiting for access, which can delay research and lead to limited or outdated conclusions.

By making it quicker and simpler to access public sector data - without sacrificing security - Research Data Scotland aims to speed up research, making it easier for decision-makers to base their insights on up-to-date, good quality research built on good quality data.

Find out more about the work we're doing to improve access to data.

What is data linkage?

Data linkage is when an individual’s pseudonymised data can be connected across multiple datasets. This allows researchers to discover relationships between different areas of the public sector and understand how people interact with different services.

Before an individual’s data can be linked, their personal information is removed and replaced with a linkage key – a unique identifier which allows a person’s data to be connected across different datasets without containing information that could be used to identify them.

Find out more about data linkage in this short video, supported by funding from Health Data Research UK (HDR UK) and the Medical Research Council (MRC):

Click here to watch this video with British Sign Language (BSL) interpretation.

Open data

Lots of public sector data is sensitive and can be linked to individuals, meaning it needs to be kept secure. However, once it has been anonymised and aggregated to ensure it cannot be traced back to an individual, some public sector data can be safely shared publicly. This anonymised data is known as open data.

Open data is freely accessible to anybody and can help provide broad insights into certain areas. It includes information on topics such as business and economy, crime and justice, education, health, transport and others.

Platforms such as the Scottish Government’s open data platform provide portals for accessing open data from across Scotland.

Discover more about data

Illustration of a lock.

What is the Five Safes framework?

Discover what the Five Safes framework is and how it's used to keep data secure.

Learn more
Illustration of a line graph made to look like mountains.

What are Trusted Research Environments?

Learn about Trusted Research Environments and how they help researchers access data.

Learn more
Illustration of a filing cabinet with numbers spilling out of an open drawer.

What is data de-identification?

One way of keeping data confidential is to remove personal information. Learn more about this process, known as de-identification.

Learn more

Subscribe to our updates 

To stay updated with Research Data Scotland, subscribe to our mailing list and follow us on X (Twitter) and LinkedIn

Sign up here Sign up here
Illustration of an envelope with a letter sticking out and a mobile phone with a person

Was this information helpful?