Making it easier for researchers to discover data: recent improvements to the RDS metadata catalogue

The Research Data Scotland metadata catalogue collates information on datasets available for research in Scotland. Since launching the catalogue in its current form in 2024, we have made a series of improvements to the layout and content in response to input from users of the catalogue.

In a recent phase of improvements, we aimed to improve consistency across records in the catalogue and to display information about each variable within a dataset.

Clarifying scope and guidance

We started by clearly defining the scope of the catalogue to be datasets held in Trusted Research Environments (TREs) in Scotland that can be requested to conduct research in the public good.

We also improved the guidance and templates we use to collect information from data providers, to make it clearer how we expect each record to be completed.

This change should result in more consistent information being provided, which will make it easier for researchers to look across datasets to identify variables they need for their research.

Adding a data dictionary section to records

Utilising funding from Administrative Data Research Scotland (ADR Scotland), we worked with our digital providers, MetadataWorks and Nexer to add a ‘Data dictionary’ section to catalogue records. The data dictionary has information on all the variables available in each dataset. This information is crucial for researchers planning projects to understand exactly what the dataset contains.

Previously, this information was presented in separate documents within each catalogue record, but researchers told us that they wanted to more easily see this detail up front. Now, researchers are able to see variable names, descriptions and data types on the page, while also having the option to download this information in files.

Adding data dictionary content

We have added data dictionaries for several datasets and will work to populate this section for all datasets over the coming months. The availability of data dictionaries should reduce back and forth between researchers and data providers to understand what is in the dataset, resulting in clearer research proposals that don’t request variables that aren’t needed.

Re-routing the flow of data

The underlying principle for data flow to/from the catalogue is to only edit in one place, then feed metadata between catalogues, with the aim to reach as many researchers as possible. Another change we implemented was to make information flow from the RDS catalogue to the HDR Gateway, which is a catalogue for UK-wide health datasets. This change allows us to provide more support to health data providers sharing their metadata with us so we can work with them to improve quality and completeness.

There were challenges with making changes to the catalogue, because data from it feeds into several other platforms and processes, so changes had knock-on impact. Our Digital and Data teams worked closely together and with our digital providers to support each other, and developed better understanding of all the platforms throughout the change process.

Summary and next steps

Together these improvements have delivered more detailed information in the catalogue, and we have clearer end-to-end processes and guidance, which should result in more consistent content. Thanks to funding from ADR Scotland, we were able to deliver these improvements within around seven months.

Looking forward, we have ideas for further improvements to include additional fields like missingness (where no data is recorded), summary statistics and previous use of datasets. We are also interested in understanding the impact of improved metadata on research projects.

If you have any feedback about the catalogue, please get in touch with us.