Barriers to public sector data sharing identified – and how to overcome them
The coronavirus response led to an increase in data sharing across the public sector. A Global Government Forum webinar brought together senior UK figures to discuss how the lessons from the pandemic might unlock better collaboration
During the webinar, Lisa Allen, the director of data and services at the Open Data Institute (ODI) highlighted the main barriers to sharing data in the public sector. Among them are poor availability of data and unsuitable formats; the technology it is stored on, which may not promote shareability; and woolly policies and processes which lead to confusion around who can approve the sharing of data.
But the big one that can get in the way is organisational culture, she said. “This can stop people sharing data widely, because actually, people may not realise that the data will be beneficial to others, and we’re not always the best at understanding how others will use it.”
Read more: Indian government urged to prioritise privacy as it embarks on data-sharing plan
When Allen worked at the Environment Agency, she was part of the team that released 2,000 open data sets in 2015. As well as helping catapult the UK into the global top 10 for data maturity, this also led to the agency’s data cropping up in some unexpected areas.
“When we released that data, it was picked up in things like Minecraft – stuff where we would never have guessed it would have been used. That just shows that you’re not always the best to anticipate how your data is going to be used once it’s shared.”
Allen also spoke of the risks associated with data sharing, pointing the webinar’s audience towards a risk assessment guide created by the ODI which aims to help organisations “identify, assess and manage risks related to sharing data that they hold”. The guide sets out early steps to consider real and perceived data-sharing risks and identify suitable mitigating actions.
There is a spectrum of data, from open to closed, Allen said. Personal data or that related to national security are at the closed end of the spectrum but there is plenty of data that can be shared for public good, as the coronavirus response revealed. “During the pandemic, data was shared across the public and private sector to see if the [public health] policy was working or the impact on the high streets,” she said.
Piecing together the COVID puzzle
Pete Stokes, the director of the Integrated Data Programme at the UK’s Office for National Statistics (ONS), set out more information about how data had been used in the COVID response.
In particular, he focused on how the ONS had used census data to help government understand the prevalence of the virus in different parts of the population.
“Early on in COVID, a lot of people might have seen stories saying that people from ethnic minorities appeared to be suffering from COVID more frequently and more severely than white people. That was clear and evident from the statistics the NHS was producing, but what they couldn’t tell was why. So we in ONS worked really closely with NHS Digital to get more data to understand the problem,” Stokes explained.
Using a secure research space that the ONS created to help share public sector data, it supplemented NHS data with census information to provide a more complete picture of ethnicity. The ONS was also able to add in information on jobs and employment status and household living structures.
“This means you can look beyond the simplistic correlation and look for a more causal relationship,” he said. “Then you can see that people in some minority groups are more likely, for example, to live in multigenerational households and are therefore more likely to have children catching COVID, or working adults catching COVID, then bringing it home and then transmitting it to old parents and grandparents who live with them, who were then more susceptible to more serious illness.
“That explains a lot of the difference we’ve seen. People in some ethnic groups are also demonstrably more likely to work in public-facing roles in cities, for example, or more likely to work in the health service, and therefore more likely to be exposed to higher viral load.
“Being able to understand that helped directly inform the government response and understand the genuine causes of the differences we were seeing.”
Read more: Crystal clear: Sir Ian Diamond on using data to profile the pandemic
Later in the COVID response, ONS was also able to add vaccine uptake data to this information which helped to inform the government’s vaccine campaign. This, Stokes said, was “driven by secure and robust and appropriate linkage, and use of data from across the public sector”.
These data sharing examples were “really challenging to do”, he said, even when the pandemic gave “a clear sense of urgency” to find a way forward.
“If we hadn’t had that, we probably would still be trying to link the data now is the reality. And part of that is, when we look legally at how to share data, there’s a distinct difference between health data and other data held in the public sector, but linking those data is hard without that push.”
Unlocking ‘effervescent data’
Nadun Muthukumarana, data analytics partner at Deloitte, which was the webinar’s knowledge partner, told the audience that before COVID, most data was “long and narrow” – specific data related to a service covering a long time period, usually kept in one organisation. But, according to Deloitte research, when the pandemic happened, the information needed was different.
It was, as Stokes indicated, related to specific policy developments around the pandemic – what Muthukumarana called “short and wide” data. “You needed short-term data, it could be in a few days, or it could be a few weeks, but it certainly didn’t need to go back three years or five years. And it is wide in that it is not only data created by your own organisation, but data collected and shared across your value chains, both across public sector and private sector.”
Muthukumarana called this “effervescent data”, as the information keeps changing and is spread across a wide network.
“The sharing of effervescent data has added a new dynamic to cross-public sector data sharing, and I think it’s a very positive addition to the nature of the data that is now being shared,” he said.
The question now is whether the sharing of this kind of data will stop post-COVID.
“Hopefully a lot of the organisations who started to share this sort of data as well as the historic long and narrow data will continue with both types of data being shared across public sector, and with some adjacent private sector organisations, to create that rich data,” Muthukumarana said.
Read more: Australian government agencies ‘struggling to make data-driven decisions’
This could have benefits for services, but he also flagged that work is needed to ensure there is public trust in this kind of sharing in non-emergency situations.
“I think it’s well-rehearsed in the way you acquire loads of long and narrow or historical datasets and you are able to identify patterns or do research. But there is learning curve around how do you not only share, but preserve the accuracy of data as it changes all the time,” Muthukumarana added.
Why public trust is a must in data sharing
The issue of trust was also picked by Ben Lyons, the head of external affairs and insight at the Centre for Data Ethics and Innovation, who said there was “tenuous trust” from the UK public in government data sharing. Citizens are not particularly aware of how data about them is used and shared, he said, but the issue of trust isn’t often as well discussed as the other issues.
“Headline figures around public attitudes towards data sharing suggest reasonable levels of acceptance, but where this is potentially at risk is in situations where the public are unaware of the ways in which they might be operating, or become uncomfortable about it.”
This, in turn, creates an environment where there is uncertainty among public and civil servants about what types of data sharing is considered acceptable by citizens, he said. As a result, the data holders themselves are not necessarily willing to share data, even in cases where there is a compelling public benefit.
Stokes said that legal barriers are the most cited reason for data not to be shared “but there are usually legal ways to share data provided you can demonstrate there is value in doing so”.
Instead, picking up on what Allen had said earlier, the real barriers in Stokes’s opinion are organisational culture ones. “Those cultural barriers exist for good reason, because every public body that holds data on the general public has collected those data with the expressed condition that they will keep them secure. We all make that commitment when we bring data in, but that then drives a reluctance to share data because if something bad were to happen, you would be breaching that trust.
“It’s that cultural caution, and I would argue over-caution, that is the biggest blocker, and the only way really to unblock that, I think, and this is what we’re trying to do, is to show that it can be done without the sky falling in.”
Lyons said some examples of data sharing, such as the ONS’s work as described by Stokes earlier in the session, would have been difficult pre-pandemic, often for “a perceived legal risk as much as a legal risk itself”, but pandemic response had illustrated there was a real public benefit to them. “Clearly, as we move out of the immediate emergency response, there’s a need to ensure that we share data in a way that is trustworthy and responsible. It’s about setting an expectation that data can and should be used for the public benefit, but that needs to be done in the right way.”
Allen said the specific challenges to data sharing varies depending on the maturity of the organisation. “If you look at the ONS and the stuff that [Stokes] is doing, they’ve got the data fundamentals nailed, but for other government departments, that won’t be the case. Some of them will be starting at the basics – they might not even have a data catalogue and know the data they’ve got, let alone to be able to share it.”
Each organisation needs to understand its barriers, she said, which then informs the steps they can take to overcome them.
“Back when I was sharing those 2,000 data sets, I remember the barriers that people put up. Normally it was about culture, and it was really about fear. They would say people won’t understand my data or say there’s legal reasons, but it came back to fear most of the time. So I think it’s about really understanding when those barriers are real, and when they’re not.”
Stokes’s top tip to individuals looking to share data was to “start from a position of assuming that it’s possible, because we quite often assume that it isn’t”.
“If you have challenges reach out to those of us that have shown it’s possible because we want to help [and] the more we can make best use of data across the public sector, the better.
“But when you’re looking to make data available, remember that that you have to do it in a way that is not only within the bounds of the legislation, but it’s within the bounds of public acceptability. You have to do things in a way that looks like it’s in the public interest, as well as being in the public interest.”
Lyons agreed, urging people to “act in a way that builds trust”.
“What is the governance in what you’re doing that keeps people safe and their interests protected? That’s not just thinking about legal compliance – that’s really important but it’s also thinking through engagement with people who might be affected by the programme, and ensuring you’re giving citizens routes to influence the approach you’re taking, and being transparent about why and how you’re doing it.”
Muthukumarana added that public and civil servants should think about how to share data across whole networks, rather than just bilaterally between organisations. “In order to continue to enhance and improve public sector services, data sharing will not only be critical, it’s essential, and it will go on and it will evolve,” he said. “In the past, most data sharing has been bilateral between two organisations. But the future is multilateral – sharing data to be consumed across a network.
“So my guidance to professionals in the public sector is how can we drive public sector data sharing to that level – and be absolutely transparent and tell citizens exactly what you’re doing, and why.”
The Global Government Forum webinar ‘Joining up digital and data for better services’ was held on 12 May. You can watch the 75-minute webinar via our dedicated event page.
Like this story? Sign up to Global Government Forum’s email news notifications to receive the latest updates in your inbox.
About Richard Johnstone
Richard Johnstone is the executive editor of Global Government Forum, where he helps to produce editorial analysis and insight for the title’s audience of public servants around the world. Before joining GGF, he spent nearly five years at UK-based title Civil Service World, latterly as acting editor, and has worked in public policy journalism throughout his career.
Related Posts
