The UK government is sitting on a user research goldmine. So, why isn’t it being used?

Image by vectorjuice on Freepik

Despite almost drowning in rich user data, poor knowledge sharing across government is damaging productivity and driving up mental load. Dr Jon Rimmer, chief experience officer at Mercator Digital, discusses how technology, change management and upskilling could help – without a complete systems overhaul

For many years now, UK government departments have been sitting on a goldmine of user research that could not only help build new and transformative digital services for the general public, but could do so at great pace.

And yet, archaic processes are making knowledge hard to access, and the most valuable insights remain buried. The National Audit Office’s latest review of the government’s Shared Services Strategy, which aims to improve data sharing across departments, tells us this is an ongoing problem. Despite efforts stretching back to the early 2000s, departments are still failing to standardise, share or join up their data, with the NAO’s report pointing to governance issues, interoperability and inconsistent departmental commitment.

This means civil servants have to resort to time-consuming workarounds: ‘asking around’, laboriously combing through multiple PDFs and free-text documents, duplicating research, or second-guessing decisions without full context. And when effort is spent navigating fragmented systems rather than solving problems, productivity also begins to suffer and is felt daily in workload and stress levels. This leads to mistakes being repeated and opportunities missed, and confidence in the system itself begins to falter.

On the surface, this might look like a simple data problem. But this isn’t about a lack of information: it’s about the effort required to find and trust it.

Why more technology alone won’t fix this

It’s tempting to assume that more advanced technological tools – particularly artificial intelligence – will solve this issue in its entirety. But if the solution were purely technical, this problem would have been solved many years ago. Yes, technology can help, but it will only ever be as effective as the data and behaviours that support it. If the data going in is poor, the output will be too. As such, there’s still a long way to go before governments can rely on these systems at scale, and the idea that we are already operating in some kind of AI-driven ‘nirvana’ simply doesn’t hold up.

The real challenge isn’t the technology itself, but how people respond to it. Some of the most common behaviour-related barriers include data often being treated as something to protect rather than share, even within government. Not because departments are actively competing, but because sharing is perceived to introduce risk. If something goes wrong downstream, accountability doesn’t disappear and that creates understandable caution.

There’s also inertia. Many processes have been built and refined over years, decades even. Changing them isn’t just about introducing new tools, it means rethinking how work gets done – something that is rarely straightforward. And this is where scale makes these behaviours harder to shift. In large departments, with thousands of people and deeply embedded legacy systems, even small changes can feel risky. The bigger and more complex the organisation, the easier it is for caution to win out over collaboration.

In addition, there is the fear that by using technology like AI, we risk making ourselves lazier. But this misunderstands what the problem actually is. Think of how we used to remember dozens of phone numbers by heart. Now, most people couldn’t tell you more than one – that’s not a failure, offloading trivial tasks is the point. And it’s the same with AI applied to knowledge sharing. The goal has never been to remove thinking, but to remove the low-value effort that gets in the way of it.

What ‘good’ looks like in practice

In spite of the above challenges, however, there are examples of what progress can look like. In UK government trials, giving civil servants access to AI tools has already delivered measurable gains, with staff saving around half an hour a day on average.With the government’s current push to increase productivity, these measurable outcomes will be welcomed. Shaving an hour or two off administrative work might sound small fish, but across thousands of employees it quickly adds up. More importantly, it shows how targeted interventions, as opposed to a complete transformation, can start to unlock value bit by bit.

The question then becomes: what happens to that freed-up time? If it’s simply filled with more tasks, the benefit is lost. But if it reduces cognitive overload, it can improve the quality of work as well as the pace.

There’s also a longer-term consideration. Foundational tasks – the kind often handed to junior staff – play a critical role in building expertise. If those are removed entirely, where does that experience come from? In a knowledge-sharing context, this matters especially: the work of finding, reading and synthesising past research is part of how civil servants develop judgment about what good evidence looks like. As such, any productivity gains must be balanced with capability building.

As well as small, tech-enabled changes, we must also ensure there is confidence to share and the space to experiment. That means sharing case studies – not just the successes, but also the things that didn’t go to plan, which can be just as valuable as celebrating what went right. There also needs to be environments where teams can test new approaches without fear of failure, which is particularly important when the goal is to build a culture of openness around data. Any AI deployed in this context must also operate in a ‘walled garden’, trained only on internal content. This addresses fears around data protection and trust, and also ensures recommendations and outputs are based on real service data and not biased public internet content.

Upskilling is another critical piece. As AI becomes more embedded, people need to understand how to use it effectively – and where human judgement still matters. That requires investment at scale, particularly in large organisations.

Last but not least, it’s about balance. In areas like user research, the shift towards remote methods has made things more efficient, but not always better. There’s still real value in being out in the field, observing behaviour first-hand and engaging directly with people. As many have said before, technology should support that, not replace it.

Making better use of what’s already in front of us

The bottom line here is that the UK government doesn’t have a shortage of data; it has a challenge in using what it already knows.

Technology can help, but it won’t solve the problem on its own and a complete systems overhaul is certainly not warranted – or indeed practical. What we need is a cultural shift – towards sharing, learning and continuous improvement – or the same barriers will remain.

Get that right, and the impact moves well beyond internal productivity, to better decisions, better services, and ultimately better outcomes for the public.

Author

Jon Rimmer is the chief experience officer for Mercator Digital. He has been immersed in research and design since the early ’90s and still loves uncovering insights that shape products, services, and brands. His work spans user, customer, and audience research across diverse sectors, including government, software, retail, finance, health, education, engineering, media, and gaming. He specialises in user-centred design, user experience and customer experience research, brand experience, and innovation – using methods like usability testing, eye tracking, ethnography, interviews, workshops, and prototyping to refine digital and physical experiences. Jon has collaborated with leading names such as Microsoft, Amazon, NHS, HMRC, American Express, William Hill, and Government Digital Service.