A license to think afresh: how the Data Challenge empowered civil service innovators
Under the UK’s Civil Service Data Challenge, staff were invited to send in their ideas for how to use data better in government. Matt Ross profiles a programme that drew on the expertise and enthusiasm of civil servants across the workforce – and put the best ideas on the road to implementation
A year ago, Damien Hicks and his colleagues faced a problem that seemed insurmountable. A data modeller in Natural England’s Statistics and Modelling Service, Hicks was involved in peatlands restoration projects – an important plank of efforts to combat climate change. But the work required to hit the government’s 2018 Peatlands Strategy target – to restore two million hectares of the degraded habitat by 2040 – looked unaffordable.
Healthy peatlands have the highest carbon density of any ecosystem in the world, storing twice as much carbon as woodlands. But over the centuries, many have been cut through with drainage channels; and as the peat dries out, it releases vast quantities of CO2. According to the British Ecological Society, rather than absorbing carbon, the UK’s peatlands are currently emitting 23m tonnes every year. These drains – known as ‘grips’ – also degrade plant and wildlife habitats, and heighten flood risks by weakening peatlands’ sponge-like ability to retain water.
Restoration work involves damming the grips and planting sphagnum moss; there is no short cut around this laborious work. But before grips can be blocked, they must be located – a task currently requiring arduous fieldwork in remote and inhospitable environments. Hicks and his colleagues believed they had a way to dramatically simplify this task.
Organic creativity, artificial intelligence
Artificial Intelligence (AI) systems known as Convolutional Neural Networks can identify grips from aerial photography, but first they must be ‘trained’ on labelled images – and Natural England didn’t have enough genuine photographs to train them. This problem can be overcome using Generative Adversarial Networks (GANs): another form of AI that can produce large numbers of artificial images, indistinguishable from the real ones. Hicks’s team, however, lacked the funding, time, expertise or resources to develop suitable GANs and thus build an AI system able to spot out the grips.
Then the Civil Service Data Challenge was launched – providing a route through Hicks’s conundrum. Run by the Cabinet Office, the Office for National Statistics, Global Government Forum (GGF) and technology firm NTT DATA UK, the programme aimed to first identify good ideas for making better use of government data, then to provide the skills, time, support and profile required to pursue them.
“Nobody knows the challenges in an organisation like the people who work with the detail every day; often these people have great ideas for solving problems,” explained then-Cabinet Office minister of state Lord Agnew at the Challenge’s December Final. “Government has great data assets, and civil servants with the capability, expertise and imagination to make the best use of them.” But staff with plans for realising the potential of government’s data require both help to test, research and develop their ideas, and ways to get them in front of the senior managers who can put them into action.
Alex Chisholm invites ideas
In March 2021 Alex Chisholm, the Cabinet Office permanent secretary and civil service chief operating officer, invited civil servants across government to send in their ideas for how government could make better use of data. Nearly 200 people did so – among them Damian Hicks, who put forward his peatlands GANs project.
Programme managers trawled through the entries, consulting with relevant departments on the stronger ideas – about a quarter of the total. Then all this information went to the judges: seven senior digital leaders, including Joanna Davinson, head of the Cabinet Office’s Central Digital and Data Office; deputy national statistician Alison Pritchard; and NTT DATA UK’s head of public sector, Vicki Chauhan. Drawing on the judging criteria, the judging panel picked out the eight most promising concepts; and when this longlist was announced, Hicks was delighted to find that his submission had made the grade.
Browsing through the data provided by nearly 300 volunteers, the organisers – a joint NTT DATA/GGF team – built suitable teams around each of these eight ideas. Importantly, they also provided each team with an expert ‘facilitator’: an NTT DATA consultant able to provide advice and support. And they included in each team people with the specific skills, knowledge and connections required to develop their idea.
Cross-government collaboration
Building cross-departmental, interdisciplinary teams was a core aspect of the Challenge concept, and it soon paid off. In the case of the peatlands idea, recalls team facilitator Henrietta Marsh-Smith – an NTT DATA UK delivery director – the organisers brought in scientific, environmental and digital specialists whose expertise complemented Hicks’ peatlands knowledge. “Damian came in with an idea based on his skills and experience. But suddenly we had nine other people in the room who had completely different experience and could really enhance the idea,” she says. As a result, “the solution became much more fitting to the problem.”
Including staff from a wide range of departments also helped identify other fields in which the new GANs could be valuable, says Marsh-Smith – enabling the team to show how their concept met the judging criteria that it be ‘replicable and scalable, with potential applications across government’. “Bringing together expertise and enthusiasm from different bits of the civil service has worked wonders to make this idea shine,” Hicks’s manager Mary Vayou, Natural England’s principal adviser on data science, told GGF.
That was certainly the experience of Tom Duffield of the Foreign, Commonwealth and Development Office, whose idea – to build a real-time data dashboard providing estimates of the number of UK citizens around the world, aiding disaster response – made it all the way to the final.
“The idea needed proper cross-government collaboration,” Duffield told GGF, noting that “within our day jobs, it’s very difficult to motivate other government departments to tackle an issue that isn’t necessarily affecting them”. With members drawn from all the key departments, Duffield’s team was able “to get them interested… It’s been a fantastic experience, stepping outside of our own departments and working with one another”.
So the teams’ interdisciplinary, cross-departmental nature was important to pursuing and developing their ideas – but it was just as valuable for the team members themselves, helping them to develop their skills and make new contacts. As Agnew pointed out, “for the Data Challenge team members, the experience of working closely with people from a range of disciplines – and, in many cases, in an environment where the usual grade hierarchy has been put to one side – has been a huge benefit”.
Team efforts
With support and advice from NTT DATA UK, the teams spent the summer and autumn researching and developing their ideas. And when they ran into problems, they could reach out for support from the judges or the programme’s ‘champions’: these very senior figures – including Alex Chisholm and the permanent secretaries leading the policy and operational delivery professions, Tamara Finkelstein and Peter Schofield respectively – played crucial roles in both promoting the Challenge, and engaging departments with the successful ideas.
In Duffield’s case, he recalled, his team was struggling to broker data-sharing agreements with relevant departments – but judges Alison Pritchard and Joanna Davinson are “at the forefront of driving the standardisation of data-sharing agreements” and were able to help out. With this kind of support available, said Duffield, the Challenge represented “the perfect opportunity to push this solution forward”.
In October, the eight teams met their first big test: facing the judging panel at the ‘Dragon’s Den’-style semi-final, they pitched their ideas and tackled the judges’ questions. Just four would go through to the final – but the judges were impressed by all eight teams, and pledged to mentor and support the other four outside the formal programme.
“The judges have already said that no matter what happens, they’re going to do what they can to help, because the truth is that all of these ideas – whether they get through to the final or not – are things that we should be doing,” said judge Henry de Zoete, a digital entrepreneur and Cabinet Office non-executive director. “They’re all valid and fantastic and something should be done for them – and these are all ideas that we wouldn’t know about if it wasn’t for the Challenge.”
The final four
Both Tom Duffield’s citizens overseas database and Damien Hicks’s peatlands project made it through, alongside two other ideas. One, put forward by Aaron Lohan, proposed creating a new data link between the Department of Work and Pensions (DWP) and HM Revenue and Customs (HMRC) to automatically cross-check people’s benefits claims against their earnings from self-employed construction jobs. “I was working in Salford job centre, and I was noticing that for self-employed claimants there can be discrepancies in what they were declaring [to DWP and HMRC] – and there’s no mechanism for checking that at the moment,” he explained. “When I heard about the Data Challenge, I developed the idea and wrote the entry.”
The fourth idea involved monitoring prisoners’ social contacts with friends and family: evidence shows that prisoners without strong support networks are more likely to reoffend, and gathering this data would enable prison staff to provide them with additional help and support.
After the semi-final, the four remaining teams had two months to further research and develop their ideas – this time with the assistance of NTT DATA digital specialists, who helped them to build demonstration models and proofs of concept. And on 9 December they returned to face the judges – whose ranks now included Alex Chisholm and Tamara Finkelstein – at a swish event in London’s County Hall. Accompanied by presentations including animations, visualisations and videos, they each had half an hour to pitch and answer questions.
Returning after the judges’ deliberation, their chair Vicki Chauhan summed up their feelings about what they’d heard. “It’s been amazing to see the professionalism and commitment and passion that you’ve had for your ideas, while working remotely over the last six months – and, of course, doing all this in the margins of your day jobs. It is just a massive achievement,” she said. The Challenge has “allowed us all to show that working collaboratively can create innovative ideas that are going to make such a difference to public services”.
Like de Zoet, Alex Chisholm was keen that all eight shortlisted ideas be implemented. “Every single one of those proposals we absolutely loved,” he said. “We want all of them to go forward; every one of the shortlisted ones… we are absolutely determined that they should happen.”
A winning idea
Then came the moment of truth. Announcing the peatlands project as the winner, Chisholm noted that their plans would generate AI capabilities with broad applications and a one-year delivery timescale. The judges were also impressed by the potential to create valuable intellectual property, the scheme’s contribution to the UK’s net zero goals, the opportunity to reduce flood risk, and the very positive forecasts for return on investment – with a £288,000 (US$380,000) investment averting the need for manual work costing £6m (US$8m).
The project will now benefit from £50,000 (US$68,000) worth of development work provided by NTT DATA UK – providing a substantial chunk of the investment required. And with Tamara Finkelstein among the programme’s champions, the prospects for delivery look good: alongside her role as head of the policy profession, she is permanent secretary of the Department for the Environment, Food and Rural Affairs (Defra) – whose responsibilities include the health of the peatlands and oversight of Natural England.
Speaking after the announcement, Hicks credited the programme with turning a good idea into a live implementation programme. When he sent in his application, “we had no thought of getting this far. It was a desire to try it, see how the Data Challenge works, and hopefully to get involved in a team across the civil service,” he recalled. The very concept of using GANs to train a peatlands drainage-spotting AI system “was seen as a vague possibility, but thought to never be actually possible to put into practice, because there’s only a handful of people [working on the agenda] in Natural England. But it gathered so much more momentum when the rest of the team got involved”.
Listening to staff – and acting
For Finkelstein, the programme has had yet another benefit: promoting the Challenge to Defra’s workforce, her leadership team has gathered in loads of ideas. “We’re going to take forward the whole lot that came from the department, and we can use that to drive the theme around data,” she said.
As Lord Agnew said: “Innovation isn’t something that works top down. Instead, it should be built on the imagination, commitment, expertise and confidence of our staff. This only happens when people know that their ideas will be heard and taken seriously.” Here, the Data Challenge should give civil servants confidence: within a year, it has transformed Damian Hicks’s “vague possibility” into a live delivery programme, armed with careful research and the resources to begin development work.
The Challenge “gave a way in which people could, bottom-up, think about ideas innovatively, form teams across departments and groups, and get excited about an area [better use of data] that’s totally critical to transformation of the civil service,” concluded Finkelstein. “It provided a vehicle for people to do things that need to happen. It’s such an interesting way to do it, and it brought out the best in people.”
Watch the four finalists give their presentations and answer the judges’ questions via the Data Challenge website https://www.datachallenge.uk/, where you can also learn more about the Challenge. Additional reporting by Richard Johnstone.
Like this article? Sign up for Global Government Forum’s email news notifications
About Matt Ross
Matt is Global Government Forum's Contributing Editor, providing direction and support on topics, products and audience interests across GGF’s editorial, events and research operations. He has been a journalist and editor since 1995, beginning in motoring and travel journalism – and combining the two in a 30-month, 30-country 4x4 expedition funded by magazine photo-journalism. Between 2002 and 2008 he was Features Editor of Haymarket news magazine Regeneration & Renewal, covering urban regeneration, economic growth and community development; and from 2008 to 2014 he was the Editor of UK magazine and website Civil Service World, then Editorial Director for Public Sector – both at political publishing house Dods. He has also worked as Director of Communications at think tank the Institute for Government.
Related Posts
