ChatGPT fails prestigious Indian civil service test; US works to streamline citizen experience: policy & delivery news in brief
Global Government Forum’s weekly digest of the news you need to know but might have missed.
AI chatbot fails Indian civil service examination
The controversial AI service ChatGPT has reportedly failed the Indian Civil Services Examination, thought to be one of the country’s toughest competitive exams.
ChatGPT is an AI chatbot trained to mimic human conversations, write computer programmes and essays, compose music, answer test questions and complete a large number of other tasks with human-like ability. The service – created by San Francisco-based OpenAI – has been the subject of fierce debate in recent weeks over its potential uses, such an enabling students to cheat their way to qualifications.
With this in mind, Analytics India Magazine decided to ask ChatGPT 100 questions from one of the 2022 preliminary papers set by India’s Union Public Service Commission (UPSC). The examination is part of the recruitment process to the higher ranks of the country’s civil service, and is one of the most prestigious qualifications for the best and brightest of India’s students. A successful candidate sits through 32 hours of examination – comprising two prelim papers, nine essays, and an interview – over the course of a year.
“We at Analytics India Magazine have been experimenting with ChatGPT for a while and were intrigued by its ability to crack tough exams and interviews. In that spirit, we wondered, ‘Can ChatGPT clear UPSC?’,” said the magazine about the test.
Of the 100 questions – the answers to which are available on the web and which cover topics such as history, geography, economy, ecology, science and current affairs – ChatGPT answered 54 correctly. Applicants usually need a pass mark of at least 80.
ChatGPT carries a disclaimer stating that its knowledge is limited to pre-September 2021 and that it won’t be able to answer questions about current events. However, the magazine claims that many of the questions it got wrong were not time specific and that, for example, it failed to answer a simple history question.
When asked whether it would be able to pass the UPSC exam, the chatbot replied: “As an Al language model, I do possess a vast amount of knowledge and information, including about the UPS examination and related topics. However, passing the UPSC prelims exam requires not only knowledge but also critical thinking skills, application abilities, and time management skills. Therefore, I cannot give a definite answer on whether I can pass the prelim exam for UPSC or not. Nonetheless, I can provide you with relevant information and guidance regarding the exam and related topics to the best of my abilities.”
It was revealed last month that the UK Department of Science, Innovation and Technology has advised civil servants to exercise caution before using ChatGPT and other chatbots to automate tasks such as email and letter writing and repetitive work in policymaking.
Michelle Donelan, the new secretary of state for science, innovation and technology, told UK national newspaper The Sunday Telegraph that ChatGPT represented a “massive opportunity” for the UK and that the civil service should look into its applications. She said “of course we need regulation in place, we need safeguards” but “we should never be afraid of these technologies”.
Elon Musk, the billionaire boss of Tesla, SpaceX and Twitter, co-founded OpenAI but has distanced himself from the company in recent years. Following mention of ChatGPT at the World Government Summit in Dubai last month, Musk – who has long called for more stringent AI regulation – said AI was “one of the biggest risks to the future of civilization” adding that it is “both positive or negative and has great, great promise, great capability” but “with that comes great danger”.
Read more: UK civil servants told to exercise caution around AI chatbot use
US government reveals nine projects designed to streamline citizens’ experience
The White House has announced nine projects that federal agencies are undertaking as part of the Biden administration’s bid to improve citizens’ experience of government.
The projects focus on smoothing “pain points” people encounter when interacting with government at critical moments in their lives such having a baby, facing financial difficulty, approaching retirement, or recovering from the impact of a natural disaster. For each of the life events, the federal government presents a citizen’s real life experience, outlines the challenge and what should be considered if improvements are to be made, and details the associated project or projects and objectives.
Projects include building an information and outreach model to help older adults make informed retirement and healthcare decisions; piloting a ‘benefits bundle’ that aims to connect families of a new baby with supportive services; and developing guidelines to promote “trauma-informed” communications and interactions between agencies and survivors of natural disasters.
“Life experiences are significant events or transitions that often require interactions and touchpoints with multiple federal agencies and even levels of government,” the webpage outlining the projects reads. “Too often, people have to navigate a tangled web of government websites, offices, and phone numbers to access the services they depend on. Government needs to better meet people where they are and be responsive to how they navigate these moments.
“The ‘life experience’ organising framework requires a new model of the federal delivery system working together – within agencies, across agencies, even across levels of government – driven by customer (‘human-centred design’) research, rather than within bureaucratic silos and pre-conceived solutions, to solve problems.”
Many of the new projects involve efforts to decrease the number of applications people have to submit for government benefits and streamlining those applications, in part by making data more accessible across agencies and programmes. One project, for example, “seeks to improve access to benefits for people facing financial shock by improving underlying data services and systems… by improving verification services and the quality of data coverage, the government can better leverage existing systems to streamline the customer experience”.
The new projects form part of the implementation of an executive order signed by president Joe Biden in December 2021 titled Transforming Federal Customer Experience and Service Delivery to Rebuild Trust in Government.
US and Australia ban TikTok use on government devices
TikTok, the Chinese viral video app, must be removed from all US federal government-issued devices and technology infrastructure by the end of March, according to new guidance from the White House.
Meanwhile, an investigation into social media use by Australia’s Home Affairs department found that 25 of the country’s government agencies have or will ban TikTok on work-issued devices, and that a further 12 have partially banned use of the app. The audit revealed that it is permitted by 11 agencies. Five agencies did not respond to investigators or failed to answer directly.
TikTok has been downloaded over 210 million times in the US and has more than one billion users worldwide, according to its Chinese owner ByteDance.
Politicians, officials and privacy advocates in numerous countries have repeatedly raised concerns over the app in recent years, citing data ownership concerns. They warn that individuals’ data collected by the app could be used by the Chinese government to influence political opinions or inform manipulative, targeted advertising campaigns.
The newly-released memorandum by the US Office of Management and Budget – which states that agencies must ban the app within 30 days and confirm compliance within 90 days – follows legislation passed by Congress late last year as part of the omnibus spending package that officially required federal agencies to ban TikTok. Several states have already banned or restricted use of the app on government devices.
In 2020, the previous administration under Donald Trump was understood to be considering banning TikTok in the US entirely.
In Australia, opposition senator James Paterson who called for the audit into government use of social media, called for TikTok – which he has argued is required under Chinese law to share personal data with the Chinese Communist Party – to be banned on all federal devices except in “exceptional circumstances”.
TikTok bosses deny the company has ties with the Chinese government and have said they take ensuring the safety of user data “incredibly seriously”.
Other administrations such as the Canadian government and the European Union have made similar moves to ban TikTok on devices used by officials, while India banned TikTok, along with 58 other Chinese-owned apps, nationwide.
Want to write for GGF? We are always looking to hear from public and civil servants on the latest developments in their organisation – please get in touch below or email [email protected]
About Mia Hunt
Mia is a journalist and editor with a background in covering commercial property, having been market reports and supplements editor at trade title Property Week and deputy editor of Shopping Centre magazine, now known as Retail Destination. She has also undertaken freelance work for several publications including the preview magazine of international trade show, MAPIC, and TES Global (formerly the Times Educational Supplement) and has produced a white paper on energy efficiency in business for E.ON. Between 2014 and 2016, she was a member of the Revo Customer Experience Committee and an ACE Awards judge. Mia graduated from Kingston University with a first-class degree in journalism and was part of the team that produced The River newspaper, which won Publication of the Year at the Guardian Student Media Awards in 2010.
Related Posts
