US agencies lack ‘comprehensive and accurate’ AI inventories, GAO report finds
Most major US federal agencies do not have clear and precise data on their current and planned use of artificial intelligence, the Government Accountability Office (GAO) has found.
By examining inventories submitted to the Office of Management and Budget (OMB) by 20 federal agencies, GAO – the auditing agency of the US federal government – said it aimed to discern departments’ “reported current and planned uses of AI”, whether their AI reporting was “comprehensive and accurate”, and to what extent each “complied with selected federal policy and guidance on AI”.
Each of the 20 agencies scrutinised hold what GAO called “government-wide roles in AI implementation”. They include the education, health, labour, justice, commerce, state, interior, energy, agriculture, transportation and treasury departments, NASA, and the OMB itself.
According to guidance drawn up by the Chief Information Officers (CIO) Council, agencies are required to identify AI use cases in their inventories, and include the relevant contact information, stage of life cycle and other details.
The inventories submitted by the 20 agencies include a total of 1,241 use cases. However, GAO found that only five of the agencies gave “comprehensive information” for each of their reported use cases, while the remaining 15 inventories contained “data gaps and inaccuracies”.
Read more: Governments set out plan to ensure AI safety as US publishes standards
“Thirteen [of the] agencies did not include required data elements that OMB requested for the initial inventory, such as life cycle stage, the office responsible for the AI use case, a relevant point of contact, or whether the use case should be withheld from the public inventory,” the GAO explained.
It added that nine agencies provided data on the life cycle of their AI use case that did not conform to the CIO Council’s template instructions. This accounted for 151 of the 1,241 use cases.
It was found that 277 use cases included in the inventories, such as those related to research and development, were outside the focus of review, while some departments – including the Department of Homeland Security and the State Department – included use cases that were “later determined to be not AI”.
GAO also reported eight “duplicative use cases” across four agencies – the Department of Commerce, the Department of Energy, NASA, and the U.S. Agency for International Development.
Three further agencies – the Department of Housing and Urban Development, the Nuclear Regulatory Commission, and the Small Business Administration – are described as having government-wide roles in AI implementation but reported no use cases for AI and therefore did not submit inventories to the OMB.
Accurate AI inventories ‘essential’
Reasons offered by agencies for the poor quality of their data included “errors made by staff, a difference in the interpretation of CIO Council instructions, and the anticipation of a change in life cycle stage”, the GAO said.
GAO stressed that accurate inventories for AI are essential tools for agencies to “plan and execute AI projects”, including projects based on “collaboration with [other] federal agencies”.
“Maintaining comprehensive and accurate AI use case inventories with quality information is critical for the government to have awareness of its AI capabilities and for agency leaders to make important decisions,” it said.
“Without an accurate inventory, the government’s implementation, oversight, and management of AI can be based on faulty data.”
The GAO has made 35 recommendations to 19 agencies. The main recommendation is that agencies “update [their] AI use case inventory to include all the required information, at minimum, and take… steps to ensure that the data in the inventory aligns with provided instructions”.
An executive order signed by President Joe Biden in December 2020 set out principles for promoting trustworthy AI in government. It directed agencies to “catalogue their AI use cases, and call… on the General Services Administration and the Office of Personnel Management to enhance AI implementation expertise at agencies”.
Read more: President Biden sets out blueprint for AI Bill of Rights
About Jack Aldane
Jack is a British journalist, cartoonist and podcaster. He graduated from Heythrop College London in 2009 with a BA in philosophy, before living and working in China for three years as a freelance reporter. After training in financial journalism at City University from 2013 to 2014, Jack worked at Bloomberg and Thomson Reuters before moving into editing magazines on global trade and development finance. Shortly after editing opinion writing for UnHerd, he joined the independent think tank ResPublica, where he led a media campaign to change the health and safety requirements around asbestos in UK public buildings. As host and producer of The Booking Club podcast – a conversation series featuring prominent authors and commentators at their favourite restaurants – Jack continues to engage today’s most distinguished thinkers on the biggest problems pertaining to ideology and power in the 21st century. He joined Global Government Forum as its Senior Staff Writer and Community Co-ordinator in 2021.
Related Posts
