AI Hallucinations in Government Documents Are Becoming a Contractor Risk
A recent article by Ananya Bhattacharya in Rest of World should be read carefully by federal contractors that use generative artificial intelligence to support research, policy analysis, proposal development, consulting deliverables, technical reports, or contract performance. The article describes several examples in which AI-generated errors, fabricated citations, or unsupported references appeared in government or government-commissioned documents. These incidents are not merely embarrassing. They illustrate a developing procurement and contract-performance risk: when contractors use AI without verification controls, the resulting work product may compromise trust, quality, and accountability.
The most striking example involved South Africa’s Draft National Artificial Intelligence Policy, which reportedly was withdrawn 17 days after publication because it cited fake research. According to the article, South Africa’s minister of communications and digital technologies stated that the most plausible explanation was that AI-generated citations had been included without proper verification and that “consequence management” would follow for those responsible for drafting and quality assurance.
The article also identifies incidents involving Deloitte reports prepared for government clients in Australia and Canada. In Australia, Deloitte reportedly confirmed that generative AI tools had resulted in inaccurate outputs, including citation and reference-list errors, in a report commissioned by the Department of Employment and Workplace Relations. The article states that Deloitte later republished the corrected study and refunded part of the government’s payment. In Canada, the article reports that a Deloitte healthcare report for Newfoundland and Labrador included fake citations, after which the government reportedly updated its procurement language to require disclosure of intended AI or machine learning uses and reserved the right to assess AI-related risks before or after award.
For federal contractors, the lesson is direct. AI use is no longer simply an internal productivity issue. It can affect contractual quality, professional responsibility, invoice support, representations to the government, data integrity, and the credibility of contractor work product. If a contractor submits a report, policy analysis, research memorandum, technical recommendation, market study, or proposal section that contains hallucinated sources, false quotations, or unsupported factual claims, the issue is not that the contractor used AI. The issue is that the contractor delivered unverified work.
The risk is especially acute in knowledge-based services. Management consulting, legal support, grant support, cybersecurity advisory services, acquisition support, program evaluation, scientific research, and policy analysis all depend on the reliability of cited sources and analytical reasoning. In those settings, citations are not decorative. They are part of the evidence chain. Once that chain is corrupted, the government may question the entire deliverable.
Contractors should respond by creating practical AI controls. Those controls should require human review of all AI-assisted deliverables, independent verification of citations, preservation of source materials, disclosure where required, and clear internal rules identifying when AI may and may not be used. Contractors should also train employees that AI-generated references, quotations, legal authorities, technical standards, and factual claims are not reliable until verified against primary sources.
The procurement implication is equally important. Agencies may begin asking vendors to disclose AI use, certify human review, document quality assurance, or accept contractual responsibility for AI-assisted errors. Contractors that develop these controls now will be better positioned than those that wait until a solicitation, audit, or customer complaint exposes the weakness.
The central point is not that contractors should avoid AI. The point is that contractors should not outsource truth to AI. In federal contracting, accuracy is not optional. It is part of performance.
Disclaimer
This post is for informational purposes only and does not constitute legal advice. The discussion is based on publicly reported incidents and general observations about AI use, procurement risk, and contractor quality assurance. Contractors should consult qualified counsel or appropriate advisors before adopting AI policies, contract language, disclosure practices, or compliance controls.