Deloitte's AI Report: Fabricated Sources and Refund

What Deloitte was hired to review

Australia's Department of Employment and Workplace Relations (DEWR) commissioned Deloitte in December 2024 to run an independent assurance review of the Targeted Compliance Framework, the IT system that automatically penalizes welfare recipients who miss job-search appointments or other mutual obligation requirements. The contract was worth roughly AU$440,000 (about US$290,000). Deloitte delivered the report and DEWR published it on the department's website in mid-2025. [2][1]

The review covered whether the Framework's automated penalty system operated in line with government policy, business rules and the law. That subject matter carried weight beyond the contract's size: Australia's prior experience automating welfare compliance was the Robodebt scheme, which wrongly raised debts against hundreds of thousands of people using unlawful income-averaging and was later the subject of a royal commission. A review meant to reassure the public that the successor system was sound instead became a case study in unverified AI output. [1]

What was made up, and how it was found

Dr. Christopher Rudge, deputy director of the University of Sydney's Sydney Health Law research centre and a law lecturer specializing in health and welfare law, read the published report and found citations that did not check out. The report referenced a book, "The Rule of Law and Administrative Justice in the Welfare State," roughly a dozen times; the book does not exist. It also attributed nonexistent papers to real academics, including University of Sydney law professor Lisa Burton Crawford and Lund University's Björn Regnell, on subjects outside their actual published work. [1][3]

The report also quoted a Federal Court judge who never said what it claimed. It attributed a passage, described as coming from paragraphs 25 and 26 of the robodebt-related judgment Deanna Amato v Commonwealth [2019] FCA 2031, to "Justice Davis" (misspelling the judge's actual name, Jennifer Davies). The judgment contains no such passage; the quote and the paragraph numbers were invented. Rudge, who flagged the errors and brought them to journalists, called Deloitte's later disclosure of AI use a "confession" and argued the underlying methodology could not be trusted regardless of whether the recommendations happened to be sound: "You cannot trust the recommendations when the very foundation of the report is built on a flawed, originally undisclosed, and non-expert methodology." [1][3]

What Deloitte admitted, and what it didn't

After the errors became public in early October 2025, DEWR republished a corrected version of the report. Its updated Statement of Assurance disclosed that Deloitte had used "a generative AI large language model (Azure OpenAI GPT-4o) based tool chain," licensed by DEWR and hosted on DEWR's own Azure tenancy, to help address what it described as "traceability and documentation gaps." The correction stripped out the fabricated judge's quote and more than a dozen bogus references and footnotes, and fixed other errors. [2][1]

Atlas interpretation: Deloitte's admission was narrower than it might first appear. The firm and DEWR both maintained that the corrections did not change the report's substance, findings or recommendations, and Deloitte's internal review reportedly attributed the errors to human error in checking the AI's output rather than to a failure of the AI tool itself. That framing lets Deloitte concede the tool was used without conceding that using it, undisclosed, on a government report about a system built to avoid another Robodebt was itself the failure. [1]

Deloitte agreed to refund the final installment of the contract: AU$97,587, government officials confirmed at Senate estimates, out of the roughly AU$440,000 total. That is about a fifth of the contract value, not the full amount and not a fee tied to the specific hours spent on the flawed sections. [4]

The political reaction

Australian Greens Senator Barbara Pocock raised the report at Senate estimates and pressed the Finance Department on the size of the refund. She argued Deloitte should return the entire AU$440,000, not just the final installment, and compared the errors to work that would fail a first-year university assignment. On the underlying practice of a paid consultant using an AI tool on government work without disclosure, she said: "If the contractors outsourced to do government work are further outsourcing that work to AI, at the bare minimum, you'd expect this work to be checked." [4]

Atlas interpretation: What makes this case useful beyond Australian politics is how ordinary the failure was. Nothing about generating a compliance review with a language model is unusual; the gap was that nobody checked the citations against real sources before a government client received the final document. A fabricated academic reference and a fabricated judicial quote are exactly the kind of error a single verification pass, checking that each cited work and each quoted passage actually exists, would have caught. That check did not happen before publication; it happened afterward, in public, because an outside academic ran it himself. [2][1]