Login
iscover
The Australian National Audit Office has found the ATO's governance arrangements for supporting the adoption of AI to be partly effective and outlined areas for improvement.
Newsletter
ANAO releases audit report into ATO's arrangements for AI
26 February 2025
•
9 minute read
An audit report into the governance of artificial intelligence at the ATO was published by the Australian National Audit Office (ANAO) this week, with the report making seven recommendations in total.
You’re out of free articles for this month
To continue reading the rest of this article, please log in.
Create free account to get unlimited news articles and more!
The report found the ATO had partly effective arrangements supporting the design, development and deployment of its AI models.
The ATO introduced a policy in December 2023 on the use of publicly available generative AI tools and developed an automation and AI strategy in October 2022.
However, the report said that the ATO had not established fit-for-purpose implementation arrangements for this strategy or defined enterprise-wide roles and responsibilities.
The ATO is also yet to establish AI-specific risk management arrangements and implement its data ethics framework sufficiently for AI.
"The ATO does not have specific policies and procedures for the design, development and deployment of its AI models, although there are enterprise policies and procedures which are relevant," the report said.
"The lack of approved and embedded policies and procedures creates risks to the effective implementation of models."
The report also found that the Tax Office had failed to sufficiently integrate ethical and legal considerations into its design and development of AI models.
"This impairs the ability of the ATO to demonstrate that its AI models are: fair and free from bias; reliable and safe; protecting privacy; transparent and explainable; contestable; and have appropriate accountability arrangements," it said.
ANAO also found no evidence of structured and regular monitoring of ATO-built AI models in production.
However, the report said this was being addressed through the development of a monitoring and reporting framework for the ATO's AI models.
The report noted that the ATO currently uses AI in a variety of contexts and plans to expand its use of AI over the coming years.
As of 14 May 2024, the ATO had 43 ATO-built AI models in production. ANAO found that around three quarters or 74 per cent of these ATO AI models did not have a completed data ethics assessment as of August 2024.
The report also found that the ATO did not have policies and procedures support the monitoring and evaluation of its in-house built AI models.
"For the 14 AI models built and deployed between 1 July 2023 and 14 May 2024, there was no evidence of ongoing performance monitoring and reporting," it said.
"For the work-related expenses AI models, there were some examples of monitoring and evaluation. Baselines were not reported in ongoing monitoring and reporting to show the impact of introducing the models."
In response to the ANAO findings, the ATO said it was adapting existing data management and data governance arrangements and introducing new arrangements to support its adoption of AI, including the support of support risk management and assessments of ethical considerations.
“We take our responsibility around data ethics and data stewardship extremely seriously,” the ATO said.
“The ATO only uses AI technologies in limited ways that recognise the importance of security, privacy, transparency and the ethical use of data with appropriate human oversight.”
The ANAO’s report provided seven recommendations agreed to in full by the ATO to support the commitment to managing taxpayer data with integrity and ensuring ethical decision making in all actions.
The ATO said the seven recommendations would assist the ATO in further strengthening its AI governance and aid in enhancing its AI frameworks.
The seven recommendations aimed at expanding the ATO’s policies and guidance to reference AI more included:
1. Improving alignment between the automation and AI strategy to effectively meet enterprise-wide program and project management requirements.
2. Clear definition and communication with enterprise-wide organisational structures and governance arrangements to support AI adoption. This would include defining accountabilities and responsibilities at the model and system level.
3. Review the ATO’s misuse of data and analytics enterprise risk and the associated controls and explicitly incorporate controls relating to the impact of AI on this risk.
4. Improvement of the ATO’s arrangements in support of the design, development, deployment and use of AI that aligned with ethical principles.
5. Progressing the development and implementation of AI-specific policies and guidance to support the effective design, development, deployment and assurance of AI models.
6. Establishment of performance measurement and evaluation arrangements for the ATO’s automation and AI strategy.
7. Ensuring the ATO’s approach to managing information supported transparency and accountability with respect to its adoption of AI.
The Tax Office said it appreciated the opportunity to assist the ANAO benchmark, its approach to conducting similar AI use and governance audits in the future, as well as providing insights to other Australian Public Service (APS) agencies.
“We acknowledge what is considered leading practice for AI use and governance is still evolving and we will continue to not only strive to achieve leading practice, but also assist the broader APS,” the ATO said.
“We will continue to implement and build on the recommendations identified to help us evolve and remain current in the face of rapidly advancing AI capability, and ensure we leverage AI in a safe way to create a better tax system for all Australians.”
You need to be a member to post comments. Become a member for free today!
