National AI Centre flags delegation and oversight as top agentic AI concerns

The National AI Centre has identified human oversight, delegation boundaries and continuous risk assessment as the dominant concerns among Australian organisations grappling with agentic AI, following a series of information sessions and workshops held throughout September.

The government-backed centre consulted businesses, researchers, AI developers, government representatives and community organisations across Australia to help shape forthcoming resources on agentic AI and AI risk assessment.

Five key themes emerged from the engagement.

Organisations cited meaningful productivity gains from agentic AI - reducing repetitive work, supporting complex workflows and giving smaller organisations access to capabilities that previously required larger teams - but participants cautioned that benefits should not be assumed without building capability and implementing accountability structures first.

On the question of authority, participants drew a sharp distinction between genuinely agentic systems and other automated products being marketed as such. The centre said organisations evaluating agentic tools should ask what the system can access, what it can do, what authority it has been granted, and who remains accountable.

"It's a delegation problem, more than a technology problem," one participant said.

Human oversight was consistently emphasised, but participants challenged the assumption that placing a human in the loop is sufficient on its own.

Effective oversight, the discussions found, requires appropriate training, knowledge of both the business and the AI system, the ability to intervene, and adequate resourcing - combined with monitoring, logging, escalation pathways and clear approval thresholds.

Risk assessment was described as a living capability rather than a one-off exercise, given that AI models are updated, workflows change and permissions expand over time. Participants stressed the need to track what a system did, what information it accessed and what changed, in support of continuous governance.

The most consistent message across the sessions was demand for practical, accessible guidance - including worked examples, templates, checklists and vendor assessment questions - particularly for organisations without mature governance teams in place.

The National AI Centre said it would refine and test resources on agentic AI and AI risk assessment in the coming weeks.