Why Australia’s National Governor for the Global Council for Responsible AI Wants AI Leaders to Think About People First

For Nicki Doble, the newly appointed National Governor (Australia) for the Global Council for Responsible AI, her pivot towards AI governance wasn’t a slow burn; it was a single meeting that crystallised a growing unease.

Around 18 months ago, Nicki, who has spent more than 20 years steering organisations through complex transformations, found herself in a room where executives were excitedly calculating how admin staff could be shed by rolling out the Microsoft Copilot chatbot.

“I was sitting there and I thought, ‘We’re talking about a technology that has amazing opportunities for humanity [but introduces] enormous risk for humanity and we’re talking about letting low-level staff go’”, Nicki tells ADAPT.

Nicki decided that she didn’t want to be part of an excited conversation about making staff redundant to deliver what she describes as ‘measly’ cost savings.

“So I went, ‘I’m off the bus.’”

Bringing responsible AI governance to Australia

In her new role with the Global Council for Responsible AI, Nicki is tasked with scaling a global framework that has gained significant traction in the United States and the European Union but hasn’t taken hold in Australia.

Her brief is to translate lessons learned from overseas operations into something workable for Australian organisations and then take it to boards and executive teams.

“What I liked about the council was that it has a framework that is very adaptable so you can sit down, ask yourself the questions and turn it into a playbook”, she says.

This means having direct conversations with organisational leaders about whether their AI is actually doing good things within an organisation, not just ticking compliance boxes.

Nicki says she is finding that most AI-related conversations across Australian enterprises are very tech-led rather than focused on human impact and governance.

“I think that will start to change…partly because people will be going out and talking about it, like myself, and we will probably start seeing some of these cracks show in organisations where we might have issues when we’ve used AI. People are going to take a step back and say, ‘How are we doing this? How are we applying this?’ Because we don’t want to end up in the press over these sorts of things.”

Where AI orchestration breaks down

Nicki says AI success depends on correctly orchestrating governance, leadership, culture, and execution. She argues that this orchestration falls apart when program governance itself isn’t working, particularly when it’s not seen as part of a wider ecosystem. 

She points to a significant disconnect between tech architecture and business architecture, which are often moving at two different speeds.

“There’s been programs that I’ve worked on where there’s been massive KPIs around the use of agents or automation [tools], millions of dollars have been spent and there’s been no benefit to customer outcomes.”

Nicki recently spoke with one organisation planning an AI platform investment aimed at lifting productivity, with no corresponding examination of how its customer offering would actually change.

“It’s almost like that business architecture is too big to involve in the question of what they’re doing. They’re just focusing on the tech, and that’s not going to work because you are going to spend a lot of money and not have any change to the business.”

Patchwork AI regulation

Globally, governments are largely taking a light-touch approach to AI, including Australia, which Nicki describes as ‘patchwork’ and in need of fixing.

“Countries can roll out their AI Acts…but if they’re not actually enforcing them, they’re useless. We know that there are countries that have got them [laws], but they’re not enforced.”

Nicki draws a direct comparison to the trajectory of cyber security and privacy regulation: organisations didn’t find the money or the will to act properly until they were compelled to. She expects the same dynamic to play out with AI.

A regulatory vacuum, she argues, creates an open door for overseas investment and influence that may not be in Australia’s or the public’s interest.

“The trillionaires and the billionaires aren’t going to do it [regulate AI] for us”, she says. 

Robodebt and the accountability gap

When it comes to the human consequences of getting AI or automation deployments wrong, Nicki cites the Australian government’s Robodebt scandal as a cautionary tale.

The automated debt recovery system caused genuine harm, including deaths, yet accountability was diffuse and hard to pin down.

“There was no accountability as to who was to blame.  That was a system that if you had said, ‘I’s actually going to cause people death and destruction, you would have said ‘no it’s not.’ [But] it did. We blamed an algorithm, we didn’t hold people accountable, and when we say, ‘human-in-the-loop’, there was an opportunity for people to intervene. People tried to but weren’t heard.

“What I take from that is [during] AI conversations, we keep getting told to trust the tech because we’ve got humans in the loop. But we’re not having very good discussions about what does a human in the loop look like? Are we going to make people fill in three-page reports when they see something that doesn’t look right?” she asks.

Nicki adds that organisations are retrenching business analysts and human-centred designers because they believe that Anthropic’s Claude LLM can complete those requirements.

“That to me sounds counterintuitive; if you’re trying to convince me that you’re [including] humans in the loop, you’re taking that out of the process in the beginning.

She says this same tension is playing out in healthcare, where medical professionals with deep subject-matter expertise are being told to simply trust the technology.