Enterprise AI is a renovation, not a repair job, says Dr. Jon Whittle

The former MD at the CSIRO’s Data61, Dr Jon Whittle, uses a construction metaphor to describe the renovations organisations need to make if they’re to gain value from AI. 

Whittle asks executives to picture an ugly 1970s brick house that needs a lot of attention: door handles come off, paint is peeling and the carpet is lifting.

He says you could spend months fixing each fault one at a time, but after all that effort, you would still be living in the same old house. 

“What you really need to do to transform that is to do a full reno. AI really is like that”, he says. 

Whittle says changing dozens of small, low-value use cases is not harmful in itself, but it will never change the nature of the organisation.  

He says real transformation means treating AI as a renovation of how a business works, rather than as a running repairs list in which licenses are rolled out, staff are trained, a few pilots are run and a growing catalogue of use cases is gathered from every corner of the business. 

Whittle, who is founder and CEO of Goldilocks AI and a professor at Melbourne Business School, spends much of his time helping organisations move enterprise AI out of pilot mode and towards something that delivers a return at scale. 

The failure pattern that Whittle sees most often is what he describes as ‘use case chaos.’

This is where teams fan out across an organisation, running around like headless chickens and collecting as many potential use cases as they can find. 

One of his clients amassed 500 potential use cases this way. 

“The problem with that approach is that it actually creates new problems. If you get 400 use cases, you can’t possibly implement all of them; many will be low-value use cases. So, you’ve then got a problem of asking: ‘How do you prioritise them and find the right resources to serve all of those use cases?” he says. 

Whittle also argues that leadership has not “sat down and really taken the time to articulate a very clear ambition for AI.” 

“I always try and work with my clients on that, and as a result, to focus on maybe just a very small number of high-value, well-tested, rigorous AI use cases, rather than this kind of scattergun approach that I think seems to be all too common out there”, he says.

Why most of the work is not technical

Whittle’s own research has found that roughly 80% of AI success is organisational rather than technical.  

“We’re three-and-a-half years into the kind of gen AI phase of AI and that means we’ve actually got some pretty good data now about what’s working and what’s not”, he says. 

He points to Stanford Digital Economy Lab’s Enterprise AI Playbook, which examined 51 companies that had successfully adopted AI. Stanford also found that 77% of factors determining success had nothing to do with technology. 

Whittle has combined that research with other studies and his own consulting experience to produce what he calls the 14 dimensions of AI adoption, grouped into three categories: leadership and vision, people and culture, and governance and trust. 

“They’re all the core ingredients of what any good transformation looks like. They’re things like…making sure you’ve got a proper process in place so you can actually scale pilots to enterprise scale with ‘stage gates’ and ways to invest as things scale up. 

What’s also important is mapping out and having a plan for dealing with ‘pockets of resistance’ from employees, making sure you are also accounting for the full cost of an AI rollout not just the cost of AI licenses and tokens. 

“The bulk of the cost is in the change management and the training that you have to do, the rethinking of roles…and making sure that you are rethinking your operating model and making changes there”, he says. 

Governance and trust sit in the same non-technical bucket and they’re not optional extras.

Whittle points to research from the University of Melbourne and KPMG, which found that Australia records some of the lowest levels of trust in AI globally, even as Australians rank first in the world for using tools like Claude on a per capital basis. 

Treat the rollout as a series of experiments

None of this makes budgeting any easier.

Token costs have fallen sharply, yet overall AI investment keeps climbing and the unpredictable nature of these systems makes it genuinely hard for IT teams and CFOs to plan. 

Whittle’s answer is to change the posture entirely. Rather than pretending to know the numbers up front, he tells clients to approach the rollout of AI as a series of experiments. 

“This technology is new. People are still figuring out how to apply it in practice. So, you’re not going to have all the answers up front because we don’t have the experience”, he says.  

Whittle says that each new use case should be framed as a hypothesis. 

“You think it’s going to save a certain amount of cost or improve productivity by a certain amount. But you’ve got to put tests in, you’ve got to monitor and find a way of measuring that and run an experiment so that within a few weeks or months, you can actually test whether that hypothesis holds.” 

The payoff from getting this right is not just discipline; it’s discovery. 

Whittle points to a large e-commerce retailer that ran seven generative AI use cases and A/B tested them across millions of customers, measuring the real lift in revenue each one delivered.

The results upended expectations. 

“The gen AI use cases that they thought were going to be the best were not actually the best. So, that really demonstrates the value of actually measuring and experimenting”, he says. 

That is the thread running through Whittle’s advice.

The organisations getting AI right are not those with the longest list of use cases or the biggest licence bill. 

They’re the ones that have named a clear ambition, accepted that most of the hard work is cultural and organisational rather than technical, and treated every rollout as a testable claim rather than an article of faith.

Understand how AI works

AI systems use probabilistic models that run on hardware and operating systems built for deterministic software.

Unlike traditional systems, AI calculates the likelihood of a particular outcome based on data patterns. 

Whittle says people need to make sure they understand how these systems work.  

“I think most people know by now that statistics is running the show. They know that LLMs are essentially next word predictors. But there’s a lot more going on than just that”, he says. 

Whittle says that LLMs involve reasoning and what looks like planning; they output what appears to be their thought process as they try to solve a problem.

“But that is still all statistical and this is perhaps something people don’t realise because…it [AI] looks like a human’s thought process. They assume that there’s some kind of clever planning algorithm behind as a series of steps [that] says, ‘do X, do Y, do Z.’ 

“But that’s not true. It’s still all statistical. It’s just that the LLMs have now been trained on examples where these reasoning traces are part of the training”, he says. 

There are parts of modern AI systems that are more deterministic, in what he says is typically called ‘the harness.’ 

“If you take Claude Code or Claude Cowork, for example, there’s the core AI model, but then there’s a lot of deterministic software around it that will make sure that what it’s doing is satisfying the right permissions, isn’t accessing things it shouldn’t access and things like that. 

“My main advice would be to educate yourself about how these things work, because then they’re not [viewed as] magic. Once you understand how something works, that gives you a lot of information about how you can control it and use it properly.”