AI governance and risk can’t default to the CISO alone, says LaunchPad Co-Founder

Many AI projects are not quite delivering what is promised and some have even been abandoned. 

What is clear, however, is that the secure and successful adoption of AI requires clear ownership and governance.

In the absence of clear governance structures, accountability can default to the CISO. In some organisations, AI is being incorrectly classified as a security problem, with cyber chiefs absorbing accountability by default.

But who ultimately owns AI risk is still unclear. 

Regulators are enforcing accountability, transparency and control and company boards are concerned about personal and organisational liability.

This creates a disconnect and organisations need to design accountability before regulators do it for them.

For Matt Sumner, co-founder at LaunchPad, an organisation that markets an AI security, governance and compliance credential program, AI risk is different from cyber risk and treating it as such is a mistake organisations are making right now.

Matt says that traditional cyber security concerns itself with the outside world, protecting organisations from malicious insiders, infrastructure failures and software vulnerabilities.

He says that the natural owners of these risks are already well established.

“But AI changes the equation. It’s not simply something we want to defend against, it’s something that we actively use. Our goal is to enable safe use of AI simply by reducing that liability and risk, particularly for the business and the directors”, he says.

Matt says the biggest AI risk isn’t necessarily that someone hacks the model, but that an organisation trusts the model more than it should.

To illustrate what happens when governance is absent, Matt points to a real-world example from a manufacturing company.

Its CEO, who had genuinely good intentions, pushed through an AI solution designed to monitor workers for repetitive stress injuries.

The system, which used infrared technology to detect physical strain building up in muscles and joints, was meant to help prevent workplace injuries before they occurred.

In this instance, the CEO secured buy-in from unions through personal relationships.

The project did not succeed. Staff were worried about going to the kitchen too often or being monitored when they went to the bathroom.

They were also concerned about what would happen to their personal data if they were injured and how this might impact their insurance.

Matt says this is an example of where one person alone made a decision that should have been made by many.

He says the solution needed to be discussed at a board level and involve HR, and people and culture teams, as well as CISOs.

“Inevitably, somebody would have asked the question or looked at it from a different lens and said, ‘this could have a negative impact.’”

This is not an edge case, as AI is now being embedded in everything from CRM and payroll systems to HR tools and operational workflows. This means that the potential for well-meaning employees to make consequential decisions without appropriate oversight is growing.

Boards are accountable for AI risk

Australia’s National AI Plan, launched last December, is a framework that puts the onus on organisations operating in Australia to take reasonable steps to control AI risk.

“’What that actually means is the board and the directors are accountable. There’s not some blanket approach where you go, ‘I’ve done a ‘tick and flick’ exercise, and I put a minimum level in [the framework]”, says Matt.

In Europe, the EU AI Act requires organisations running high-risk AI workflows to have controls in place. These controls are necessary when organisations are managing customer and health data and, in the case of the manufacturing company, monitoring the wellbeing of their staff.

“When we look at it through that lens, we can see that…risk and liability sit at a business level, not an employee level. So, until you have the guardrails in place, particularly the education piece, a well-meaning employee who has done the wrong thing actually carries none of the risk.

“If you look at it from a business point of view, an employee trying to do the right thing [but it] causes a catastrophic event, whether it’s loss of revenue or customers, or data leakage, you’re going to wear that until you have a governance structure in place”, he says.

What regulators want to see

Matt says regulators around the world want to see is evidence of oversight, not just a policy document but a genuine governance infrastructure.

“They want to see that you’re on top of your technology risk, that you’ve got a governance framework, and that you’ve got privacy and legal [considerations] in place. A keynote point is that APRA and the Federal Court of Australia recently [indicated that having] an AI policy [alone] doesn’t stack up.

“They want to see that you’ve got controls in place and guardrails…being everything from executive awareness to ensuring the workforce has been educated and [that there is] continuous improvement in that direction, not a one-off exercise”, he says.

What does stack up is what Matt refers to as “minimum defensible governance”, a demonstratable set of controls and guardrails that show regulators an organisation took its obligations seriously.

That includes executive and board-level education, workforce credentialing, continuous improvement processes, risk registers, maturity assessments and, critically, an AI committee that draws on diverse parts of the organisation.

Matt says the Australian government and other regulatory bodies locally and globally are taking a sensible approach by putting the onus for AI governance on businesses.

“I think it will catch up to a minimum level of guardrails that [organisations] do need in place, because currently it’s the wild west out there.”

Enabling people, not replacing them

There is one more dimension to AI governance that Matt is keen to address: the human one.

Fear of job displacement is real, and he acknowledges the industry has not done enough to counter it. 

But the organisations navigating this well are using governance as an enabler, not a constraint.

“One of the [attendees] at our recent roundtable put it in a very culturally positive way. They use AI governance to enable innovation. They empower their people to extend from their current roles rather than [viewing AI as] a vehicle that [means] they’re going to lose a role.

“They’re given the tools to go and experiment in a safe way in a controlled environment…and the business benefits from that as do the employees, [who are not] sitting there looking for another job or questioning their career decisions.”