Why AI scale needs real-time observability, according to McKinsey’s Chief Digital Risk Officer

AI is creating new opportunities for organisations. It is also exposing them to new forms of risk at unprecedented speed.

At Digital & AI Edge, Jim Boehm, Expert Partner and Chief Digital Risk Officer at McKinsey & Company, joined ADAPT Advisor and Influential CISO David Gee to explore how organisations can build the trust, visibility and resilience required to scale AI safely.

Drawing on McKinsey’s own experience managing an AI enabled cyber incident, Jim argued that competitive advantage will increasingly come from an organisation’s ability to manage risk as effectively as it deploys innovation.

Key takeaways:

• Digital trust enables organisations to adopt AI faster by embedding risk management into everyday operations.
• AI threats are already operating at machine speed, exposing the limits of traditional security and response models.
• Observability provides the visibility organisations need to manage AI systems safely and at scale.

Digital trust helps organisations move faster

Organisations that treat risk as a core business capability will be better positioned to capture value from AI.

Cyber security has traditionally been viewed as a compliance requirement or a constraint on innovation.

Jim argued that this mindset creates unnecessary friction.

As organisations become more dependent on data, AI and digital services, trust becomes part of the product itself.

He described cyber security as a cost of goods sold rather than a supporting function.

Customers, employees and stakeholders increasingly expect secure digital experiences by default.

Trust is now a baseline expectation.

This changes the role of risk leaders.

Rather than acting as gatekeepers, security and risk teams help create the conditions for faster adoption by ensuring organisations can move quickly without exposing themselves to unacceptable levels of risk.

The organisations that move fastest in the AI era will be those that embed trust into how products, services and digital experiences are designed and delivered.

AI threats are already operating at machine speed

Many organisations are still developing AI strategies while threat actors are already using AI to identify vulnerabilities and accelerate attacks.

Jim shared McKinsey’s experience responding to an AI enabled cyber incident where a researcher used an LLM powered testing harness to gain access to the backend of an AI chatbot application.

The incident reinforced how quickly AI is changing the threat landscape and how difficult it can be for human led processes to keep pace.

The challenge extends beyond individual attacks.

As organisations apply AI to vulnerability discovery and red teaming, they are uncovering issues at a scale that traditional response models struggle to manage.

McKinsey’s own AI powered red teaming efforts revealed significant vulnerabilities that existing security tools had not surfaced.

Responding required the mobilisation of more than 100 engineers and prompted a broader reassessment of how security operations, engineering teams and risk functions work together.

The lesson is becoming increasingly clear. AI is amplifying both offensive and defensive capabilities.

Organisations that rely solely on human capacity to detect, prioritise and respond will find it increasingly difficult to keep pace with machine speed threats.

Observability creates the visibility needed for AI scale

As AI adoption grows, visibility becomes more important than static governance mechanisms.

Many organisations begin by building inventories and registries to track AI assets.

While these systems remain important, Jim argued they should function as systems of record rather than systems of operation.

What matters more is continuous visibility into what AI systems are doing, how they are behaving and whether they are operating within acceptable boundaries.

McKinsey’s approach increasingly focuses on observability, capturing telemetry, monitoring agent behaviour and maintaining oversight across AI systems in real time.

This includes the ability to intervene when agents behave outside their intended parameters and ensuring that access controls, governance policies and accountability structures extend to both people and AI agents.

As organisations deploy larger numbers of autonomous systems, observability becomes essential for maintaining trust, managing risk and scaling AI safely.

Human resilience remains part of the equation

Technology alone cannot solve the challenges created by AI driven risk.

Jim reflected on the human impact of responding to major security events, describing the pressure placed on teams working extended shifts while confronting a growing volume of vulnerabilities and emerging threats.

The experience highlighted the limits of relying on human effort alone to close the gap.

Sustainable resilience requires organisations to rethink how security, engineering and risk teams operate together.

It also requires investment in automation, better visibility and stronger coordination across functions.

AI may be accelerating the pace of change, but organisational readiness will determine how effectively leaders respond to it.

For Jim, the future belongs to organisations that build trust into the foundations of their digital operations.

As AI becomes embedded into every aspect of business, the ability to manage risk, maintain visibility and respond at speed will increasingly separate those that scale confidently from those that struggle to keep up.