AI Governance

Vicky Withey, Governance Risk & Compliance Director, shares her thoughts on what happened.

The incident involving OpenAI and Hugging Face raises important questions about AI governance, cyber security and security architecture. While the root cause remains under investigation, there are several aspects that concern me.

I personally struggle with the narrative that an AI system simply ‘broke out’ of a sandbox. In cyber security, we rarely rely on a single security boundary. A sandbox should never be considered the only line of defence.

I think of it like protecting a house. The sandbox is the house itself, but good security architecture doesn’t stop there. You would also expect a secure perimeter fence, CCTV, motion detection, alarms, locked gates, access controls and continuous monitoring surrounding the property. If someone leaves the house, they should still encounter multiple additional layers of protection before reaching the outside world.

The same principle should apply to AI environments.

My question is therefore not simply ‘How did the AI escape?’ but ‘Where were the additional perimeter security controls that should have detected, contained or prevented movement beyond the sandbox?’

This is a classic defence-in-depth challenge. Modern security assumes that individual controls will eventually fail. The objective is to ensure that no single failure results in a wider compromise.

I also find the language surrounding the incident interesting. Some reports suggest the AI agent ‘attacked’ Hugging Face. Whilst technically that may describe the sequence of events, I think it risks distracting from the more fundamental issue.

If we use another analogy, if someone leaves every door and window open, should we really be surprised when an intruder enters?

That doesn’t make the intrusion legal, ethical or acceptable. However, it does highlight that an opportunity existed.

The AI agent simply identified and exploited weaknesses far more quickly than a human attacker might have done.

The lesson therefore is not that AI has suddenly become uncontrollable, but that AI dramatically changes the speed, scale and persistence with which existing vulnerabilities can be exploited.

From a governance perspective, this reinforces several priorities:

• AI environments should never rely on sandboxing alone as the primary security control.
• Defence-in-depth must include network segmentation, zero trust principles, least privilege, continuous monitoring, behavioural analytics and automated containment.
• AI agents should be treated as privileged workloads with defined operational boundaries rather than trusted software.
• Security architecture should assume that AI systems will continuously probe for weaknesses, even when operating as intended.
• AI red-teaming should extend beyond model safety into infrastructure, identity, APIs, third-party integrations and lateral movement scenarios.
• Organisations should implement ‘kill switches’ and policy enforcement capable of immediately restricting autonomous agents when unexpected behaviour is detected.
• Governance frameworks such as ISO/IEC 42001 should be integrated with ISO 27001, secure architecture reviews and cyber resilience programmes rather than treating AI governance as a standalone exercise.

Ultimately, I don’t see this as evidence that AI is inherently dangerous because it can ‘escape’. Rather, I see it as a reminder that autonomous systems operate at machine speed, while many organisations still defend themselves at human speed.

The real question is not whether AI will discover vulnerabilities it almost certainly will. The question is whether our governance, security architecture and operational controls are mature enough to detect, contain and respond before those vulnerabilities can be exploited.

This incident should encourage organisations to move beyond thinking about AI model safety in isolation and instead view “AI as another highly capable actor operating within their enterprise, subject to the same principles of layered security, zero trust, resilience and governance as any other critical technology.”