Why Traditional Security Is Failing Sensitive AI Workloads and Why Confidential Computing Is Now Required
Over the last few years, AI has moved from experimental side projects to the center of critical business processes. Since the launch of ChatGPT, employees and consumers alike have integrated AI into their daily work.
Enterprises, in turn, are embedding AI into workflows across financial services, government, healthcare, and other regulated industries.
That shift creates a fundamental tension. Organizations feel real pressure to adopt AI quickly, yet they must do so while protecting sensitive data and remaining compliant with increasingly complex regulations.
Many are trying to solve this with the same playbook they used for pre-AI applications: perimeter defenses, network controls, and, more recently, AI firewalls and guardrails.
Those tools still matter. But for sensitive AI workloads, they are no longer enough.
To understand why, it helps to look at what modern AI systems actually handle, where they run, and where they are most exposed.
The Three Assets at the Heart of Every AI System
When AI moves into production, it doesn’t just process low-value logs or anonymized datasets. It typically operates on three distinct, high-value assets.
- Model weights (parameters): These are the learned parameters of the model—the representation of all the data and training that went into it. For proprietary or closed-weight models, the weights are the core intellectual property. If an attacker can copy them, they can replicate or fine-tune the model without permission, effectively stealing the product.
- User data (prompts and responses): Prompts and outputs often contain extremely sensitive information: financial records, healthcare details, legal strategies, internal plans, or personal data. In regulated industries, this information is subject to strict privacy and compliance requirements.
- Context / enterprise data: To make AI truly useful, organizations connect it to internal systems such as SharePoint, Slack, Confluence, and databases. This enterprise data becomes context for retrieval-augmented generation and reasoning. In many cases, it contains the company’s most sensitive documents, communications, and records.
Traditional security architectures were not designed with these three AI-specific assets in mind especially not at runtime, when models are actively processing them.
Why Perimeter‑Only Security Falls Short
Most organizations still rely on perimeter-oriented tools to secure AI, including network firewalls and segmentation, API gateways and web application firewalls, identity and access management controls, zero trust networking frameworks, and AI-specific firewalls, policy engines, and safety guardrails.
All of these are necessary. They control who can call a model, what kinds of prompts are allowed, and how outputs are filtered. But they share a critical assumption: once traffic passes through the perimeter, the system can safely operate on data and model weights in the clear.
In AI workloads, that assumption is dangerous.
In typical deployments:
- Model weights are loaded into CPU or GPU memory in plaintext to execute inference.
- Prompts and responses are often logged for debugging or analytics, sometimes across multiple services.
- Retrieved context is stored in vector databases, caches, and intermediate stores that may not be hardened at the same level as core databases.
If an attacker or a malicious or compromised insider gets past the perimeter, there is often little to prevent direct access to model weights, prompts, and enterprise context. Perimeter controls do nothing to protect data and models while they are in use.
For sensitive AI workloads, especially in financial services, healthcare, and government, that is not an acceptable risk profile.
The Sovereign and Regulated AI Challenge
The limitations of traditional security become even more obvious when you look at how organizations want to deploy AI in regulated and sovereign environments.
The default model today is simple: large model providers host their models in their own infrastructure or cloud accounts. Customers access them via APIs over the internet. This model works well for many use cases, but it breaks down when:
- Regulations (such as the EU AI Act or data residency laws) limit where data can be processed.
- National or sectoral rules (for example, those interacting with the US CLOUD Act) restrict sending data to foreign jurisdictions.
- Critical infrastructure operators, governments, and highly regulated enterprises require that AI run in environments they own or tightly control.
These customers want powerful models, but they cannot simply send sensitive data to a shared public endpoint. They need models to run inside their own data centers, sovereign AI clouds, or trusted GPU providers.
From the model owner’s perspective, this creates a serious trust problem. If they deploy models into third‑party infrastructure without strong protections, they risk:
- Theft of model weights
- Tampering with model parameters, altering behavior
- Unauthorized variants being run without their knowledge or control
Traditional perimeter security does not solve this. It can’t prevent a privileged operator or compromised host from reading model weights out of memory or from a local file system, or from inspecting prompts and responses as they flow through the runtime.
What’s needed is a way to protect models and data even when they run in someone else’s environment and to do so in a way that is provable and enforceable.
Why Confidential Computing Is Now Required
This is where confidential computing and, specifically, confidential AI becomes essential.
Confidential computing uses hardware-enforced Trusted Execution Environments (TEEs), often referred to as secure enclaves. Inside these environments, code and data are protected at runtime. They remain encrypted in memory and are isolated from the operating system, hypervisor, and even infrastructure administrators. In other words, the underlying host cannot inspect or tamper with what’s happening inside the enclave.
Applied to AI, this changes the security model in meaningful ways. Model weights can remain encrypted at rest and in transit and are decrypted only inside an attested confidential environment.
Prompts, responses, and enterprise context are processed within that same protected runtime, rather than being exposed in plaintext to cloud operators or third-party infrastructure providers.
Even key management becomes attestation-aware: encryption keys are released only to runtimes that can cryptographically prove they are running in an approved confidential configuration.
This makes possible a new, closed-loop distribution model for sensitive AI. Model providers can distribute encrypted weights to customers or sovereign AI data centers while retaining control of the keys under strict policy. Those keys are released only after remote attestation verifies that the target environment meets security requirements.
In this model, even when a system runs inside a customer-owned or third-party AI factory, the model owner maintains strong assurance over how and where it executes. Infrastructure operators cannot easily extract the weights or inspect the data being processed.
For sensitive workloads, particularly in cross-border or sovereign deployments — this is no longer a theoretical enhancement. It is rapidly becoming a baseline requirement.
Building Security into AI Factories from Day One
We are at the beginning of a large-scale build‑out of AI-optimized infrastructure—so‑called AI factories and AI data centers. These facilities will power the next generation of AI‑native applications and agents.
The key strategic question for technology leaders is whether security will be bolted on after the fact or built in from the start.
Relying solely on traditional, perimeter‑centric security for AI means accepting that model weights, user data, and enterprise context will be exposed whenever models are running. That is incompatible with the posture risk required for many regulated and mission‑critical workloads.
Confidential computing offers a path forward by adding a new, stronger layer of protection around data and models in use. Confidential AI, built on that foundation, treats model weights, prompts, responses, and context as first‑class assets to be protected at runtime, no matter where the model runs.
For organizations planning the future of AI, cloud platforms, and digital services, the lesson is clear: perimeter controls remain necessary, but they are no longer sufficient on their own.
Protecting sensitive AI workloads now requires confidential computing as a foundational part of the stack built into AI factories and platforms from day one, not added as an afterthought later.
