Your Data, Their GPUs: Self-Service Confidential AI with Fortanix Confidential Containers and Rafay

How Fortanix and Rafay extend one-click confidential AI from Confidential VMs to Kubernetes pods, so enterprises can run AI on sensitive data using GPUs they don't own 

Regulated enterprises want AI on their most sensitive data. A bank wants to transcribe recorded customer calls full of account numbers, names, and dates of birth. The fastest way to get modern GPUs is through a neo-cloud provider, but those GPUs belong to someone else.  

Encryption protects data at rest and in transit, yet the moment an AI workload is processed, i.e., it is in use, the data and the model weights sit exposed in memory, visible to the host OS, the hypervisor, and the infrastructure operators.  

Confidential computing closes that gap by protecting data in use, inside hardware-based Trusted Execution Environments (TEEs). 

Where we started: Confidential VMs 

Earlier this year, Fortanix and Rafay introduced a joint integration that delivers Confidential AI as a turnkey, self-service service on neo-cloud GPUs (see Scaling Trust: The Fortanix and Rafay Integration for Enterprise Confidential AI).  

Fortanix supplies the trust layer: the Confidential Computing Manager (CCM) performs composite attestation of the hardware, BIOS, and software stack, and the Data Security Manager (DSM) is a FIPS 140-3 Level 3-certified Hardware Security Module (HSM) with built-in Key Management (KMS) that releases keys only after CCM validates the attestation.  

Rafay supplies orchestration: repeatable SKUs, multi-tenancy, and a self-service portal. A user clicks Deploy in the Rafay portal, and within minutes, a Confidential VM (CVM) boots, attests through CCM, and receives its keys from DSM, with an NVIDIA Parakeet speech model running inside. 

What's new: Confidential Containers 

Fortanix Confidential AI now supports Confidential Containers based on the CNCF CoCo (Confidential Containers) projects (see Confidential AI at Container Speed), the Kata runtime, and the Rafay integration supports them alongside CVMs.  

Teams keep the container workflow they already use, with the same Fortanix attestation and key management, so security policy does not change with the form factor. 

Architecture 

On the host, Kubernetes works as usual: kubelet hands the pod to the container, which uses kata-runtime to launch it in its own lightweight VM, and the kata-agent inside the guest manages the containers over gRPC on vsock. With CoCo, that VM is a hardware TEE (AMD SEV-SNP or Intel TDX) with an NVIDIA confidential-computing GPU passed through, so memory is encrypted and integrity-protected and the host, hypervisor and operators sit outside the trust boundary. Images are pulled and unpacked inside the guest by CoCo's image-rs (backed by the nydus snapshotter on the host), so the host never sees decrypted image contents. 

Fortanix extends CoCo in the guest with two modules and an update to two other modules. The CoCo AA module is updated to use ccm_as for attestation. ccm_as, the CCM attestation service, measures the container itself. ccm_kbc, the key broker client, retrieves keys from Fortanix DSM. The CoCo CDH module is updated to use ccm_kbc for key release. On the host, the Fortanix Node Agent (deployed by the Rafay blueprint) brokers communication with CCM and collects hardware-level measurements of the CPU, GPU and firmware. Fortanix CCM is the verifier, and Fortanix DSM, a KMS backed by a FIPS 140-3 Level 3 HSM, is the relying party that releases keys. Both deploy as SaaS, on-premises or fully air-gapped.

Figure 1. Confidential Containers with Fortanix extensions, orchestrated by Rafay. Numbers follow the pod-start flow below. 

Demo walkthrough: confidential speech-to-text, self-service 

The demo follows a hypothetical neo-cloud, CloudAway, serving a hypothetical bank, ApexTrust, across four personas. CloudAway provides the hardware, Rafay automates the platform, and Fortanix provides the Confidential AI capability. 

1. Neo-cloud admin. In the Rafay Operations Console, CloudAway onboards its GPU data centers and creates an isolated tenant organization for ApexTrust. 

2. Enterprise admin. ApexTrust's platform admin creates a Kubernetes cluster from the fortanix-coco blueprint. Rafay syncs the add-ons, including CoCo with the Fortanix modules and the Fortanix Node Agent, which brokers attestation with CCM. 

3. Application developer. The developer packages the speech-to-text app, encrypts the image (stt-app:1.0-encrypted) and registers the build in Fortanix CCM as a Confidential Container (CoCo) build for AMD SEV-SNP with NVIDIA GPU. CCM records the build's measurement and HOST_DATA, the app's fingerprint; once approved, the key is released only to a pod that matches it with a valid certificate. 

4. Enterprise admin, again. In Rafay SKU Studio, the admin publishes a "Speech-to-Text (Confidential)" compute SKU that pins the encrypted image, GPU count and the kata runtime class, so users cannot launch it any other way. 

5. Enterprise user. In the Rafay Developer Hub, the user clicks Start. The pod launches as a confidential container, attests, receives its key and moves to Running. The app reports "Confidential container · Attested by Fortanix" and transcribes calls that are processed only inside the TEE. GPU time is billed by the hour, so the app stays off when idle.  

What happens when the pod starts 

1. Schedule. Rafay schedules the pod with the kata runtime class, and kata-runtime boots it in a guest TEE with the GPU attached. 
2. Measure. The Node Agent collects hardware measurements (CPU, GPU, firmware) and ccm_as measures the container; together the evidence go to CCM through the Node Agent. 
3. Attest. CCM verifies the composite evidence against the approved build (measurement and HOST_DATA) and hands the workload a certificate. 
4. Release. CDH calls ccm_kbc, which presents that certificate to DSM over mutual TLS. DSM releases the image key only to the attested workload. 
5. Decrypt and run. CDH passes the key to image-rs, which decrypts the image pulled from the registry, and the container starts. If any check fails, no key is released and the image stays encrypted. 

Why it matters 

• AI model owners can run proprietary models with sensitive IP (weights) on customer or neo-cloud infrastructure, because the entire container stays encrypted until the environment is attested. 
• Neo-clouds and AI factories can give each tenant verifiable isolation on GPUs and sell confidential AI as a catalog SKU, not a custom project. 
• Sovereign and regulated buyers keep keys in their own HSM, on-premises or air-gapped, with proof of what runs before any data is decrypted. 

Learn more:  Fortanix + Rafay CVMs   |   Confidential AI at Container Speed