Encryption for AI workloads: confidential computing, format-preserving and homomorphic encryption, and key rotation.
When privileged matters are pinned to an on-prem model, the trust boundary shrinks to the physical server running the weights. But on-prem is not the same as…
AES ciphertext of an SSN looks like a 256-bit blob; a downstream system expecting cannot validate it, index it, or route on it. In production systems that is…
In a conventional RAG pipeline, the user's query is embedded on the application server and then sent in plaintext (as a vector of floats) to the vector…
A legal-tech platform accumulates secrets quickly: per-tenant encryption keys, vault data-encryption keys, model-provider API tokens, OAuth client secrets…
AI workloads handle sensitive material in more places than a typical web application: raw documents during ingestion, embeddings in a vector store, prompts and retrieved context in memory during inference, logs of model output, and the provider credentials that tie it together. Standard encryption at rest and TLS in transit are necessary but leave gaps. This section covers four techniques that close them: confidential computing for protecting data while the model runs, format-preserving encryption for keeping protected fields usable by downstream systems, homomorphic encryption for searching embeddings without revealing them, and disciplined secret and key rotation.
Data at rest is protected with envelope encryption: a data-encryption key encrypts the content and is itself wrapped by a key-encryption key held in a KMS or HSM. Data in transit uses TLS, ideally mutual TLS between internal services. Data in use is the hard case, because a model must see plaintext to compute on it. Trusted execution environments encrypt memory and attest to the code running inside; homomorphic schemes compute on ciphertext directly, at a large performance cost, which today makes them practical mainly for narrow operations such as similarity scoring.
| Technique | Protects | Trade-off |
|---|---|---|
| Confidential computing (TEE) | Prompts and weights in memory on shared or on-prem hosts | Hardware support required; attestation adds operational work |
| Format-preserving encryption | Structured identifiers that must keep their format | Weaker than AES-GCM for small domains; needs careful tweak use |
| Homomorphic embedding search | Query and document vectors from the vector store operator | Much slower and larger ciphertexts than plaintext search |
| Secrets and key rotation | Every other layer, by limiting key exposure time | Requires automation and versioned keys to avoid outages |
Related: LLM Defense.