L14. AI IAM: Least Privilege for Models and Pipelines
Course outlineLesson 14 of 21
Identity and access management for AI systems extends beyond user authentication. Models, pipelines, agents, and training jobs all need identities, and all of them are over-privileged in most real-world deployments.
The IAM Gap in AI Deployments
Most organizations apply rigorous IAM to human users: role-based access control, just-in-time access, privileged identity management. The same rigor often does not extend to AI workloads. Models and pipelines frequently run with overly broad service accounts, shared credentials, or even developer credentials left behind from testing.
The result: a compromised AI pipeline or agent has the same access as a careless developer, not the minimum access the task requires.
Identities in an AI System
| Identity Type | What It Needs | Common Mistake |
|---|---|---|
| Inference service | Read access to model weights; access to AI API | Running with full storage owner permissions |
| AI agent | Access to specific tools and data sources | Granting all available tools for convenience |
| Training job | Read access to training data; write to model registry | Read access to entire data lake, not just training partition |
| Fine-tuning pipeline | Read base model; write fine-tuned model; read training data | Using a shared service account with production access |
| Retrieval pipeline (RAG) | Read the retrieval corpus; write to vector index | Read access to all organizational documents |
Workload Identity for AI Services
Workload identity (managed identities in Azure, service accounts with Workload Identity Federation in GCP, IAM roles for service accounts in AWS) eliminates the need for API keys and secrets in AI workload configuration. Instead, the compute resource has an assigned identity, and access is granted to that identity.
Benefits over API keys:
- No secret to rotate, leak, or manage
- Identity is tied to the compute resource, not portable to other systems
- Access is revocable at the IAM layer without changing application code
- All access is attributed to a specific workload identity in audit logs
API Key Management for External AI Services
When calling external AI APIs (OpenAI, Anthropic, etc.), API keys are the current credential mechanism. Controls:
- Store keys in a secrets manager (Key Vault, AWS Secrets Manager), never in code or environment variables
- Use separate keys per environment (dev, staging, prod) and per use case
- Set spending limits on keys to bound the impact of a compromised key
- Rotate keys on a defined schedule and immediately on any suspected compromise
- Audit key usage: log which application, which endpoint, how many tokens per key per day
Just-in-Time Access for AI Admin Operations
Administrative operations on AI systems (retraining, model deployment, configuration changes) should use just-in-time access: temporary elevation that expires after a defined window. This applies to:
- Model registry write access (for deploying new model versions)
- Training infrastructure provisioning
- Prompt template or system prompt modifications in production
- ✓AI workloads (inference services, training jobs, agents, RAG pipelines) each need a unique, scoped identity; shared credentials and over-broad permissions are common gaps
- ✓Workload identity eliminates API keys: identity is tied to the compute resource, not portable, and all access is attributed in audit logs
- ✓Assign one managed identity per AI workload: inference service, training pipeline, and each agent type each get their own identity
- ✓External AI API keys: store in secrets manager, use separate keys per environment and use case, set spending limits, rotate on schedule
- ✓Just-in-time access for AI admin operations: model deployment, training infrastructure, and system prompt changes should use temporary elevation
1. What is the primary advantage of workload identity (managed identities) over API keys for AI service authentication?
2. A RAG pipeline needs to populate a vector index from organizational documents. Which permission scope is correct under least privilege?
Recommended: Pluralsight
Pluralsight's AI security courses cover threat modeling, governance, and practical red teaming for AI systems to complement what you learn here.