L13. AI Network Security: Private Endpoints & Isolation
Course outlineLesson 13 of 21
AI inference APIs are network services, and all the principles of network security apply. But AI workloads introduce specific network architecture requirements around data in transit, egress control, and isolation that deserve dedicated attention.
The Default State Is Too Open
Cloud AI services default to public endpoints. A freshly provisioned Azure OpenAI instance, AWS Bedrock endpoint, or Google Vertex AI deployment is accessible from the public internet with only API key authentication. For enterprise deployments handling sensitive data, this default is almost never acceptable.
The network security goal: ensure AI inference traffic stays within your controlled network perimeter, never traverses the public internet, and cannot reach attacker-controlled infrastructure.
Private Endpoints
Private endpoints (Azure Private Link, AWS PrivateLink, GCP Private Service Connect) create a private IP address within your VNet that routes traffic to the AI service without traversing the public internet. From the VNet's perspective, the AI service is just another private IP.
Benefits:
- Traffic never leaves your network perimeter
- DNS resolution for the AI endpoint resolves to the private IP within the VNet
- Public endpoint can be disabled entirely, preventing any non-VNet access
- Works with existing VNet-based security controls (NSGs, Azure Firewall, network policies)
- Create private endpoint in the AI service subnet
- Configure private DNS zone to resolve the AI service FQDN to the private IP
- Disable the public endpoint on the AI service
- Configure NSG rules to allow traffic only from approved source subnets
Egress Control for AI Agents
AI agents that can make outbound network calls (to retrieve documents, call APIs, browse the web) are a significant exfiltration risk if egress is not controlled. An agent compromised via prompt injection can use its HTTP tool to send data to attacker-controlled infrastructure. Egress architecture:
- Route all agent outbound traffic through an explicit proxy or firewall
- Maintain an allowlist of permitted destination domains and IPs
- Block direct internet access from agent compute
- Log all egress requests (URL, response code, bytes transferred) for anomaly detection
Network Isolation for Training Pipelines
Training workloads process raw datasets that may include sensitive content. The training network should be:
- Isolated from production networks (no direct routes between training and prod)
- Outbound restricted: training jobs should not initiate network connections except to pre-approved data sources and model registries
- Monitored for unexpected egress: data exfiltration during training is a supply chain attack vector
VNet Integration Architecture
A reference architecture for enterprise AI:
| Tier | Network Segment | Controls |
|---|---|---|
| AI inference | AI services subnet | Private endpoint only; NSG restricts inbound to app tier |
| Application | App tier subnet | Managed identity for AI calls; no direct user access to AI subnet |
| Agent compute | Agent subnet | Egress via firewall; allowlisted destinations only |
| Training | Training subnet | Isolated; no routes to production; outbound to data lake and model registry only |
| Monitoring | Management subnet | Read access to logs; no access to data plane |
- ✓Cloud AI services default to public endpoints: explicitly disable public access and use private endpoints for all enterprise deployments
- ✓Private endpoints (PrivateLink) route AI traffic to a private IP within your VNet, never traversing the public internet
- ✓Agent egress control: route outbound traffic through an explicit proxy with an allowlist; log all requests for anomaly detection
- ✓Training subnets must be isolated from production: no direct routes, outbound restricted to approved data sources and model registries
- ✓Managed identity (not API keys) should be used for application-to-AI-service authentication within the VNet
1. What is the primary security benefit of disabling a cloud AI service's public endpoint after configuring a private endpoint?
2. Why is egress control for AI agents a security control, not just a network hygiene practice?
Recommended: Pluralsight
Pluralsight's AI security courses cover threat modeling, governance, and practical red teaming for AI systems to complement what you learn here.