How to Secure Azure OpenAI Network Traffic: A Private Endpoint & Terraform Guide

The Problem: Public Network Exposure

If you're running Azure OpenAI in production, there's a good chance your API calls are traversing the public internet right now. Even with API keys and Azure AD authentication, this exposes your organization to risks:

• Data exfiltration: Sensitive prompts and responses travel over public networks
• Man-in-the-middle attacks: Despite TLS, public endpoints increase attack surface
• Compliance violations: Many regulations require private network paths for sensitive data
• Service tags aren't enough: While Azure service tags help with firewall rules, traffic still flows through public endpoints

The solution? Azure Private Link with Private Endpoints. This keeps all traffic on Microsoft's backbone network, never touching the public internet.

Architecture Overview (The Protego Approach)

Here's what we're building:

The flow: Your application (VM, App Service, AKS) connects to the Private Endpoint IP address. DNS resolves your Azure OpenAI hostname to that private IP. Traffic never leaves the Microsoft backbone.

Prerequisites

Before we start, ensure you have:

• Terraform installed (v1.5+)
• Azure CLI authenticated (az login)
• An existing Virtual Network, or we'll create one
• Appropriate Azure permissions (Contributor + User Access Administrator)

If you're storing this configuration's state in an Azure Storage backend, lock down that storage account too: see [Terraform remote state security on Azure](/blog/terraform-remote-state-azure-storage-security) for the access controls that prevent your state file from becoming its own exposure path.

Step 1: Defining the Azure OpenAI Resource (Terraform)

Let's start with the Cognitive Services account for Azure OpenAI. The critical security setting is public_network_access_enabled = false.

providers.tf:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.85"
    }
  }
}

provider "azurerm" {
  features {}
}

variables.tf:

variable "resource_group_name" {
  description = "Name of the resource group"
  type        = string
  default     = "rg-openai-private"
}

variable "location" {
  description = "Azure region for resources"
  type        = string
  default     = "eastus"
}

variable "openai_name" {
  description = "Name of the Azure OpenAI resource"
  type        = string
  default     = "openai-private-demo"
}

variable "vnet_address_space" {
  description = "Address space for the VNet"
  type        = list(string)
  default     = ["10.0.0.0/16"]
}

main.tf:

resource "azurerm_resource_group" "main" {
  name     = var.resource_group_name
  location = var.location
}

# Virtual Network
resource "azurerm_virtual_network" "main" {
  name                = "vnet-openai-private"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  address_space       = var.vnet_address_space
}

# Subnet for Private Endpoints
resource "azurerm_subnet" "private_endpoints" {
  name                 = "snet-private-endpoints"
  resource_group_name  = azurerm_resource_group.main.name
  virtual_network_name = azurerm_virtual_network.main.name
  address_prefixes     = ["10.0.1.0/24"]
}

# Subnet for compute resources (VMs, App Services, etc.)
resource "azurerm_subnet" "compute" {
  name                 = "snet-compute"
  resource_group_name  = azurerm_resource_group.main.name
  virtual_network_name = azurerm_virtual_network.main.name
  address_prefixes     = ["10.0.2.0/24"]
}

# Azure OpenAI Service
resource "azurerm_cognitive_account" "openai" {
  name                  = var.openai_name
  location              = azurerm_resource_group.main.location
  resource_group_name   = azurerm_resource_group.main.name
  kind                  = "OpenAI"
  sku_name              = "S0"
  custom_subdomain_name = var.openai_name

  # CRITICAL: Disable public network access
  public_network_access_enabled = false

  network_acls {
    default_action = "Deny"
  }

  identity {
    type = "SystemAssigned"
  }

  tags = {
    environment = "production"
    managed_by  = "terraform"
  }
}

# Deploy a GPT model
resource "azurerm_cognitive_deployment" "gpt4" {
  name                 = "gpt-4"
  cognitive_account_id = azurerm_cognitive_account.openai.id

  model {
    format  = "OpenAI"
    name    = "gpt-4"
    version = "turbo-2024-04-09"
  }

  scale {
    type     = "Standard"
    capacity = 10
  }
}

Critical Note: The public_network_access_enabled = false setting is essential. Without this, your Azure OpenAI resource will still accept connections from the public internet, defeating the purpose of your Private Endpoint.

Step 2: Configuring the Private Endpoint and DNS Zone

This is where most architects struggle. The Private Endpoint alone isn't enough; you need proper DNS configuration so your applications resolve the Azure OpenAI hostname to the private IP.

private-endpoint.tf:

# Private DNS Zone for Azure OpenAI
resource "azurerm_private_dns_zone" "openai" {
  name                = "privatelink.openai.azure.com"
  resource_group_name = azurerm_resource_group.main.name
}

# Link the DNS Zone to your VNet
resource "azurerm_private_dns_zone_virtual_network_link" "openai" {
  name                  = "openai-dns-link"
  resource_group_name   = azurerm_resource_group.main.name
  private_dns_zone_name = azurerm_private_dns_zone.openai.name
  virtual_network_id    = azurerm_virtual_network.main.id
  registration_enabled  = false
}

# Private Endpoint for Azure OpenAI
resource "azurerm_private_endpoint" "openai" {
  name                = "pe-openai"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  subnet_id           = azurerm_subnet.private_endpoints.id

  private_service_connection {
    name                           = "openai-privateserviceconnection"
    private_connection_resource_id = azurerm_cognitive_account.openai.id
    is_manual_connection           = false
    subresource_names              = ["account"]
  }

  private_dns_zone_group {
    name                 = "openai-dns-zone-group"
    private_dns_zone_ids = [azurerm_private_dns_zone.openai.id]
  }

  tags = {
    environment = "production"
    managed_by  = "terraform"
  }
}

Why DNS is Painful Here

When you create a Private Endpoint, Azure assigns it a private IP (e.g., 10.0.1.4). But your application still calls your-openai.openai.azure.com. Without proper DNS:

1. Public DNS returns the public IP
2. Your request goes to the public endpoint
3. Azure OpenAI rejects it (public access is disabled)
4. You get a 403 Forbidden error

The privatelink.openai.azure.com Private DNS Zone solves this:

1. Your VNet is linked to the Private DNS Zone
2. DNS queries from within the VNet hit this zone first
3. It returns the private IP for your Azure OpenAI resource
4. Traffic flows privately through the Private Endpoint

Step 3: Verifying Connectivity (The CloudOps Check)

Don't just deploy and assume it works. Here's how to verify your traffic is truly private.

Deploy a Test VM

test-vm.tf (optional - for verification):

resource "azurerm_network_interface" "test" {
  name                = "nic-test-vm"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name

  ip_configuration {
    name                          = "internal"
    subnet_id                     = azurerm_subnet.compute.id
    private_ip_address_allocation = "Dynamic"
  }
}

resource "azurerm_linux_virtual_machine" "test" {
  name                = "vm-test-openai"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  size                = "Standard_B2s"
  admin_username      = "azureuser"
  network_interface_ids = [
    azurerm_network_interface.test.id,
  ]

  admin_ssh_key {
    username   = "azureuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts"
    version   = "latest"
  }
}

Verification Steps

SSH into your VM and run these commands:

1. Verify DNS Resolution

nslookup your-openai.openai.azure.com

Expected output (private IP):

Server:    168.63.129.16
Address:   168.63.129.16#53

Non-authoritative answer:
your-openai.openai.azure.com canonical name = your-openai.privatelink.openai.azure.com.
Name:      your-openai.privatelink.openai.azure.com
Address:   10.0.1.4

If you see a public IP, your DNS configuration is wrong.

2. Test Connectivity

curl -I https://your-openai.openai.azure.com/

You should get a response (even if it's a 401 Unauthorized; that means connectivity works, you just need authentication).

3. Test API Call

curl https://your-openai.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2024-02-15-preview \
  -H "Content-Type: application/json" \
  -H "api-key: YOUR_API_KEY" \
  -d '{"messages": [{"role": "user", "content": "Hello!"}], "max_tokens": 50}'

Common Errors and Troubleshooting

Error: 403 Forbidden

Cause: Traffic is hitting the public endpoint, which is disabled.

Fix:

• Verify the Private DNS Zone exists and is linked to your VNet
• Check that DNS resolution returns a private IP (10.x.x.x)
• Ensure your VM/application is in a subnet that's linked to the DNS zone

Error: DNS Resolution Failed

Cause: Private DNS Zone not linked to VNet, or VNet link not active.

Fix:

az network private-dns link vnet list \
  --resource-group rg-openai-private \
  --zone-name privatelink.openai.azure.com

Verify the link shows provisioningState: Succeeded.

Error: Virtual Network Link Missing

Cause: Terraform apply didn't create the VNet link, or it was deleted.

Fix:

az network private-dns link vnet create \
  --resource-group rg-openai-private \
  --zone-name privatelink.openai.azure.com \
  --name openai-vnet-link \
  --virtual-network vnet-openai-private \
  --registration-enabled false

Error: Connection Timeout

Cause: NSG rules blocking traffic to the Private Endpoint subnet.

Fix: Ensure your NSG allows outbound traffic to the Private Endpoint subnet on port 443.

Outputs

Add these outputs to easily retrieve important values:

outputs.tf:

output "openai_endpoint" {
  description = "Azure OpenAI endpoint URL"
  value       = azurerm_cognitive_account.openai.endpoint
}

output "private_endpoint_ip" {
  description = "Private IP of the Azure OpenAI Private Endpoint"
  value       = azurerm_private_endpoint.openai.private_service_connection[0].private_ip_address
}

output "openai_id" {
  description = "Resource ID of Azure OpenAI"
  value       = azurerm_cognitive_account.openai.id
}

Conclusion

By implementing Private Endpoints for Azure OpenAI, you achieve:

• Zero public internet exposure: All traffic stays on Microsoft's backbone
• Compliance alignment: Meet requirements for private data paths
• Reduced attack surface: No public endpoint to target
• Infrastructure as Code: Reproducible, auditable deployments

The complete Terraform configuration creates everything you need: VNet, subnets, Azure OpenAI with public access disabled, Private Endpoint, and Private DNS Zone with VNet link.

Next steps:

1. Clone this configuration to your repository
2. Customize variables for your environment
3. Run terraform plan to review changes
4. Deploy with terraform apply
5. Verify with nslookup from inside your VNet

For the complete Terraform files, check out the examples in this guide and adapt them to your specific requirements.

If your team is moving from raw Azure OpenAI to Azure AI Foundry, the same Private Link pattern applies to hubs and projects: see [Azure AI Foundry Private Link setup](/blog/azure-ai-foundry-private-link-setup) for the Foundry-specific DNS zones and managed identity considerations.

Frequently Asked Questions

Why does disabling public network access on Azure OpenAI cause a 403 error even with a Private Endpoint deployed?

The most common cause is a DNS resolution failure. When public access is disabled, Azure rejects connections that arrive via the public endpoint IP. If DNS is still resolving the Azure OpenAI hostname to its public IP rather than the Private Endpoint's private IP, the request never reaches the private path and is rejected. Verify DNS resolution from inside your VNet using nslookup and confirm the returned IP is in your private subnet range, not a public Azure IP.

What is the privatelink.openai.azure.com Private DNS Zone and why is it required?

Azure Private Link uses a canonical name (CNAME) redirect so that the Azure OpenAI hostname resolves to a privatelink subdomain. Without a Private DNS Zone for privatelink.openai.azure.com linked to your VNet, the CNAME chain terminates at the public DNS record and returns the public IP. The Private DNS Zone intercepts the CNAME resolution for resources in the linked VNet and returns the private IP assigned to your Private Endpoint. Without this zone, the Private Endpoint is deployed but unreachable because applications cannot resolve its address.

Can Azure App Service or Azure Functions access a private Azure OpenAI endpoint without a VNet?

No. Private Endpoints are only reachable from within the VNet or from networks connected to it via VNet peering or VPN/ExpressRoute. Azure App Service and Azure Functions must have VNet integration enabled, with outbound traffic routed through a delegated subnet in the VNet that has connectivity to the Private Endpoint subnet. The application's VNet integration subnet and the Private Endpoint subnet must be in the same VNet, or connected via peering with proper DNS forwarding.

What is the Terraform deployment order dependency between the Private Endpoint and disabling public access?

The correct Terraform order is to create the Private Endpoint resource and Private DNS Zone link before or simultaneously with setting public_network_access_enabled = false. If you set public access to false in a separate apply before the Private Endpoint is provisioned, any subsequent Terraform operations that need to reach the Azure OpenAI resource from outside the VNet (such as deploying model configurations) will fail. Using depends_on between the cognitive account network configuration and the private endpoint resource ensures Terraform sequences the deployment correctly.

How can multiple spoke VNets in a hub-and-spoke landing zone access the same private Azure OpenAI endpoint?

Deploy the Private Endpoint and its Private DNS Zone in the hub or connectivity subscription. Link the Private DNS Zone to the hub VNet, and use VNet peering between spoke VNets and the hub VNet with DNS resolution enabled. Applications in spoke VNets send DNS queries through the Azure DNS resolver, which traverses the peering link to the hub where the Private DNS Zone resolves the Azure OpenAI hostname to the Private Endpoint's private IP. Do not deploy separate Private Endpoints in each spoke VNet: that creates redundant endpoints, adds cost, and complicates DNS management.