AI Security Risk Assessment Template

Training data contains PII, credentials, or regulated data (HIPAA, PCI, GDPR)

What data was used for training or fine-tuning? Has it been classified?

Sensitive data leaks through model outputs (memorization, prompt extraction)

Are output filters in place to detect and block PII/credential exposure?

User prompts stored without retention policy or access controls

Where are prompts logged? Who has access? What is the retention period?

RAG knowledge base contains data above the user's clearance level

Does the retrieval layer enforce per-user access controls on source documents?

Prompt injection: user input manipulates system instructions

Is there input validation and system/user prompt separation?

Indirect prompt injection via retrieved documents or external data

Are external data sources sanitized before injection into the context window?

Jailbreaking: users bypass safety guardrails through adversarial prompts

Is there a prompt firewall or classifier screening inputs before the model?

System prompt leakage exposes internal instructions or business logic

Can users extract the system prompt through conversational tricks?

Model provenance unknown: no verification of model integrity

Is the model from a trusted source? Is there a hash/signature to verify integrity?

Dependency on a single AI provider with no fallback

What happens if the provider has an outage or changes terms/pricing?

Model updates change behavior without notice (breaking downstream apps)

Are model versions pinned? Is there a testing pipeline for model updates?

Fine-tuned models inherit vulnerabilities from base model or training data

Has the fine-tuning process been reviewed for data poisoning risks?

API keys for AI services stored insecurely or over-permissioned

Are API keys rotated regularly? Are they scoped to minimum required permissions?

No rate limiting on AI endpoints (cost explosion, abuse)

Are per-user and per-endpoint rate limits enforced? Are cost alerts configured?

AI agents can take actions (tool use) without human approval for high-risk operations

Is there a human-in-the-loop for destructive actions (delete, send, purchase)?

Shared AI accounts with no per-user audit trail

Can each user's interactions be attributed to their identity for audit purposes?

Hallucinated outputs presented as facts in customer-facing applications

Are outputs grounded in retrieved sources? Is there a confidence threshold?

AI-generated content published without human review

Is there a review workflow before AI content is published externally?

Model outputs used for decisions without bias testing (hiring, lending, moderation)

Has the model been tested for demographic bias in its outputs?

No monitoring for model degradation or drift over time

Are output quality metrics tracked? Is there an alerting threshold?

No AI use policy defining approved models, use cases, and prohibited uses

Does the organization have an AI acceptable use policy? Is it enforced technically?

Cross-border data transfer violations (EU data processed by US-hosted models)

Where is data processed? Are data residency requirements met?

No incident response plan specific to AI failures (hallucination incident, data leak)

Is there a documented runbook for AI-specific incidents?

Regulatory requirements (EU AI Act, NIST AI RMF) not mapped to controls

Which AI regulations apply? Are required controls documented and implemented?