L21. AI Security Monitoring: Dashboards & Continuous Assurance
Course outlineLesson 21 of 21
AI security cannot be assessed once at deployment and forgotten. Continuous monitoring detects model drift, emerging attack patterns, and control failures before they become incidents. This lesson covers the metrics, dashboards, and processes that make AI monitoring operational.
Why AI Requires Continuous Monitoring
A traditional application does not spontaneously change behavior after deployment. AI systems can drift due to:
- Distribution shift: real user inputs differ from the evaluation set; behaviors not tested emerge over time
- Model updates: the underlying model changes (API-delivered models can be updated by the provider without notice)
- Prompt drift: system prompts or templates change, sometimes through informal channels
- Attack evolution: new jailbreak techniques emerge and are tested against production systems before the security team is aware
Continuous monitoring is the operational mechanism that catches these changes.
Key Metrics to Track
Safety and security metrics:- Defect rate per harm category (calculated weekly on a rolling sample of production traffic)
- Safety filter trigger rate (volume and trend over time)
- Anomalous tool call rate (fraction of sessions with anomalous tool call patterns)
- Prompt injection detection rate (alerts from injection detection logic)
- Average input token length (spikes may indicate context flooding attacks)
- Error rate by endpoint (unusual patterns may indicate probing)
- Latency distribution (outliers may indicate adversarial inputs designed to trigger expensive computation)
- Session length distribution (extreme outliers may indicate automated attack sessions)
- Data residency compliance rate (fraction of requests processed in approved regions)
- PII detection rate in outputs (outputs flagged by DLP containing PII)
- Human oversight coverage (fraction of high-impact actions that received human review)
Dashboard Architecture
A practical AI security dashboard stack:
| Layer | Tool | What It Shows |
|---|---|---|
| Raw data | SIEM (Sentinel, Splunk) | All raw inference logs, tool call logs, filter trigger logs |
| Metrics | Time-series DB (Prometheus, Azure Monitor) | Aggregated metrics, trends, anomaly detection |
| Alerting | Alert manager | Threshold breaches, anomaly detections, compliance failures |
| Executive view | BI tool (Power BI, Grafana) | Weekly defect rate trends, safety posture summary, incident count |
Continuous Assurance Process
Monitoring provides data; continuous assurance turns data into action:
- Weekly defect rate review: sample production traffic, classify outputs, compare to thresholds
- Monthly red team touch: brief manual red team session to test for new attack techniques
- Quarterly full evaluation: run the complete evaluation set against the current system; update thresholds if the risk profile has changed
- Incident-triggered review: any confirmed AI security incident triggers an immediate out-of-cycle evaluation
- Model update validation: any model version change triggers an evaluation run before traffic is migrated
The output of this process is a living AI security posture document that shows, at any point in time, what the current defect rates are, when the last evaluation was run, and what the trend is over time.
- ✓AI systems drift after deployment: distribution shift, provider-side model updates, prompt changes, and new attack techniques all require continuous monitoring
- ✓Weekly defect rate calculation on a rolling sample of production traffic is the core continuous assurance metric
- ✓Model update validation: any model version change (including provider-side updates) triggers an evaluation run before traffic is migrated
- ✓Input token length spikes may indicate context flooding; latency outliers may indicate expensive adversarial inputs; these operational metrics are security signals
- ✓Continuous assurance cycle: weekly defect review, monthly manual red team, quarterly full evaluation, incident-triggered and model-update-triggered evaluations
1. Why can an AI system's behavior change after deployment without any action by the deploying organization?
2. A spike in average input token length in your AI application monitoring is detected. Why is this a potential security signal?
Recommended: Pluralsight
Pluralsight's AI security courses cover threat modeling, governance, and practical red teaming for AI systems to complement what you learn here.