AI Security: Risks You Need to Know and How to Mitigate Them

The AI security problem nobody's talking about

Every company is rushing to adopt AI. ChatGPT, Copilot, custom LLMs, and now autonomous agents that browse, write code, and take actions on their own: they are everywhere. Security teams are struggling to keep up, and the risks are real.

I've seen employees paste customer data into ChatGPT, companies deploy AI assistants without input validation, and "AI-powered" applications that trust everything the model outputs. None of this is exotic. It is the default outcome of adopting AI faster than you govern it.

What are AI security risks? AI security risks are the ways an organization's data, systems, or decisions can be compromised through its use of AI: models prompted or fine-tuned with sensitive data, inputs crafted to manipulate model behavior, AI agents given more access than they need, and outputs trusted without verification. They span the full lifecycle of enterprise AI use, from an employee pasting text into a chatbot to an autonomous agent executing code in production.

AI security risks: a practical taxonomy

The risks below cover that full lifecycle, from a single employee typing into a chat window to a fleet of autonomous agents with production access. A few of these deserve a deep technical guide of their own; where that exists on protego.me, it's linked inline instead of repeated here.

1. Data leakage and PII exposure

This is the most common risk and the easiest to prevent, yet organizations still get it wrong.

What happens: an employee pastes confidential data into a public AI service. That data might be used for training, stored in logs, or accessed by the provider under its own retention policy.

Real example: Samsung engineers pasted proprietary source code into ChatGPT in 2023. The company responded by banning consumer AI tools company-wide.

Prevention:

• Use enterprise AI tiers with data processing agreements that exclude your data from model training
• Deploy DLP policies that detect sensitive data leaving for AI services
• Train employees on what is and is not acceptable to share with an AI tool

2. Prompt injection

This is the SQL injection of the AI world, and most AI applications remain vulnerable to it.

What happens: attackers craft inputs that make the AI ignore its instructions and do something else instead, whether that means leaking a system prompt, bypassing a content filter, or triggering an unintended action.

Prevention:

• Never trust user input directly
• Separate system prompts from user input structurally, not just with formatting
• Use output filtering to catch unexpected responses
• Implement rate limiting and monitoring on every AI-facing endpoint

3. Indirect prompt injection

Even sneakier: malicious instructions hidden in documents, emails, or web pages the AI processes on your behalf. Our AI attack surfaces guide breaks down exactly where these payloads enter your stack, and our prompt injection detection guide for enterprise copilots covers detection specifically for Microsoft 365 Copilot and similar tools.

Prevention:

• Sanitize all external content before AI processing
• Use separate AI instances for different trust levels
• Do not let AI directly execute actions based on external content without a human or policy gate

4. Insecure output handling

When AI outputs are used without validation, bad things happen. AI-generated code that gets executed, or content displayed without escaping, is an instant path to remote code execution or XSS.

5. Shadow AI and unsanctioned tool use

Employees adopt AI tools faster than IT can approve them: browser extensions, personal ChatGPT accounts, AI features quietly switched on inside SaaS products they already use. None of it goes through security review, and none of it shows up in an asset inventory. Our shadow AI detection and governance guide covers how to find AI tools employees are using without approval, and how to enforce policy without blocking legitimate work.

6. Model supply chain risk

A model pulled from a public registry or a third-party fine-tune carries the same supply chain risk as any other dependency you did not write yourself: it can contain a deserialization payload, a backdoored weight, or an undisclosed data source. Our AI supply chain verification guide walks through provenance checks and scanning pipelines before a model reaches production compute.

7. Over-permissioned AI agents and non-human identities

Agents need credentials to act: API keys, service principals, OAuth tokens. Teams under deadline pressure tend to grant broad access once rather than scope it per task, and that access rarely gets reviewed again afterward. Our non-human identities guide covers why machine identities are now the fastest-growing, least-governed attack surface in most enterprise environments.

8. Hallucination-driven business risk

A model that confidently states something false is not just an accuracy problem, it is a liability problem. In February 2024, Canada's Civil Resolution Tribunal held Air Canada responsible for a bereavement fare discount its website chatbot had invented, ruling that the airline was accountable for information its own AI presented to a customer.

Prevention: treat customer-facing AI output about policy, pricing, or legal terms as if a human agent said it, with the same review process and audit trail you would require from a person.

Building secure AI applications

Architecture principles

Every AI application should have:

• Input filtering to block injection attempts
• Rate limiting to prevent abuse
• Output filtering to validate and sanitize
• Action gates requiring human approval for sensitive actions

If you are building directly on OpenAI or Claude APIs, our practical guide to securing OpenAI and Claude API deployments covers concrete configuration steps for each of these layers.

Enterprise AI governance

Create policies covering:

• Approved tools list
• Data handling rules
• Development standards
• Incident response procedures

A practitioner checklist for rolling out AI tools safely

Before an AI tool moves from pilot to production, or before you approve a new one for company-wide use, walk it through these questions. Use it as a gate, not a formality.

• Data flow: where does input data go, and does the vendor's DPA prohibit training on it?
• Retention: how long is data stored, and can you force deletion on request?
• Identity: does the tool authenticate with a scoped service account, or a shared credential with broad access?
• Output trust: is anything the model produces executed, displayed, or acted on without a validation step?
• Blast radius: if this tool is compromised or manipulated, what is the worst action it can take on its own?
• Logging: are prompts, outputs, and tool actions logged somewhere your SOC can actually query?
• Ownership: who owns this tool in the asset inventory, and who gets paged if it misbehaves?
• Exit plan: can you revoke access and delete data cleanly if you drop the vendor?

Common failure modes in real AI deployments

A few patterns show up repeatedly in AI rollouts that get flagged during a security review after the fact, rather than before:

• Security review happens after the tool is already handling production data, not before
• A pilot approved for low-sensitivity data quietly expands to production datasets without a second review
• Nobody in the organization owns the AI tool inventory, so shadow deployments outnumber approved ones
• An agent is granted a broad service-account credential "just for now," and it is still active a year later
• Legal reviews the vendor contract; security never reviews the technical integration
• Output validation exists in the demo and gets cut under deadline pressure before launch

The tradeoff: speed of AI adoption vs governance maturity

Every organization sits somewhere on a curve between two failure modes: move fast and accumulate ungoverned AI risk, or govern so heavily that teams route around security to get anything shipped. Neither extreme holds up for long.

NIST's AI Risk Management Framework (AI RMF 1.0) is a useful anchor here, not because it prescribes specific tools, but because its four functions (Govern, Map, Measure, Manage) describe the maturity curve directly: an organization that has not mapped its AI use cases or measured their risk has no real basis for "moving fast," because it does not know what it is moving fast into.

In practice, this plays out in stages:

• Ungoverned experimentation: individuals and teams adopt AI tools independently, no inventory exists, shadow AI proliferates
• Reactive governance: a policy exists on paper, but nothing enforces it, and rollout speed does not actually slow down
• Embedded governance: security review is a step in the AI adoption path itself, not an audit that happens afterward
• Continuous assurance: guardrails are automated (DLP, prompt and output filtering, scoped identities), so speed and governance stop trading off against each other

The fastest route to that last stage is not more policy on paper. It is buying down the manual review cost with automated controls, so security stops being the thing that slows adoption down.

Quick wins for today

1. Audit current AI usage. What tools are employees using, and what data are they sharing? Our shadow AI detection and governance guide covers how to find AI tools employees are using without IT approval.
2. Block unauthorized AI tools. Use your proxy or firewall to control access at the network level.
3. Enable enterprise features. Switch from consumer to business AI tiers wherever one is offered.
4. Add basic monitoring. Log who's using what, and where the data goes.
5. Train your team. A 30-minute session on AI security basics closes most of the gap fast.

AI tools are powerful. Used carelessly, they are powerful liabilities. Take security seriously from the start, and treat governance maturity as something you build deliberately, not something that catches up on its own.

Frequently asked questions

What is prompt injection and why is it the most critical AI security risk?

Prompt injection is an attack where a malicious user crafts input that overrides an AI application's system instructions, causing the model to ignore its rules and perform unintended actions. It is considered the most critical AI-specific risk because it exploits the fundamental architecture of LLMs: they cannot reliably distinguish between trusted instructions and untrusted user input. Every AI application that accepts user input and acts on the model's output is potentially vulnerable.

How can organizations prevent employee data leakage into public AI services?

The three most effective controls are: switching to enterprise AI tiers with data processing agreements that prohibit training on your data, deploying Data Loss Prevention (DLP) policies on proxies and endpoints to detect sensitive data being sent to AI services, and running mandatory training so employees understand what types of data are prohibited. Blocking unauthorized AI services at the network level is a useful backstop but not sufficient on its own, as employees will find workarounds.

What is the difference between direct and indirect prompt injection?

Direct prompt injection happens when a user types malicious instructions into a chat interface or form that feeds an AI application. Indirect prompt injection occurs when malicious instructions are embedded in content the AI processes from external sources, such as a document the AI summarizes, a webpage it browses, or an email it reads. Indirect injection is harder to defend against because the attack arrives through what appears to be normal content rather than explicit user input.

What is shadow AI and why is it a security risk?

Shadow AI refers to AI tools employees adopt without going through IT approval or security review: personal accounts on public chatbots, AI browser extensions, or AI features enabled inside SaaS tools already in use. It is a risk because none of it appears in an asset inventory, none of it has a reviewed data processing agreement, and none of it is covered by DLP or monitoring, so sensitive data can leave the organization through a channel security never knew existed.

What are the key principles for building secure AI applications?

Every secure AI application should implement four layers: input filtering to catch injection patterns before they reach the model, rate limiting to prevent abuse and cost attacks, output filtering to sanitize responses before displaying them to users (preventing XSS and data leakage), and action gates that require human approval before the AI takes any sensitive action like sending emails or modifying records. No single layer is sufficient; defense in depth is required.

How should organizations govern AI tool adoption to reduce security risk?

Effective AI governance requires four elements: an approved tools list specifying which AI services are sanctioned for what data classifications, clear data handling rules defining what categories of data may never be entered into AI systems, developer standards for building AI features securely including input validation and output handling requirements, and an incident response procedure specifically for AI-related incidents such as prompt injection exploits or data leakage through AI services.

How do we balance moving fast on AI adoption with keeping it governed?

Treat adoption speed and governance maturity as something to match deliberately, not trade off by accident. NIST's AI RMF frames this as four functions: Govern, Map, Measure, Manage. In practice that means building an inventory of AI use cases before scaling them, scoring risk per use case rather than applying one blanket policy, and investing in automated guardrails (DLP, output filtering, scoped identities) so security review stops being the bottleneck that makes teams route around it.