# AI Security for Users > [!abstract] What this section is > The risks facing people who **use** AI tools, as opposed to the people who build models. Assistants have become agents. They read your mail, browse on your behalf, run code, and hold access to your accounts, and that changes what can go wrong. Every incident in this section is cited to a primary or reputable source, checked on 2026-09-13. The subject moves fast, so read the dates as a shelf life. ## What changed When an assistant could only talk, the worst case was a wrong answer that you had to catch. Once it can act, a wrong instruction becomes a wrong action, and the instruction no longer has to come from you. Three capabilities arrived together: - **It reads things you didn't write.** Web pages, email, shared documents, calendar invites, code repositories, screenshots. - **It holds your access.** Connectors and OAuth grants let it read and write inside your accounts, with your authority. - **It can reach the outside world.** Send a message, load a URL, open a pull request, render an image from a remote server. Each one alone is manageable. All three in one session is what Simon Willison named **the lethal trifecta**: private data, untrusted content, and a way to communicate externally [2]. Most of the incidents in these notes are that combination. ## The risks | Risk | In one line | Note | |---|---|---| | **Prompt injection** | Text inside something the AI reads gets followed as an instruction | [[Prompt Injection]] | | **Agents with too much reach** | Broad connectors and auto-approval turn one mistake into real damage | [[Agents and Connectors]] | | **Poisoned supply chains** | Malicious extensions, skills, MCP servers, and packages the model invents | [[AI Supply Chain Risks]] | | **Exposure** | Pasted secrets, indexed share links, memory, and AI-written material you publish | [[AI Data Exposure]] | | **AI-powered fraud** | Cloned voices, deepfaked video calls, phishing written for you personally | [[AI-Powered Scams]] | | **Believing the output** | Confident errors, including an agent's account of what it just did | below | The industry lists agree on the order. OWASP's Top 10 for LLM Applications (2025) puts prompt injection first and sensitive information disclosure second, with excessive agency at sixth [1]. Its separate Top 10 for Agentic Applications, released December 2025, opens with agent goal hijack and includes memory and context poisoning [3]. ## Believing the output A model delivers a wrong answer in the same tone as a right one. That is an old problem, and two things make it a security problem now. **Invented dependencies.** In a USENIX Security 2025 study of 2.23 million generated code samples, 19.7% of recommended packages did not exist, which came to 205,474 unique fake names [4]. Anyone can register those names. See [[AI Supply Chain Risks]]. **Agents reporting on themselves.** In July 2025 a coding agent deleted a company's production database during an explicit code freeze, and then told the user a rollback was impossible. It wasn't [5]. An agent's summary of its own actions is output like any other, and it needs the same check. The general habit is to ask **what a claim is based on**. A model reasoning from a local copy, a cached page, or a stale summary will describe the live system with complete confidence. ## A short checklist 1. **Treat everything the AI reads as untrusted**, your own inbox included. You don't control who writes to it. 2. **Grant the narrowest access that does the job.** Read-only by default. Separate accounts or profiles for experiments. 3. **Keep a human on irreversible actions**: sending, paying, deleting, publishing. Enforce it with the tool's permission settings, not with a request in the prompt. 4. **Don't auto-approve** in a session that also reads untrusted content. 5. **Install connectors, MCP servers, skills and extensions the way you'd install software**, because they are software. 6. **Check that a suggested package exists and is the one you meant** before installing it. 7. **Never paste secrets**, and assume anything shared by link can become public. 8. **Review what the assistant remembers**, and delete what shouldn't be there. 9. **Verify requests for money or credentials on a channel you already had**, never the one the request came in on. 10. **Check claims against the source**, especially claims about what an agent did. ## Sources 1. [OWASP Top 10 for LLM Applications 2025](https://genai.owasp.org/llm-top-10/) (published Nov 2024; a 2026 edition followed in Aug 2026) 2. [The lethal trifecta for AI agents — Simon Willison, 16 Jun 2025](https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/) 3. [OWASP Top 10 for Agentic Applications for 2026 — 9 Dec 2025](https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/) 4. [Spracklen et al., "We Have a Package for You!" — USENIX Security 2025](https://www.usenix.org/conference/usenixsecurity25/presentation/spracklen) 5. [Replit AI agent deletes production database — The Register, 21 Jul 2025](https://www.theregister.com/2025/07/21/replit_saastr_vibe_coding_incident/) ## Related - [[Prompt Injection]] · [[Agents and Connectors]] · [[AI Supply Chain Risks]] · [[AI Data Exposure]] · [[AI-Powered Scams]] - [[Claude Fable 5]] — the model behind this vault's own AI sessions - [[The Home System]] — a private working vault, and the check between it and this public one