Skip to content

AI Security

Every piece of text the model reads is a potential instruction.

prompt injectionindirect prompt injectionjailbreakingtool abusedata exfiltrationRAG poisoningleast privilegehuman confirmation

Lessons

  1. 29.1AI security: assume the model will be hijackedAn AI model can be talked into things by any text it reads. So you build the system so that even a fooled model can't do real harm. 30 min Prompt injection lab
  2. 29.2Indirect prompt injection: when the data talks backSomeone hides instructions in a web page or email. Weeks later, your agent reads it for an innocent user. Here's how to contain that. 30 min Prompt injection lab
  3. 29.3RAG poisoning: when your knowledge base lies on purposeIf anyone can add documents your assistant searches, anyone can put words in its mouth. Here's how to keep the knowledge base honest. 30 min Prompt injection lab
  4. 29.4Agent permissions: designing what an agent is allowed to doWhatever your agent is allowed to do, someone can talk it into doing. So you decide carefully what it's allowed to do. 30 min Pick the tools Prompt injection lab

On the job: try it on your own project

Pick one AI feature that reads outside content (emails, web pages, documents, user uploads) or can take actions. Ask Claude Code to map every input it reads and every tool it can call. Then try three harmless prompt-injection tests in a test environment. Write a short threat note: what an attacker could make it do, and the permission limits you will add.

Can you do this now?

0/4. Anything you can't tick yet: reopen that lesson's practice ladder.