AI Hacking 101
Course Duration
1 day
Audience
Employees of federal, state and local governments; and businesses working with the government.
Prerequisites
A preliminary understanding of penetration testing methodology is suggested.
Course Description
The AI Hacking 101 course teaches students the fundamentals of penetration testing AI/LLM-based applications such as customer-facing chatbots. The course focuses on demonstrating how to detect and exploit common AI vulnerabilities including prompt injection, sensitive information disclosure, improper output handling, system prompt leakage, misinformation, and excessive agency. Students spend hands-on time in a custom-built lab environment exploiting and uncovering these vulnerabilities using the TCM Vulnerable Chatbot — a customer service chatbot with Retrieval Augmented Generation (RAG) capabilities.
Course Outline
1 – AI Fundamentals and Threat Modeling
- AI Fundamentals Review: model parameters, temperature, top-p, inference, training, LLMs
- AI Threat Model: threat actors, assets, adversary goals, and attack surfaces
- Reconnaissance, model mapping, baseline behavior, and fingerprinting
2 – Prompt Injection and Jailbreaking
- Common prompt injection and jailbreaking techniques
- Prompt injection tools and repositories
- Bypassing common input/output filtering protections
3 – AI Application Exploitation
- Testing for harmful output, hate speech, misinformation, and resource drainage
- Data exfiltration via Retrieval Augmented Generation (RAG)
- RAG and vector database attacks
- Excessive agency exploitation and testing