AI Fundamentals and AI Hacking 101

Course Duration

2 days

Audience

Employees of federal, state and local governments; and businesses working with the government.

Prerequisites

A preliminary understanding of penetration testing methodology is suggested.

Course Description

The AI Fundamentals and AI Hacking 101 course teaches students the fundamentals of how AI works under the hood and then how to break it. The first day of the course focuses on the fundamentals of how AI works, covering neural networks, natural language processing, large language models, and self-hosted LLMs. The hacking portion focuses on penetration testing AI/LLM-based applications such as customer-facing chatbots, demonstrating how to detect and exploit common AI vulnerabilities including prompt injection, sensitive information disclosure, improper output handling, system prompt leakage, misinformation, and excessive agency. Students spend hands-on time in a custom-built lab environment training their own neural networks, tweaking LLMs, and exploiting vulnerabilities using the TCM Vulnerable Chatbot — a customer service chatbot with Retrieval Augmented Generation (RAG) capabilities.

Course Outline

1 – Introduction to Neural Networks

• How neural networks function, the math behind them, and how they are trained
• Neural network lab: train a network to perform basic image recognition of numerals

2 – Natural Language Processing

• What NLP is and how it works
• Word vectors and a word2vec lab and visualization
• Neural network bigrams and trigrams
• Recurrent neural networks (RNN)

3 – Large Language Models

• Evolution of NLP to the LLM and the transformer decoder architecture
• LLM attention and how the attention mechanism adds context
• Self-hosting LLMs using Ollama and interacting with them programmatically
• Writing a basic chatbot and interacting with AI APIs via scripting

4 – AI Threat Modeling

• AI Fundamentals Review
• AI Threat Model: threat actors, assets, adversary goals, and attack surfaces
• Reconnaissance, model mapping, baseline behavior, and fingerprinting

5 – Prompt Injection and Jailbreaking

• Common prompt injection and jailbreaking techniques
• Prompt injection tools and repositories
• Bypassing common input/output filtering protections

6 – AI Application Exploitation

• Testing for harmful output, hate speech, misinformation, and resource drainage
• Data exfiltration via Retrieval Augmented Generation (RAG)
• RAG and vector database attacks
• Excessive agency exploitation and testing