Practical AI Threat modeling LLM apps
28 snips
Aug 22, 2024 
Donato Capitella, Principal Security Consultant at WithSecure, specializes in threat modeling for AI applications. He discusses the critical need for threat modeling in the context of large language models (LLMs) and shares insights on vulnerabilities, such as prompt injection risks. Donato emphasizes the importance of validating outputs to maintain trustworthiness and explores innovative strategies for secure integration in AI systems. The conversation also touches on the exciting future of LLM technology and the role of ethical hackers in enhancing cybersecurity.
AI Snips
Chapters
Transcript
Episode notes
Untrusted LLM Output
- Treat LLM output as untrusted, similar to handling emails from unknown senders.
- Apply security controls to mitigate risks from this untrusted data, verifying and validating information before acting on it.
LLM Security Canvas
- Donato Capitella's LLM Security Canvas focuses on input and output controls for secure LLM application deployment.
- It prioritizes output validation, checking for harmful content and format, especially in rendered outputs like Markdown or HTML.
Input Validation
- Implement input validation using semantic routing, topical guardrails, and prompt injection detection models.
- Check input length, format, and character set, especially considering potential exploits with low-resource languages.