Barrchives How Vanta Helps Build Secure and Compliant AI Products, with Christina Cacioppo and Iccha Sethi
15 snips
Apr 8, 2025 
In this engaging discussion, Christina Cacioppo, Co-founder and CEO of Vanta, and Iccha Sethi, VP of Engineering, share their insights into the intersection of AI with security and compliance. They explore how compliance serves as an ideal landscape for AI innovation and reveal the importance of building reliable, explainable systems. The duo discusses their innovative approach to using large language models for automating security questionnaires and maintaining 'golden datasets,' all while emphasizing the crucial human oversight needed to foster trust in automated processes.
AI Snips
Chapters
Transcript
Episode notes
Subtle User Signals Guide AI Improvement
- User feedback comes from subtle signals like answer acceptance, editing, and rejection, beyond just thumbs up/down.
- This detailed feedback guides dataset updates, prompts adjustments, and product improvements continuously.
Optimize Document Chunking for Retrieval
- Experiment with different document chunk sizes for embeddings to optimize recall and precision.
- Semantic variable chunking works best for balancing model input and embedding quality in compliance documents.
Golden Dataset Complexity Revealed
- Building golden datasets for AI is a challenging, ongoing effort requiring continuous updates.
- Real data and corner cases make even seemingly easy AI tasks complex in compliance environments.