Ep139: Human-in-the-Loop by Design: Building AI Systems Responsibly

Sep 1, 2025

Guest

Kayvan Alikhani

Guest

Umberto Milletti

Guest

Oliver Sharp

Join Oliver Sharp, Chief AI Officer at Highspot; Umberto Milletti, R&D Officer at Demandbase; and Kayvan Alikhani, Global Head at Archer, as they dive into the future of AI governance. They discuss the importance of keeping humans in the loop and tackling data privacy challenges. Expect insights on managing compliance across different regions, reducing bias in AI through detailed assessments, and prioritizing expert review for accountability. With a focus on building transparent AI systems, this panel sheds light on responsible AI development.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

EU AI Act Shapes Global Governance

The EU AI Act's risk-based principles are becoming a de facto governance framework even for U.S. companies.
Firms now expect AI solutions to be auditable, explainable, and correctable under these standards.

ADVICE

Mandate Justifications And Expert Reviews

Publish AI justifications and pit multiple LLMs against each other to validate outputs and reduce single-model risk.
Implement expert-in-the-loop reviews for mid-to-high risk results and surface review percentages to clients.

ADVICE

Address The Risk Of AI Inaction

Train and re-skill employees quickly: laggards who don't adopt AI risk obsolescence.
Be honest with staff about AI-driven role changes and redefine job expectations across levels.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

AI executives from Archer, Demandbase and Highspot and AWS reveal how they're tackling AI's biggest challenges—from securing data, managing regulatory changes and keeping humans in the loop.

Topics Include:

Three AI leaders introduce their companies: Archer, Demandbase and Highspot's approaches to enterprise AI
Demandbase's data strategy: Customer data stays isolated, shared data requires consent, public sources fuel training
Geographic complexity: AI compliance varies dramatically between Germany, US, Canada, and California regulations
HighSpot tackles sales bias: Granular questions replace generic assessments for more accurate rep evaluations
SBI framework applied to AI: Specific behavioral observations create better, more actionable sales coaching
AI transparency through citations: Timestamped evidence lets managers verify AI feedback and catch hallucinations
Archer handles 20-30K monthly regulations: AI helps enterprises manage overwhelming compliance requirements at scale
Two compliance types explained: Operational (common across companies) versus business-specific regulatory requirements
EU AI Act adoption: US companies embracing European framework for responsible AI governance
Human oversight becomes mandatory: Expert-in-the-loop reviews ensure AI decisions remain correctable and auditable
The bigger AI risk: Companies face greater danger from AI inaction than AI adoption
Agentic AI security challenges: Data layers must enforce permissions before AI access, not after
AI agents need identity management: Same access controls apply whether human clicks or AI acts
Human oversight in high stakes: Chief compliance officers demand transparency and correction capabilities
Future challenge identified: 80% of enterprise data behind firewalls remains invisible to AI models

Participants:

Kayvan Alikhani - Global Head of Engineering- Emerging Solutions, Archer Integrated Risk Management
Umberto Milletti - Chief R&D Officer, Demandbase
Oliver Sharp - Co-Founder & Chief AI Officer, Highspot
Brian Shadpour - General Manager, Security, Amazon Web Services

Further Links:

Archer Integrated Risk Management: Website – LinkedIn – AWS Marketplace
Demandbase: Website – LinkedIn – AWS Marketplace
Highspot: Website – LinkedIn – AWS Marketplace

See how Amazon Web Services gives you the freedom to migrate, innovate, and scale your software company at https://aws.amazon.com/isv/