
Unsolicited Feedback Evaluating AI Models, Unlocking Unstructured Data, and Achieving Reliability w/ Ben Kus
39 snips
May 23, 2024 Tech expert and Box CTO, Ben Kus, discusses AI's impact on productivity and decision-making in organizations, strategies for ensuring reliability, challenges of AI change management, evaluating AI quality, unlocking unstructured data, and the potential of AI agents for user experience.
AI Snips
Chapters
Transcript
Episode notes
Learn By Probing Not Just Reading
- Read core AI research if you lead strategy, but you don't need to master it to build useful products.
- Learn by probing models and iterating prompts to discover their strengths and failure modes.
Bucket Models By Tier Not Name
- Models cluster into buckets by price, performance and trust attributes rather than unique names.
- Treat model choice as tiered (standard, premium, super-premium) to simplify product decisions.
Design For Uncertainty With Human Checks
- Expect nondeterminism and build human checks, gates, and review steps into AI workflows.
- Use humans to verify or oversee AI outputs until reliability is proven for full automation.

