Aligning Multiple Modalities Later

Tanishq Abraham, CEO and co-founder of Sophont.ai, joins us to talk about building foundation models specifically for medicine.

Sophont is trying to be something like an OpenAI or Anthropic but for healthcare  - training models across pathology, neuroimaging, and clinical text, to eventually fuse them into one multimodal system. The surprising part: their pathology model trained on 12,000 public slides performs on par with models trained on millions of private ones. Data quality beats data quantity.

We talk about what actually excites Tanishq, which is not replacing doctors, but finding things doctors can't see. AI predicting gene mutations from a tissue slide, or cardiovascular risk from an eye scan.

We also talk about the regulation and how the picture is less scary than people assume. Text-based clinical decision support can ship without FDA approval. Pharma partnerships offer near-term impact. The five-to-ten-year timeline people fear is really about drug discovery, not all of medical AI.

Takeaways:

• The real promise of medical AI is finding hidden signals in existing data, not just automating doctors
• Small, curated public datasets can rival massive private ones
• Multimodal fusion is the goal, but you need strong individual encoders first
• AI research itself might get automated sooner than biology or chemistry
• FDA regulation has more flexibility than most people think

Timeline

(00:12) Introduction and guest welcome

(02:32) Anthropic's ad about ChatGPT ads

(07:26) XAI merging into SpaceX

(13:32) Vibe coding one year later

(17:00) Claude Code and agentic workflows

(21:52) Can AI automate AI research?

(26:57) What is medical AI

(31:06) Sofont as a frontier medical AI lab

(33:52) Public vs. private data - 12K slides vs. millions

(36:43) Domain expertise vs. scaling

(41:54) Cancer, diabetes, and personal stakes

(47:52) Classification vs. prediction in medicine

(50:36) When doctors disagree

(54:43) Quackery and AI

(57:15) Uncertainty in medical AI

(1:03:11) Will AI replace doctors?

(1:07:24) Self-supervised learning on sleep data

(1:10:10) Aligning modalities

(1:13:17) FDA regulation

(1:22:28) Closing

Music:

• "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
• "Palms Down" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

Changes: trimmed

About

The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.