Irwan Bello

Irwan Bello, a research scientist formerly with Google Brain and now part of a stealth AI startup, dives into the world of sparse expert models. He discusses his recent work on designing effective architectures that improve performance while managing computational costs. The conversation uncovers how the mixture-of-experts technique can extend beyond NLP to various tasks, including vision. Bello also shares insights on enhancing alignment in language models through instruction tuning and the challenges of optimizing these large-scale systems.