The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Advances in Neural Compression with Auke Wiggers - #570
May 2, 2022
Auke Wiggers, an AI research scientist at Qualcomm, dives into the exciting realm of neural data compression. He discusses how generative models and transformer architectures are revolutionizing image and video coding. The conversation highlights the shift from traditional techniques to neural codecs that learn from examples, and the impressive real-time performance on mobile devices. Auke also touches on innovations like transformer-based transform coding and shares insights from recent ICLR papers, showcasing the future of efficient data compression.
AI Snips
Chapters
Transcript
Episode notes
Neural vs. Traditional Codecs
- Neural codecs borrow heavily from traditional coding concepts like motion compensation.
- They replace some operations of handcrafted codecs for better rate-distortion performance.
Perceptual Quality Evaluation
- Evaluating perceptual quality ideally involves user studies, but proxy metrics like VMAF and FID are used due to cost.
- These metrics offer decent approximations before conducting more expensive user studies.
Vision Transformers in Compression
- Vision transformers, treating images as collections of patches, are used in compression.
- Swin transformers, more memory-efficient than original ViTs, are particularly useful for large image/video compression.