Vanishing Gradients Episode 11: Data Science: The Great Stagnation
01:45:53
Failed RL Startup For Game AI
- Mark started a startup building RL tools for game developers and struggled with adoption and technical readiness.
- The venture failed partly because games needed domain changes and customers weren't receptive.
Favor Lindy Tools Over Shiny Releases
- Prioritize enduring tools (Lindy) over chasing every new library.
- Invest time in SQL, Python, and stable tools that have long-term value.
Docs Should Produce Early Wins
- Design docs to give immediate wins that make users feel smart.
- Then progressively disclose complexity so users can learn without feeling stuck.
Get the Snipd Podcast app to discover more snips from this episode
Get the app 1 chevron_right 2 chevron_right 3 chevron_right 4 chevron_right 5 chevron_right 6 chevron_right 7 chevron_right 8 chevron_right 9 chevron_right 10 chevron_right 11 chevron_right 12 chevron_right 13 chevron_right 14 chevron_right 15 chevron_right 16 chevron_right 17 chevron_right 18 chevron_right 19 chevron_right 20 chevron_right 21 chevron_right 22 chevron_right 23 chevron_right 24 chevron_right 25 chevron_right 26 chevron_right 27 chevron_right 28 chevron_right 29 chevron_right 30 chevron_right 31 chevron_right 32 chevron_right 33 chevron_right 34 chevron_right 35 chevron_right 36 chevron_right 37 chevron_right 38 chevron_right 39 chevron_right 40 chevron_right 41 chevron_right
Introduction
00:00 • 2min
I'm a Programmer, but I've Always Been Interested in Computers
01:36 • 2min
What Prompted You to Go From Neuro Science to Product?
03:10 • 3min
What Happened After Micro Soft?
05:53 • 4min
Doda
09:35 • 3min
Interviewing Hirig Managers and Not Risktakers
12:24 • 2min
Are You Trying to Hire People Who May Fail?
14:07 • 3min
The Great Stagnation
16:59 • 3min
Why Is Open Source Important to Meter?
20:24 • 2min
Open Source Software - What's Your Interest in Open Source?
22:21 • 4min
Deaf Tooling
26:17 • 2min
The Longer You're Alive, the More Chance You Are of Living Longer
28:43 • 3min
The Power of Open Source
31:30 • 4min
What Can We Learn From Video Games?
35:07 • 4min
Do You Really Need Distributed Systems in Your Date?
39:23 • 5min
Machine Learning, the Great Stagnation
44:39 • 2min
Taking a Risk With Machine Learning Researches
46:30 • 4min
Deep Learning
50:15 • 3min
I'm Kind of Jealous of Taleb on Twitter
53:08 • 1min
Are You Taking Risks?
54:38 • 2min
Working Class Deep Learner Opening
56:50 • 4min
What Are the Archetypes of Deep Learning?
01:01:01 • 3min
What Is Value and How Do We Create It?
01:03:50 • 2min
I've a Totally Different, Like, an Addenden to That.
01:05:23 • 2min
What's the Best Strategy to Maximise Wealth?
01:06:58 • 3min
Hugging Face
01:09:32 • 3min
The Rise of Hugging Face
01:12:10 • 3min
I Wont to the Undertaker That You Were Boot Legging Him in the Nights.
01:15:09 • 2min
Word Sell and Shape Rotator
01:16:51 • 2min
Is There a Capab Around Machine Learning?
01:18:38 • 2min
Using a Motor Round the Castle
01:20:14 • 3min
Do You Have a Product Marketing Strategy?
01:23:28 • 2min
Are We Going to Have a Hybrid Marketer?
01:25:57 • 2min
Is It a Good Idea?
01:27:34 • 2min
Scarcity Is One Kind of Primitive Signal We Have, Right?
01:29:29 • 2min
Open Source Is Better Than Open Source
01:31:45 • 2min
Cloud Providers as Landowners
01:33:59 • 2min
Cryptocurrency - The One Coat History and the Security of Property
01:35:47 • 3min
Machine Learning - What Are You Excited About?
01:38:30 • 2min
Is There a Uscase for Digital Scarcity?
01:40:51 • 3min
Machine Learning - Is There a Job Market?
01:43:31 • 2min
Hugo speaks with Mark Saroufim, an Applied AI Engineer at Meta who works on PyTorch where his team’s main focus is making it as easy as possible for people to deploy PyTorch in production outside Meta.
Mark first came on our radar with an essay he wrote called Machine Learning: the Great Stagnation (https://marksaroufim.substack.com/p/machine-learning-the-great-stagnation), which was concerned with the stagnation in machine learning in academic research and in which he stated
Machine learning researchers can now engage in risk-free, high-income, high-prestige work. They are today’s Medieval Catholic priests.
This is just the tip of the icebergs of Mark’s critical and often sociological eye and one of the reasons I was excited to speak with him.
In this conversation, we talk about the importance of open source software in modern data science and machine learning and how Mark thinks about making it as easy to use as possible. We also talk about risk assessments in considering whether to adopt open source or not, the supreme importance of good documentation, and what we can learn from the world of video game development when thinking about open source.
We then dive into the rise of the machine learning cult leader persona, in the context of examples such as Hugging Face and the community they’ve built. We discuss the role of marketing in open source tooling, along with for profit data science and ML tooling, how it can impact you as an end user, and how much of data science can be considered differing forms of live action role playing and simulation.
We also talk about developer marketing and content for data professionals and how we see some of the largest names in ML researchers being those that have gigantic Twitter followers, such as Andrei Karpathy. This is part of a broader trend in society about the skills that are required to capture significant mind share these days.
If that’s not enough, we jump into how machine learning ideally allows businesses to build sustainable and defensible moats, by which we mean the ability to maintain competitive advantages over competitors to retain market share.
In between this interview and its release, PyTorch joined the Linux Foundation, which is something we’ll need to get Mark back to discuss sometime.
Links
The Myth of Objective Tech Screens (https://marksaroufim.substack.com/p/the-myth-of-objective-tech-screens)
Machine Learning: The Great Stagnation (https://marksaroufim.substack.com/p/machine-learning-the-great-stagnation)
Fear the Boom and Bust: Keynes vs. Hayek - The Original Economics Rap Battle! (https://www.youtube.com/watch?v=d0nERTFo-Sk)
History and the Security of Property (https://archive.ph/dRXEK#selection-21.0-21.36) by Nick Szabo
Mark on YouTube (https://www.youtube.com/marksaroufim)
Mark's Substack (https://marksaroufim.substack.com/p/machine-learning-the-great-stagnation)
Mark's Discord (https://discord.com/invite/drmuTjWZrm)
Get full access to Vanishing Gradients at hugobowne.substack.com/subscribe
Mark first came on our radar with an essay he wrote called Machine Learning: the Great Stagnation (https://marksaroufim.substack.com/p/machine-learning-the-great-stagnation), which was concerned with the stagnation in machine learning in academic research and in which he stated
Machine learning researchers can now engage in risk-free, high-income, high-prestige work. They are today’s Medieval Catholic priests.
This is just the tip of the icebergs of Mark’s critical and often sociological eye and one of the reasons I was excited to speak with him.
In this conversation, we talk about the importance of open source software in modern data science and machine learning and how Mark thinks about making it as easy to use as possible. We also talk about risk assessments in considering whether to adopt open source or not, the supreme importance of good documentation, and what we can learn from the world of video game development when thinking about open source.
We then dive into the rise of the machine learning cult leader persona, in the context of examples such as Hugging Face and the community they’ve built. We discuss the role of marketing in open source tooling, along with for profit data science and ML tooling, how it can impact you as an end user, and how much of data science can be considered differing forms of live action role playing and simulation.
We also talk about developer marketing and content for data professionals and how we see some of the largest names in ML researchers being those that have gigantic Twitter followers, such as Andrei Karpathy. This is part of a broader trend in society about the skills that are required to capture significant mind share these days.
If that’s not enough, we jump into how machine learning ideally allows businesses to build sustainable and defensible moats, by which we mean the ability to maintain competitive advantages over competitors to retain market share.
In between this interview and its release, PyTorch joined the Linux Foundation, which is something we’ll need to get Mark back to discuss sometime.
Links
The Myth of Objective Tech Screens (https://marksaroufim.substack.com/p/the-myth-of-objective-tech-screens)
Machine Learning: The Great Stagnation (https://marksaroufim.substack.com/p/machine-learning-the-great-stagnation)
Fear the Boom and Bust: Keynes vs. Hayek - The Original Economics Rap Battle! (https://www.youtube.com/watch?v=d0nERTFo-Sk)
History and the Security of Property (https://archive.ph/dRXEK#selection-21.0-21.36) by Nick Szabo
Mark on YouTube (https://www.youtube.com/marksaroufim)
Mark's Substack (https://marksaroufim.substack.com/p/machine-learning-the-great-stagnation)
Mark's Discord (https://discord.com/invite/drmuTjWZrm)
Get full access to Vanishing Gradients at hugobowne.substack.com/subscribe