The Cost of Storeage and Data Warehouse Systems

The cost of storage and data ware house systems that we have now are just, like, so incredibly a performintjust that makes more sense. Lok: We cannot still have this idea that you never override things. The old way is kind of wrong, because you are deleting the past in a way that is not undoable. You want to ask what was the price of this product every day of the year for the last year? Well, if you keep on a bathing staff, because you're constrained by the size of the databisy, you can run. Itt was a beautiful example. Itsn down to me it because this actually came up in one of the channels in slack

Play episode from 27:24

chevron_right

Transcript

chevron_right

Transcript

Episode notes

MLOps Coffee Sessions #66 with Jacopo Tagliabue, Machine Learning at Reasonable Scale.

Get the newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTNewsletter

// Abstract
We believe that immature data pipelines are preventing a large portion of industry practitioners from leveraging the latest research on ML: the truth is, outside of Big Tech and advanced startups, ML systems are still far from producing the promised ROI.

The good news is that times are changing: thanks to a growing ecosystem of tools and shared best practices, even small teams can be incredibly productive at a “reasonable scale”. Based on our experience as founders and researchers, we present our philosophy for modern, no-nonsense data pipelines, highlighting the advantages of a "PaaS-like" approach.

// Bio
Educated in several acronyms across the globe (UNISR, SFI, MIT), Jacopo Tagliabue was co-founder and CTO of Tooso, an A.I. company in San Francisco acquired by Coveo in 2019. Jacopo is currently the Director of AI at Coveo, shipping models to hundreds of customers and millions of users. When not busy building products, he is exploring topics at the intersection of language, reasoning, and learning: his research and industry work are often featured in the general press and premier A.I. venues. In previous lives, he managed to get a Ph.D., do sciency things for a pro basketball team, and simulate a pre-Columbian civilization.

// Relevant Links
Bigger boat repo: https://github.com/jacopotagliabue/you-dont-need-a-bigger-boat
TDS series: https://towardsdatascience.com/tagged/mlops-without-much-ops (ep 3 and a NEW open-source contribution on data ingestion coming up)
Open datasets for e-commerce and MLops experiments: https://github.com/coveooss/SIGIR-ecom-data-challenge

--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, Feature Store, Machine Learning Monitoring, and Blogs: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Jacopo on LinkedIn: https://www.linkedin.com/in/jacopotagliabue/

Timestamps:

[00:00] Introduction to Jacopo Tagliabue

[01:35] What Reasonable Scale means

[06:40] Biggest disconnects from Reasonable Scale

[12:32] Engineers need to do and tools to use at a Reasonable Scale

[15:25] Importance of maintenance

[17:27] Bigger boat repo demonstration of Reasonable Scale

[23:09] The Four Pillars

[27:27] ETL Paradigm

[30:16] Best practices around dragons in generic decisions and comparing the new outputs and saved snapshots

[33:32] Creating a knowledge hub