Feature Stores at Shopify and Skyscanner // Matt Delacour and Mike Moran // Reading Group #4

MLOps Reading Group meeting on February 11, 2022  

Join the Community: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTJoinIn⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

Get the newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTNewsletter⁠⁠⁠⁠

Reading Group Session about Feature Stores with Matt Delacour and Mike Moran  

--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Connect with us on LinkedIn: https://www.linkedin.com/company/mlopscommunity/
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, Feature Store, Machine Learning Monitoring, and Blogs: https://mlops.community/

Timestamps:
[00:05] Matt's intro
[00:26] Mike's intro
[01:09] Matt’s talk: Feature store system at Shopify
[01:45] What is Shopify?
[02:05] Shopify Use Case
[02:38] Choosing a solution
[03:19] Managed service vs In-house vs Open-source (Feast)
[06:01] Why did we choose Feast?
[11:25] Implementation Strategy (multi-repo vs mono-repo approaches)
[13:01] Mono-repo approach breakdown
[14:30] Internal SDK
[17:01] Q&A: Does Feast satisfy scalability for online inference of Shopify's latency requirements?
[19:05] Q&A: Do you rely on Feast to serialize data to the online store?
[20:13] Q&A: Is your mono-repo library a subset of Feast?
[21:18] Q&A: Did you consider using git submodules for a multi-repo?
[23:02] Q&A: Are you storing embeddings with Feast?
[24:30] Q&A: Regarding the mono-repo, which modules are responsible for feature engineering? How do you guarantee that different feature engineering can be used across many DS?
[27:58] Mike’s talk (Feature store at Skyscanner)
[28:08] Kaleidoscope System
[28:25] Background and context of the Feature store
[29:30] Initial state of the feature store
[30:13] How does the marketing team also leverage the feature store
[31:04] Current state of the feature store (marketing & machine learning)
[31:44] SDK approach of creating schemas with dataframes (easy access)
[32:16] Reusability across teams among the marketing and DS team
[33:06] GDPR constraints
[33:34] Data updates at the feature store
[36:09] Q&A: When a DS updates a feature, how are you communicating that across teams?
[38:25] Q&A: Are you applying different levels of feature engineering to increase the likelihood of a DS going back to a previous checkpoint of processing?
[40:55] Q&A: In what languages are you implementing the feature store?
[44:28] Q&A: Regarding performance-wise, how do you decide what code remains in Apache Spark vs SQL?
[49:00] Wrap-up