Amelia & Filip — How Pandora Deploys ML Models into Production

07/01/21 • 40 min

Amelia and Filip give insights into the recommender systems powering Pandora, from developing models to balancing effectiveness and efficiency in production. --- Amelia Nybakke is a Software Engineer at Pandora. Her team is responsible for the production system that serves models to listeners. Filip Korzeniowski is a Senior Scientist at Pandora working on recommender systems. Before that, he was a PhD student working on deep neural networks for acoustic and language modeling applied to musical audio recordings. Connect with Amelia and Filip: 📍 Amelia's LinkedIn: https://www.linkedin.com/in/amelia-nybakke-60bba5107/ 📍 Filip's LinkedIn: https://www.linkedin.com/in/filip-korzeniowski-28b33815a/ --- ⏳ Timestamps: 0:00 Sneak peek, intro 0:42 What type of ML models are at Pandora? 3:39 What makes two songs similar or not similar? 7:33 Improving models and A/B testing 8:52 Chaining, retraining, versioning, and tracking models 13:29 Useful development tools 15:10 Debugging models 18:28 Communicating progress 20:33 Tuning and improving models 23:08 How Pandora puts models into production 29:45 Bias in ML models 36:01 Repetition vs novelty in recommended songs 38:01 The bottlenecks of deployment 🌟 Transcript: http://wandb.me/gd-amelia-and-filip 🌟 Links: 📍 Amelia's "Women's History Month" playlist: https://www.pandora.com/playlist/PL:1407374934299927:100514833 --- Get our podcast on these platforms: 👉 Apple Podcasts: http://wandb.me/apple-podcasts 👉 Spotify: http://wandb.me/spotify 👉 Google Podcasts: http://wandb.me/google-podcasts 👉 YouTube: http://wandb.me/youtube 👉 Soundcloud: http://wandb.me/soundcloud Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

Previous Episode

Luis Ceze — Accelerating Machine Learning Systems

From Apache TVM to OctoML, Luis gives direct insight into the world of ML hardware optimization, and where systems optimization is heading. --- Luis Ceze is co-founder and CEO of OctoML, co-author of the Apache TVM Project, and Professor of Computer Science and Engineering at the University of Washington. His research focuses on the intersection of computer architecture, programming languages, machine learning, and molecular biology. Connect with Luis: 📍 Twitter: https://twitter.com/luisceze 📍 University of Washington profile: https://homes.cs.washington.edu/~luisceze/ --- ⏳ Timestamps: 0:00 Intro and sneak peek 0:59 What is TVM? 8:57 Freedom of choice in software and hardware stacks 15:53 How new libraries can improve system performance 20:10 Trade-offs between efficiency and complexity 24:35 Specialized instructions 26:34 The future of hardware design and research 30:03 Where does architecture and research go from here? 30:56 The environmental impact of efficiency 32:49 Optimizing and trade-offs 37:54 What is OctoML and the Octomizer? 42:31 Automating systems design with and for ML 44:18 ML and molecular biology 46:09 The challenges of deployment and post-deployment 🌟 Transcript: http://wandb.me/gd-luis-ceze 🌟 Links: 1. OctoML: https://octoml.ai/ 2. Apache TVM: https://tvm.apache.org/ 3. "Scalable and Intelligent Learning Systems" (Chen, 2019): https://digital.lib.washington.edu/researchworks/handle/1773/44766 4. "Principled Optimization Of Dynamic Neural Networks" (Roesch, 2020): https://digital.lib.washington.edu/researchworks/handle/1773/46765 5. "Cross-Stack Co-Design for Efficient and Adaptable Hardware Acceleration" (Moreau, 2018): https://digital.lib.washington.edu/researchworks/handle/1773/43349 6. "TVM: An Automated End-to-End Optimizing Compiler for Deep Learning" (Chen et al., 2018): https://www.usenix.org/system/files/osdi18-chen.pdf 7. Porcupine is a molecular tagging system introduced in "Rapid and robust assembly and decoding of molecular tags with DNA-based nanopore signatures" (Doroschak et al., 2020): https://www.nature.com/articles/s41467-020-19151-8 --- Get our podcast on these platforms: 👉 Apple Podcasts: http://wandb.me/apple-podcasts 👉 Spotify: http://wandb.me/spotify 👉 Google Podcasts: http://wandb.me/google-podcasts 👉 YouTube: http://wandb.me/youtube 👉 Soundcloud: http://wandb.me/soundcloud Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

Next Episode

Roger & DJ — The Rise of Big Data and CA's COVID-19 Response

Roger and DJ share some of the history behind data science as we know it today, and reflect on their experiences working on California's COVID-19 response. --- Roger Magoulas is Senior Director of Data Strategy at Astronomer, where he works on data infrastructure, analytics, and community development. Previously, he was VP of Research at O'Reilly and co-chair of O'Reilly's Strata Data and AI Conference. DJ Patil is a board member and former CTO of Devoted Health, a healthcare company for seniors. He was also Chief Data Scientist under the Obama administration and the Head of Data Science at LinkedIn. Roger and DJ recently volunteered for the California COVID-19 response, and worked with data to understand case counts, bed capacities and the impact of intervention. Connect with Roger and DJ: 📍 Roger's Twitter: https://twitter.com/rogerm 📍 DJ's Twitter: https://twitter.com/dpatil --- 🌟 Transcript: http://wandb.me/gd-roger-and-dj 🌟 ⏳ Timestamps: 0:00 Sneak peek, intro 1:03 Coining the terms "big data" and "data scientist" 7:12 The rise of data science teams 15:28 Big Data, Hadoop, and Spark 23:10 The importance of using the right tools 29:20 BLUF: Bottom Line Up Front 34:44 California's COVID response 41:21 The human aspects of responding to COVID 48:33 Reflecting on the impact of COVID interventions 57:06 Advice on doing meaningful data science work 1:04:18 Outro 🍀 Links: 1. "MapReduce: Simplified Data Processing on Large Clusters" (Dean and Ghemawat, 2004): https://research.google/pubs/pub62/ 2. "Big Data: Technologies and Techniques for Large-Scale Data" (Magoulas and Lorica, 2009): https://academics.uccs.edu/~ooluwada/courses/datamining/ExtraReading/BigData 3. The O'RLY book covers: https://www.businessinsider.com/these-hilarious-memes-perfectly-capture-what-its-like-to-work-in-tech-2016-4 4. "The Premonition" (Lewis, 2021): https://www.npr.org/2021/05/03/991570372/michael-lewis-the-premonition-is-a-sweeping-indictment-of-the-cdc 5. Why California's beaches are glowing with bioluminescence: https://www.youtube.com/watch?v=AVYSr19ReOs 6. 7. Sturgis Motorcyle Rally: https://en.wikipedia.org/wiki/Sturgis_Motorcycle_Rally --- Get our podcast on these platforms: 👉 Apple Podcasts: http://wandb.me/apple-podcasts 👉 Spotify: http://wandb.me/spotify 👉 Google Podcasts: http://wandb.me/google-podcasts 👉 YouTube: http://wandb.me/youtube 👉 Soundcloud: http://wandb.me/soundcloud Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected