Recsperts - Recommender Systems Experts

In episode number eight of Recsperts we discuss music recommender systems, the meaning of artist fairness and perspectives on recommender evaluation. I talk to Christine Bauer, who is an assistant professor at the University of Utrecht and co-organizer of the PERSPECTIVES workshop. Her research deals with context-aware recommender systems as well as the role of fairness in the music domain. Christine published work at many conferences like CHI, CHIIR, ICIS, and WWW.

In this episode we talk about the specifics of recommenders in the music streaming domain. In particular, we discuss the interests of different stakeholders, like users, the platform, or artists. Christine Bauer presents insights from her research on fairness with respect to the representation of artists and their interests. We talk about gender imbalance and how recommender systems could serve as a tool to counteract existing imbalances instead of reinforcing them, for example with simulations and reranking. In addition, we talk about the lack of multi-method evaluation and how open datasets incline researchers to focus too much on offline evaluation. In contrast, Christine argues for more user studies and online evaluation.

We wrap up with some final remarks on context-aware recommender systems and the potential of sensor data for improving context-aware personalization.

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.

Links from the Episode:

• Website of Christine Bauer
• Christine Bauer on LinkedIn
• Christine Bauer on Twitter
• PERSPECTIVES 2022: Perspectives on the Evaluation of Recommender Systems
• 5th FAccTRec Workshop: Responsible Recommendation

Papers:

• Ferraro et al. (2021): What is fair? Exploring the artists' perspective on the fairness of music streaming platforms
• Ferraro et. al (2021): Break the Loop: Gender Imbalance in Music Recommenders
• Jannach et al. (2020): Escaping the McNamara Fallacy: Towards More Impactful Recommender Systems Research
• Bauer et al. (2015): Designing a Music-controlled Running Application: a Sports Science and Psychological Perspective
• Dey et al. (2000): Towards a Better Understanding of Context and Context-Awareness

General Links:

• Follow me on Twitter: https://twitter.com/LivesInAnalogia
• Send me your comments, questions and suggestions to [email protected]
• Podcast Website: https://www.recsperts.com/
• (03:18) - Introducing Christine Bauer
• (09:08) - Multi-Stakeholder Interests in Music Recommender Systems
• (15:56) - Context-Aware Music Recommendations
• (21:55) - Fairness in Music RecSys
• (41:22) - Trade-Offs between Fairness and Relevance
• (48:18) - Evaluation Perspectives
• (01:02:37) - Further RecSys Challenges

In episode 24 of Recsperts, I sit down with Amey Dharwadker, Machine Learning Engineering Manager at Facebook, to dive into the complexities of large-scale video recommendations. Amey, who leads the Video Recommendations Quality Ranking team at Facebook, sheds light on the intricate challenges of delivering personalized video feeds at scale. Our conversation covers content understanding, user interaction data, real-time signals, exploration, and evaluation techniques.

We kick off the episode by reflecting on the inaugural VideoRecSys workshop at RecSys 2023, setting the stage for a deeper discussion on Facebook’s approach to video recommendations. Amey walks us through the critical challenges they face, such as gathering reliable user feedback signals to avoid pitfalls like watchbait. With a vast and ever-growing corpus of billions of videos—millions of which are added each month—the cold start problem looms large. We explore how content understanding, user feedback aggregation, and exploration techniques help address this issue. Amey explains how engagement metrics like watch time, comments, and reactions are used to rank content, ensuring users receive meaningful and diverse video feeds.

A key highlight of the conversation is the importance of real-time personalization in fast-paced environments, such as short-form video platforms, where user preferences change quickly. Amey also emphasizes the value of cross-domain data in enriching user profiles and improving recommendations.

Towards the end, Amey shares his insights on leadership in machine learning teams, pointing out the characteristics of a great ML team.

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.
Don't forget to follow the podcast and please leave a review

• (00:00) - Introduction
• (02:32) - About Amey Dharwadker
• (08:39) - Video Recommendation Use Cases on Facebook
• (16:18) - Recommendation Teams and Collaboration
• (25:04) - Challenges of Video Recommendations
• (31:07) - Video Content Understanding and Metadata
• (33:18) - Multi-Stage RecSys and Models
• (42:42) - Goals and Objectives
• (49:04) - User Behavior Signals
• (59:38) - Evaluation
• (01:06:33) - Cross-Domain User Representation
• (01:08:49) - Leadership and What Makes a Great Recommendation Team
• (01:13:01) - Closing Remarks

• Amey Dharwadker on LinkedIn
• Amey's Website
• RecSys Challenge 2021
• VideoRecSys Workshop 2023
• VideoRecSys + LargeRecSys 2024

Papers:

• Mahajan et al. (2023): CAViaR: Context Aware Video Recommendations
• Mahajan et al. (2023): PIE: Personalized Interest Exploration for Large-Scale Recommender Systems
• Raul et al. (2023): CAM2: Conformity-Aware Multi-Task Ranking Model for Large-Scale Recommender Systems
• Zhai et al. (2024): Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations
• Saket et al. (2023): Formulating Video Watch Success Signals for Recommendations on Short Video Platforms
• Wang et al. (2022): Surrogate for Long-Term User Experience in Recommender Systems
• Su et al. (2024): Long-Term Value of Exploration: Measurements, Findings and Algorithms

General Links:

• Follow me on LinkedIn
• Follow me on X
• Send me your comments, questions and suggestions to [email protected]
• Recsperts Website

In episode 23 of Recsperts, we welcome Yashar Deldjoo, Assistant Professor at the Polytechnic University of Bari, Italy. Yashar's research on recommender systems includes multimodal approaches, multimedia recommender systems as well as trustworthiness and adversarial robustness, where he has published a lot of work. We discuss the evolution of generative models for recommender systems, modeling paradigms, scenarios as well as their evaluation, risks and harms.

We begin our interview with a reflection of Yashar's areas of recommender systems research so far. Starting with multimedia recsys, particularly video recommendations, Yashar covers his work around adversarial robustness and trustworthiness leading to the main topic for this episode: generative models for recommender systems. We learn about their aspects for improving beyond the (partially saturated) state of traditional recommender systems: improve effectiveness and efficiency for top-n recommendations, introduce interactivity beyond classical conversational recsys, provide personalized zero- or few-shot recommendations.
We learn about the modeling paradigms and as well about the scenarios for generative models which mainly differ by input and modelling approach: ID-based, text-based, and multimodal generative models. This is how we navigate the large field of acronyms leading us from VAEs and GANs to LLMs.

Towards the end of the episode, we also touch on the evaluation, opportunities, risks and harms of generative models for recommender systems. Yashar also provides us with an ample amount of references and upcoming events where people get the chance to know more about GenRecSys.

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.
Don't forget to follow the podcast and please leave a review

• (00:00) - Introduction
• (03:58) - About Yashar Deldjoo
• (09:34) - Motivation for RecSys
• (13:05) - Intro to Generative Models for Recommender Systems
• (44:27) - Modeling Paradigms for Generative Models
• (51:33) - Scenario 1: Interaction-Driven Recommendation
• (57:59) - Scenario 2: Text-based Recommendation
• (01:10:39) - Scenario 3: Multimodal Recommendation
• (01:24:59) - Evaluation of Impact and Harm
• (01:38:07) - Further Research Challenges
• (01:45:03) - References and Research Advice
• (01:49:39) - Closing Remarks

• Yashar Deldjoo on LinkedIn
• Yashar's Website
• KDD 2024 Tutorial: Modern Recommender Systems Leveraging Generative AI: Fundamentals, Challenges and Opportunities
• RecSys 2024 Workshop: The 1st Workshop on Risks, Opportunities, and Evaluation of Generative Models in Recommender Systems (ROEGEN@RECSYS'24)

Papers:

• Deldjoo et al. (2024): A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)
• Deldjoo et al. (2020): Recommender Systems Leveraging Multimedia Content
• Deldjoo et al. (2021): A Survey on Adversarial Recommender Systems: From Attack/Defense Strategies to Generative Adversarial Networks
• Deldjoo et al. (2020): How Dataset Characteristics Affect the Robustness of Collaborative Recommendation Models
• Liang et al. (2018): Variational Autoencoders for Collaborative Filtering
• He et al. (2016): Visual Bayesian Personalized Ranking from Implicit Feedback

General Links:

• Follow me on LinkedIn
• Follow me on X
• Send me your comments, questions and suggestions to [email protected]
• Recsperts Website

This episode of Recsperts features Justin Basilico who is director of research and engineering at Netflix. Justin leads the team that is in charge of creating a personalized homepage. We learn more about the evolution of the Netflix recommender system from rating prediction to using deep learning, contextual multi-armed bandits and reinforcement learning to perform personalized page construction. Deep content understanding drives the creation of useful groupings of videos to be shown in a personalized homepage.

Justin and I discuss the misalignment of metrics as just one out of many elements that is making personalization still “super hard”. We hear more about the journey of deep learning for recommender systems where real usefulness comes from taking advantage of the variety of data besides pure user-item interactions, i.e. histories, content, and context. We also briefly touch on RecSysOps for detecting, predicting, diagnosing and resolving issues in a large-scale recommender systems and how it helps to alleviate item cold-start.

In the end of this episode, we talk about the company culture at Netflix. Key elements are freedom and responsibility as well as providing context instead of exerting control. We hear that being really comfortable with feedback is important for high-performance people and teams.

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.

Chapters:

• (03:13) - Introduction Justin Basilico
• (07:37) - Evolution of the Netflix Recommender System
• (22:28) - Page Construction of the Personalized Netflix Homepage
• (32:12) - Misalignment of Metrics
• (37:36) - Experience with Deep Learning for Recommender Systens
• (48:10) - RecSysOps for Issue Detection, Diagnosis and Response
• (55:38) - Bandits Recommender Systems
• (01:03:22) - The Netflix Culture
• (01:13:33) - Further Challenges
• (01:15:48) - RecSys 2023 Industry Track
• (01:17:25) - Closing Remarks

• Justin Basilico on Linkedin
• Justin Basilico on Twitter
• Netflix Research Publications
• The Netflix Tech Blog
• CONSEQUENCES+REVEAL Workshop at RecSys 2022
• Learning a Personalized Homepage (Alvino et al., 2015)
• Recent Trends in Personalization at Netflix (Basilico, 2021)
• RecSysOps: Best Practices for Operating a Large-Scale Recommender System (Saberian et al., 2022)
• Netflix Fourth Quarter 2022 Earnings Interview
• No Rules Rules - Netflix and the Culture of Reinvention (Hastings et al., 2020)
• Job Posting for Netflix' Recommendation Team

Papers:

• Steck et al. (2021): Deep Learning for Recommender Systems: A Netflix Case Study
• Steck et al. (2021): Negative Interactions for Improved Collaborative Filtering: Don't go Deeper, go Higher
• More et al. (2019): Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Bandits with Large Action Spaces
• Bhattacharya et al. (2022): Augmenting Netflix Search with In-Session Adapted Recommendations

General Links:

• Follow me on Twitter: https://twitter.com/MarcelKurovski
• Send me your comments, questions and suggestions to [email protected]
• Podcast Website: https://www.recsperts.com/

In this episode of Recsperts we talk to Flavian Vasile about the work of his team at Criteo AI Lab on personalized advertising. We learn about the different stakeholders like advertisers, publishers, and users and the role of recommender systems in this marketplace environment. We learn more about the pros and cons of click versus conversion optimization and transition to econ(omic) reco(mmendations), a new approach to model the effect of a recommendations system on the users' decision making process. Economic theory plays an important role for this conceptual shift towards better recommender systems.

In addition, we discuss generative recommenders as an approach to directly translate a user’s preference model into a textual and/or visual product recommendation. This can be used to spark product innovation and to potentially generate what users really want. Besides that, it also allows to provide recommendations from the existing item corpus.

In the end, we catch up on additional real-world challenges like two-tower models and diversity in recommendations.

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.

Chapters:

• (02:37) - Introduction Flavian Vasile
• (06:46) - Personalized Advertising at Criteo
• (18:29) - Moving from Click to Conversion optimization
• (23:04) - Econ(omic) Reco(mmendations)
• (41:56) - Generative Recommender Systems
• (01:04:03) - Additional Real-World Challenges in RecSys
• (01:08:00) - Final Remarks

• Flavian Vasile on LinkedIn
• Flavian Vasile on Twitter
• Modern Recommendation for Advanced Practitioners - Part I (2019)
• Modern Recommendation for Advanced Practitioners - Part II (2019)
• CONSEQUENCES+REVEAL Workshop at RecSys 2022: Causality, Counterfactuals, Sequential Decision-Making & Reinforcement Learning for Recommender Systems

Papers:

• Heymann et al. (2022): Welfare-Optimized Recommender Systems
• Samaran et al. (2021): What Users Want? WARHOL: A Generative Model for Recommendation
• Bonner et al (2018): Causal Embeddings for Recommendation
• Vasile et al. (2016): Meta-Prod2Vec: Product Embeddings Using Side-Information for Recommendation

General Links:

• Follow me on Twitter: https://twitter.com/LivesInAnalogia
• Send me your comments, questions and suggestions to [email protected]
• Podcast Website: https://www.recsperts.com/