Schaun Wheeler and DJ Rich delve into the intricacies of building recommender systems in this podcast hosted by Arpit Choudhury. The discussion highlights the steps to developing a recommender system, practical advice for startups, and the evolving landscape of recommender technologies.
KEY POINTS:
* Identify the Problem: Recommender systems address the "discovery problem" by helping users sift through vast amounts of options to find relevant content quickly. Recognizing this problem is crucial before diving into solutions.
* System Components: Recommender systems are complex and involve multiple components such as:
* Item Inventory: Detailed metadata about items (e.g., descriptions, categories).
* User Interaction History: Data on user interactions with items (e.g., views, purchases).
* Recommendation Model: The core model that filters and ranks items based on user preferences.
* The Learner: An important component, which trains the model and separates it from the model's deployment phase.
* Build vs. Buy: Should one build a recommender system from scratch or use existing solutions? For many startups, buying an off-the-shelf system can be more practical due to advances in data infrastructure and the complexity of developing a bespoke system. Buying a system can also free up resources for other critical areas.
* Practical Recommendations for Startups: Instead of getting bogged down by complex models initially, startups are encouraged to start with simpler models and leverage existing infrastructure to implement a functional recommender system.
* Innovations in Recommender Systems: Schaun is interested in combining traditional methods with reinforcement learning to enhance system performance. DJ is excited about research addressing causal questions and handling sequential recommendations.
REFERENCES:
* "Are we really making progress?" This paper is a replication study on recommender algorithms and shows that many DL approaches couldn't be reproduced or could be beaten with linear methods.
* "Deep Exploration for Recommender Systems" This paper talks about sequential decisioning for RSs (where you consider more than just one item recommendation).
* "Two Decades of Recommender Systems at Amazon" This paper is a retrospective on what's work well at Amazon.
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com
