# Netflix PRS 2024 - Applying LLMs to Recommendation Experiences

> Challenges and lessons from deploying LLM experiences: evals, scalability, guardrails.

- Author: Eugene Yan
- Published: 2024-05-31
- Tags: llm, engineering, production, leadership
- URL: https://eugeneyan.com/speaking/netflix-prs


Recently, I was invited to speak at the 2024 Netflix Workshop on Personalization, Recommendation, and Search. I shared about the challenges faces while building and deploying LLM-powered recommendation experiences at consumer scale.

<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vQja7_Ad4M4ZrLfJC_FzzfNqea8qoXLdUE0iz46AHMSd-JrBQHlYtf86ipAQjRw97DKTjtQdYLiC4iU/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="441" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>

It was an enlightening conference that covered a range of topics from LLMs to recsys to measurement, and it was a fun opportunity to catch up with old friends in San Francisco. I shared some observations [here](https://x.com/eugeneyan/status/1796629024829030641) and [here](https://x.com/eugeneyan/status/1797282959634104825). Here's the full list of topics and speakers.
- LLMs as Agents and How to Evaluate Them on Real-World Tasks (Alane Suhr, Assistant Professor, UC Berkeley)
- Applying Language Models to Recommendation Experiences: Challenges and Lessons (Eugene Yan, Senior Applied Scientist, Amazon)
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (Jiaqi Zhai, Distinguished Engineer, Meta)
- Conversational RecSys (Harald Steck, Senior Research Scientist, Netflix)
- Long-Term Value of Exploration: Measurements, Findings and Algorithms (Yi Su, Research Scientist, Google)
- Beyond the Binge: Recommending for Long-Term Member Satisfaction (Jiangwei Pan, Senior Research Scientist, Netflix)
- Toward Practical Robustness in AI (Alex Beutel, Model Safety Tech Lead, OpenAI)
- Building Airbnb Categories with ML and Human in the loop (Mihajlo Grbovic, Principal ML Scientist, AirBnB)
- Personalization at Spotify (Maria Dimakopoulou, Director of ML & Head of Homepage P13N, Spotify)