How LLMs Are Reshaping Recommendation Systems
- 47m
- Transcript
Summary
Written by a model from the transcript; not yet reviewed.
LinkedIn recently re-engineered its newsfeed to treat content recommendation as a sequence modeling problem, moving away from independent deep learning scoring architectures. Tim Yurka, VP of Engineering at LinkedIn, joins host Matt Merrill to discuss how the team combines large language models with traditional signals, manages inference costs at massive scale across 1.3 billion members, and uses natural language policies and agentic tooling to steer content quality and combat low-quality output. The discussion explores the engineering realities of running AI systems at scale, highlighting the necessity of hybrid architectures, CPU offloading, custom attention kernels, and deep system design over simple model dumping.
- Combine large language models with traditional quantitative features and popularity signals rather than relying on LLMs alone. 33:46
- Offload non-model filtering tasks like language checks to cheaper CPU hardware before loading items into GPU memory. 26:53
- Develop cross-functional generalist skills and product intuition because agentic coding tools dissolve traditional team boundaries. 45:34
- Write explicit natural language policies checked into version control to evaluate and guide content retrieval quality. 14:16
Chapters
- 0:00Episode Introduction and Host Overview
Matt Merrill introduces the episode and outlines the focus on LinkedIn’s feed overhaul using LLMs.
- 1:54Tim Yurka’s Background and Role
Tim explains his 13‑year journey at LinkedIn, from AI engineer to VP of Engineering overseeing consumer products.
- 2:57Problem Statement: Scale and Relevance
Tim discusses the challenges of scaling from 690M to 1.3B members and delivering relevant content instantly for new users.
- 4:36Model Training, Evaluation, and Policies
Tim details the iterative training pipeline, A/B testing, and policy‑driven evaluation to ensure high‑quality, timely content.
- 6:00Engineering Team Structure and Workflows
The conversation covers the multidisciplinary pods, generalists, and specialized roles that enable rapid iteration and deployment.
- 7:12Scaling Infrastructure and GPU Optimization
Tim describes the GPU‑centric architecture, batching, custom attention kernels, and CPU‑offloading strategies for cost‑effective inference.
- 34:17Balancing LLMs with Traditional Features
The team explains how they combine LLM embeddings with quantitative popularity signals and other features to maintain performance and relevance.
- 37:29Future Challenges and Open‑Ended Questions
The discussion turns to new sleep‑inducing problems, coordination across teams, and the need for continuous improvement in a rapidly evolving AI landscape.
- 40:27Responsible AI and Guardrails
Tim outlines LinkedIn’s responsible AI commitments, including demographic‑free signals, audit checks, and trust mechanisms like verified profiles.
- 43:05Protecting APIs from Low-Quality Content
LinkedIn explains how they guard against bots that post unoriginal material, ensuring authentic user contributions.
- 43:57Using LLMs for Input Sanitization
The host likens LLM technology to a filter that cleans user input before it enters the system, preventing future issues.
- 44:32Designing Systems with the Right Tool Size
The conversation highlights the importance of matching tool capabilities to tasks, avoiding a one-size-fits-all approach.
Highlights
- 3:01
Scaling to 1.3 Billion Membersdata
LinkedIn grew from 690 million members in 2020 to 1.3 billion members, with seven new members joining every second, creating massive scale and zero-shot onboarding challenges.
- 7:17
Sequence Modeling for Contentinsight
Instead of scoring items independently using feature crossing MLPs, LinkedIn treats content recommendations like language sequence prediction to map out where users go next.
- 14:16
Natural Language Evaluation Policiesstrategy
Product managers define content quality policies that are evaluated using tastemakers and automated via GitHub Copilot CLI in an active learning loop.
- 25:32
Custom CUDA Kernel Optimizationstrategy
To maintain throughput without buying excessive Nvidia GPUs, infrastructure engineers optimize custom attention kernels down to the CUDA level for inference time.
- 32:24
Numeric Data Misinterpretationstory
Early LLMs treated numeric data and article counts like plain text and missed popularity signals, requiring the team to combine LLMs with traditional counting systems.
- 36:43
Compounding Agentic Error Ratesinsight
If each step of a 20-stage agentic workflow operates at 99.5 percent success, the end-to-end workflow becomes completely useless.
Quotes
“LinkedIn recently rebuilt its newsfeed to treat content recommendation as a sequence modeling problem.”
Unnamed speaker · 0:24 “If you have an agentic system where each step of the agentic system works 99.5% of the time, and it's like a 20-stage agentic workflow, then like the end to end workflow is useless.”
Unnamed speaker · 36:43 “We run kind of regular audits to make sure that the kind of feed distribution, feed quality is equitable across different audiences and creators using like a series of statistical checks.”
Unnamed speaker · 40:49 “But protective of your APIs, right? Like, as far as I know, like you do a lot to prevent bots from just like putting out AI slop content.”
Unnamed speaker · 43:05 “We want to make sure that, obviously, members should use these tools, but we want them to use the tools to bring the kind of authentic, original perspectives to LinkedIn, not like super repetitive stuff through automation.”
Unnamed speaker · 43:46 “Good system design, knowing when to use the right tool for the right job, when to use the right size tool for the right job, input sanitization.”
Unnamed speaker · 44:32
About this episode
News feeds and recommendation systems have long relied on deep learning architectures that score each candidate item independently. As LLMs have matured, they have opened up a fundamentally different approach, where a system can reason about content the way it reasons about language. However, that power comes with a fresh set of engineering challenges around cost, scale, and evaluation. LinkedIn https://www.linkedin.com/ recently rebuilt its news feed to treat content recommendation as a sequence modeling problem. The general approach is to predict what a user will want next, much like an LLM predicts the next token in a sentence. Tim Jurka https://www.linkedin.com/in/timjurka has worked at LinkedIn for 13 years and is currently a VP of Engineering. In this episode, Tim joins Matt Merrill to discuss how LinkedIn re-engineered its feed, how the team combines LLMs with traditional signals, managing inference costs at massive scale, steering content quality using natural language policies, and more. Sponsorship inquiries: sponsor@softwareengineeringdaily.com mailto:sponsor@softwareengineeringdaily.com The post How LLMs Are Reshaping Recommendation Systems https://softwareengineeringdaily.com/podcasts/how-llms-are-reshaping-recommendation-systems/ appeared first on Software Engineering Daily https://softwareengineeringdaily.com.
Transcript
9,897 words · 4 speakers
Topics
- LLM
- GPU
- CPU
- LinkedIn feed
- GitHub Actions
- ML
- Claude
- X Weather
- Ori
Page updated · transcript and summary are machine-generated