Blog·Engineering·24 December 2025·9 min
Real-World Machine Learning: How Top Companies Ship AI to Production
From fraud detection at Stripe to arrival predictions at Uber: a deep dive into the ML systems powering the world's best products, and what we've learned building our own.
Pulse Local is a budgeting app that runs on your machine. The language model that answers questions about your money runs there too, not in a cloud.
A small model makes mistakes, so most of the engineering is the work around the model: a second reading of each question that must agree with the first before you see a figure, and a benchmark that counts how often a wrong figure still gets through. Two articles on this blog describe the second reading and the benchmark.
None of this is new. Teams that run machine learning at a scale far beyond ours met the same problems first, and many of them wrote down what they did.
This post is a curated tour through some of the most valuable ML case studies from companies operating at massive scale. Whether you're an engineer looking for implementation patterns or just curious about how these systems work, there's something here for you.
Stripe Radar: Real-Time Fraud Detection
Every online payment is a bet. The merchant bets the customer is legitimate. The payment processor bets the card isn't stolen. Get it wrong in one direction, and you lose money to fraud. Get it wrong in the other, and you block legitimate customers who never come back.
Stripe processes billions of transactions annually, which gives them something invaluable: a view across the entire internet economy. Their fraud detection system, Radar, uses deep learning trained on patterns that no single merchant could ever see.
| What Makes It Work |
|---|
| Network effects (patterns learned from millions of businesses) |
| Under 100 ms decision latency |
| Continuous learning from new fraud vectors |
The insight is as much strategic as technical. Stripe's moat isn't their model architecture; it's their data flywheel. Every transaction makes the system smarter.
Read Stripe's full breakdown →
Uber DeepETA: Predicting Arrival Times
"Your driver will arrive in 8 minutes." That number needs to actually be 8 minutes. Inaccurate ETAs frustrate riders, hurt driver earnings, and erode trust in the platform.
Traditional routing algorithms treat roads as edges in a graph with fixed travel times. But anyone who's driven knows reality is messier. A Giants game affects traffic differently than a concert at Chase Center. Rain slows everything down, but not uniformly.
Uber's DeepETA uses transformer architecture (the same family of models behind ChatGPT) to process routes as sequences. Think of it like treating a journey as a "sentence" of road segments, where attention mechanisms learn which segments matter most for the final prediction.
| Technical Approach |
|---|
| Transformer-based sequential modeling |
| Real-time traffic integration |
| Segment-level attention for route understanding |
The result is ETA predictions that account for complex, non-linear interactions that rule-based systems simply can't capture.
Dive into Uber's technical deep-dive →
DoorDash: Reducing Dasher Wait Times
Here's a scenario that costs DoorDash millions: a Dasher arrives at a restaurant and waits 20 minutes for an order that isn't ready. The Dasher earns less per hour. The customer gets lukewarm food. DoorDash pays for idle time.
The solution sounds simple: predict when orders will be ready and dispatch Dashers to arrive just in time. The execution is anything but.
DoorDash's engineering team built a time-series forecasting system that considers restaurant-specific prep patterns, current order volume, time of day, day of week, and menu item complexity. A burger takes different time than a custom sushi roll.
But the most valuable part of their case study is the lifecycle discussion:
"The hardest part wasn't building the model; it was integrating it into operations, monitoring for drift, and continuously improving it. ML products are never done."
| Lifecycle Stages |
|---|
| Problem definition, Model development, Integration |
| Monitoring, Drift detection, Retraining |
| Continuous improvement loop |
Read DoorDash's full lifecycle story →
Netflix: In-Video Search
You remember a specific scene (maybe a dialogue, a location, a visual moment) but not which movie it's from. Traditional search is useless. You can't Google "that scene where the guy says the thing in the rain."
Netflix built a system that actually understands video content. Not just metadata, but the visual and audio content itself: detecting scenes, objects, dialogue, and mood across their entire library.
Combined with semantic search, this enables queries that would have been impossible a few years ago. Describe what you're looking for in natural language, and the system finds the moment.
| Scale Challenges |
|---|
| 15,000+ titles to process |
| Millions of hours of video content |
| Real-time search latency requirements |
The engineering challenge: building inference infrastructure that processes their entire catalog cost-effectively.
Spotify: Podcast Search
Podcasts contain incredible content locked inside hours of audio. Traditional search relies on titles and descriptions, which means you can find "The Joe Rogan Experience" but not the specific episode where they discussed psychedelics and neuroscience.
Spotify's solution: transcribe everything, then build semantic embeddings that capture meaning rather than just keywords. Search "how to start a business" and surface relevant discussions across thousands of podcasts, even if those exact words were never spoken.
| Pipeline |
|---|
| Speech-to-text at scale |
| Semantic embedding generation |
| Vector similarity search |
They also built ML-generated podcast previews: short audio clips that give you a taste of an episode's content, generated automatically. ML can make content more accessible, not just recommend it.
Airbnb: Categories with Human-in-the-Loop
How do you categorize millions of unique homes into delightful categories like "Treehouses," "Castles," or "A-frames"? Pure ML makes embarrassing mistakes, confidently labeling a suburban house as a castle because it has a turret-shaped window. Pure human curation doesn't scale to millions of listings.
Airbnb's answer: human-in-the-loop ML. The model makes initial predictions. Human curators review edge cases and correct errors. Those corrections feed back into training data. The model improves. Rinse, repeat.
| The Feedback Loop |
|---|
| ML predicts category |
| Human curators review uncertain cases |
| Corrections become training data |
| Model improves, cycle continues |
This philosophy (using ML to scale human judgment rather than replace it) produces results that feel magical. The A-frame cabin you didn't know you wanted, surfaced because the system learned what "A-frame" really means from thousands of human decisions.
Read Airbnb's human-in-the-loop approach →
GitHub Copilot: Enterprise LLMs at Scale
Building an LLM-powered product that millions of developers trust for code suggestions is a different beast than building a chatbot. Wrong answers don't just look bad; they could introduce bugs or security vulnerabilities into production systems.
GitHub's team shared hard-won lessons from building Copilot at scale:
| Production Challenges |
|---|
| Latency must feel instant (users won't wait) |
| Cost per suggestion must be sustainable at scale |
| Quality must be measurable and improvable |
| Responsible AI: don't regurgitate copyrighted code |
Their key insight: "Evaluation is everything. If you can't measure improvement, you can't ship it."
They built extensive evaluation infrastructure before scaling the product, because you can't improve what you can't measure.
GitHub's enterprise LLM lessons →
Zillow Zestimate: Neural Home Valuation
Estimate the value of any home in America (including ones that haven't sold in decades) and do it accurately enough that people make major financial decisions based on your number.
The Zestimate evolved from simple comparable-sales calculations to ensemble ML to deep neural networks. The current system processes structured data (bedrooms, bathrooms, square footage) alongside unstructured data (listing descriptions, photos) to understand that "chef's kitchen" adds more value than "functional kitchen."
| Evolution |
|---|
| V1: Comparable sales math |
| V2: Ensemble models |
| V3: Neural networks with NLP |
| Accuracy: About 2% median error |
But Zillow's story also includes a cautionary tale. Their iBuying business (Zillow Offers) failed partly due to model overconfidence, a reminder that even sophisticated ML systems have uncertainty bounds that matter.
Patterns Worth Remembering
After reading through these case studies, some patterns stand out:
Start simple, add complexity when needed. Almost every case study describes beginning with rules or simple models, then graduating to deep learning after proving value. Don't start with transformers.
Human-in-the-loop scales judgment, not just labor. The best systems (Airbnb Categories, Uber's fraud review) combine ML efficiency with human expertise. Full automation isn't always the goal.
Your data is your moat. Stripe can detect fraud patterns because they see the entire internet economy. Your competitive advantage is data others don't have access to.
ML products are never done. DoorDash emphasizes the lifecycle: deploy, monitor, retrain, improve, repeat. Budget for ongoing investment.
Evaluation infrastructure is non-negotiable. GitHub's team is blunt: if you can't measure improvement, you can't ship it. Build evals before you build models.
Keep Exploring
This post barely scratches the surface. The full curated list contains 300+ case studies from 80+ companies, maintained by the community on GitHub:
A Curated List of ML System Design Case Studies
Have a favorite ML case study we missed? We'd love to hear about it.