In this talk, Dewang Sultania, ML Engineer at Netflix and co-author of Designing AI Systems, shares his technical journey from building early inference platforms at Adobe to deploying multimodal AI models at scale. We explore the complete architecture of enterprise AI platforms, focusing on evaluation datasets, tool calling orchestration, and the practical realities of shipping production agents.
You’ll learn about:
- The essential components needed to build an enterprise AI platform from scratch
- How traditional software engineers can transition into AI platform engineering
- The critical technical differences between session management and LLM memory
- Strategies for building, versioning, and trusting custom evaluation datasets
- Optimizing LLM tool calling and selecting the right performance benchmarks
- When to use multi-agent architectures versus single-agent systems
- Techniques for overcoming the lost in the middle problem in RAG pipelines
TIMECODES:
00:00 Core Components of Enterprise AI Platforms
11:46 Career Path to AI Platform Engineering
18:33 Session Management Versus LLM Memory Management
26:47 Best Practices for Custom LLM Evaluation Datasets
32:28 LLM Tool Calling Optimization and Benchmark Selection
39:25 Context Window Limits and AI Agent Tool Scaling
45:37 Single-Agent Versus Multi-Agent System Architecture
52:16 LLM-as-a-Judge Alignment and Human Evaluation Strategies
58:11 Solutions for the Lost in the Middle Problem in RAG Systems
01:03:27 Designing AI Systems Book Announcement
This discussion is perfect for software engineers, machine learning practitioners, and technical leaders building real-world AI applications. It offers practical insights for anyone transitioning into AI platform engineering or tasked with scaling LLM features for enterprise production.
Connect with Dewang:
- Linkedin - https://www.linkedin.com/in/dewang-sultania/
- Github - https://github.com/iron-stark
Connect with DataTalks.Club:
- Join the community - https://datatalks.club/slack.html
- Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ
- Check other upcoming events - https://lu.ma/dtc-events
- GitHub: https://github.com/DataTalksClub
- LinkedIn - https://www.linkedin.com/company/datatalks-club/
- Twitter - https://twitter.com/DataTalksClub
- Website - https://datatalks.club/
