In this talk, Suhas Suresha, Senior Machine Learning Engineer at Adobe and co-author of Designing AI Systems, shares his expertise in building large-scale generative AI infrastructure from deploying real-time ECG analysis models at his previous startup, QALY, to architecting enterprise LLM platforms. We explore the massive engineering effort required to take an AI application from a weekend prototype to a scalable production system, and how to build a unified AI platform that eliminates "AI sprawl."
Links:
- Book - https://www.manning.com/books/designing-ai-systems
- Code - https://github.com/designing-ai-systems/designing_ai_systems_repo
You’ll learn about:
- The nine core components every production-ready AI platform needs to succeed.
- How to solve "AI sprawl" and prevent fragmented engineering efforts across teams.
- Architecting scalable Data Services and Model Services for robust LLM applications.
- Implementing comprehensive observability and request tracing to identify system bottlenecks.
- Managing long-term session memory, tool registries, and complex agent orchestration.
- Navigating the "build versus buy" dilemma for enterprise AI infrastructure.
TIMECODES:
0:00 AI Prototypes to Production and the Book Origin
5:49 AI Sprawl and the Need for a Centralized AI Platform
12:42 Definition of an AI Platform and the Build vs Buy Dilemma
23:17 AI Platform Engineering Skills and Data Service Architecture
31:26 System Observability and Request Flow Bottlenecks
37:32 LLM Evaluations, Prompt Experimentation, and Cost Tracking
43:57 Session Memory, Tool Registries, and Agent Orchestration
50:56 Invocation Modes and Durable Execution for AI Agents
56:14 The Nine Core AI Platform Components and Memory Management
This episode is designed for machine learning engineers, backend developers, and platform architects looking to transition into AI platform engineering. It is highly recommended for technical leaders and teams who need to scale LLM prototypes into reliable, cost-effective production systems.
Connect with DataTalks.Club:
- Join the community - https://datatalks.club/slack.html
- Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ
- Check other upcoming events - https://lu.ma/dtc-events
- GitHub: https://github.com/DataTalksClub
- LinkedIn - https://www.linkedin.com/company/datatalks-club/
- Twitter - https://twitter.com/DataTalksClub
- Website - https://datatalks.club/
Connect with Suhas:
LinkedIn - https://www.linkedin.com/in/suhas-suresha/
Twitter - https://x.com/inceptor92
Github - https://github.com/suhas1992
Website - https://suhassuresha.com/
