Join us as Du'An digs into the real mechanics of running AI locally and in production - from GPU memory math to multi-agent architectures, observability, and the economics of self-hosted inference.
Du'An walks through how model weights and KV cache compete for GPU memory, why continuous batching matters when you have more than a handful of users, and how agent architectures like single-agent, workflow, graph, swarm, and supervisor patterns each solve different problems. You will learn how to instrument your agents with Langfuse for observability and cost tracking, when to use Ollama versus vLLM, how prompt caching can cut provider costs by up to 75%, and why GPUs should never sit idle. Episode two of three - the next episode covers deploying at scale.
Timestamps
0:00 Welcome & Introduction
1:47 Du'An's New Role at Akamai Cloud
3:10 Data Privacy and the Case for Self-Hosted AI
7:21 Anthropic and OpenAI as the New Cloud Layer
12:48 Local Models for Specific Use Cases - Cancer Detection Example
15:02 GPU Memory Math - Weights, KV Cache, and Context Windows
19:32 Continuous Batching and GPU Time Slicing
20:03 Observability with Langfuse - Live Demo
27:44 Agent Architectures - Single Agent, Workflow, Graph, Swarm, Supervisor
36:36 Token Economics, Prompt Caching, and GPU Cost Planning
Podden och tillhörande omslagsbild på den här sidan tillhör
vBrownBag. Innehållet i podden är skapat av vBrownBag och inte av,
eller tillsammans med, Poddtoppen.
