E25 Running Deep Neural Networks on Microcontrollers with Mark Deutel and Dr. Axel Plinge

We talk with Mark Deutel and Dr. Axel Plinge from Fraunhofer IIS about the art and science of running AI models on resource-constrained devices. Mark walks us through his PhD work on getting neural networks to run on microcontrollers with just 256KB of memory - a feat that requires clever optimization tricks, from neural architecture search to instruction-level tuning. Axel shares insights on why edge AI matters for robotics, where battery life and response time make local processing essential.

The conversation ranges from their multilingual airport robot (complete with culturally appropriate gestures) to the practical challenges of teaching robots to grasp objects. We explore the shift from hand-engineered features to data-driven approaches, discuss why German privacy laws favor edge processing, and learn why the simplest microcontrollers sometimes give the most predictable performance. Mark and Axel make a compelling case that not every problem needs the biggest AI model - sometimes a specialized edge model is exactly the right tool.

Key Topics:

• [03:30] Introduction to Fraunhofer IIS and its role bridging academia and industry
• [06:45] The multilingual airport robot: combining LLMs, speech processing, and local AI
• [15:20] Mark's PhD work: running neural networks on 256KB microcontrollers
• [19:40] The optimization pipeline: from neural architecture search to SIMD instructions
• [28:15] Why edge AI matters for robotics: battery life, latency, and the human analogy
• [35:50] Visual Language Action models and the challenge of world models in robotics
• [42:30] The complexity of grasping: what humans take for granted that robots must learn
• [48:20] German privacy laws (DSGVO) and why they favor edge processing

Notable Quotes:

"You're trying to run something that is way more compute-heavy on something that shouldn't actually be able to run it. It's kind of like the magic that guys in the 90s had when they wanted to run 3D games on old computers." — Mark Deutel

"You cannot optimize without leaving stuff out. Mark says it's a magic trick, but it's not that much magic - yes, he got YOLO running on a system, but he also removed a lot of categories. Specialization is always part of the deal." — Dr. Axel Plinge

"When you start to think about it, it's actually insane what considerations you make when you grasp something. You consider: is it fragile? Where's the best point to pick it? Should I pick it at the top or bottom? Are my hands sweaty? There's all kind of stuff you do without really thinking about it." — Mark Deutel

Resources Mentioned:

• Fraunhofer IIS Efficient AI Team - Research group building AI solutions with fast MLOps workflow for embedded devices
• HAI Store - Fraunhofer's platform for downloading and evaluating compressed AI models
• Mark Deutel's Research Page - Personal homepage with details on PhD research about efficient neural networks on microcontrollers
• PIB Robot - 3D-printable educational humanoid robot from Incento used in the airport demonstration
• NVIDIA Jetson Thor - Edge GPU system used for running local LLM and speech processing in the airport robot
• Agile Embedded Podcast Slack - Community discussion channel where you can reach Ryan and Luca, with a dedicated sub-channel for Embedded AI topics