Episode 159: Big Lessons from Small Models with Gwyneth Peña‑Siguenza

What can small language models teach us that the largest AI models cannot?

Kelly and Julian are joined by Microsoft Cloud Advocate Gwyneth Peña-Sigüenza to explore why working with small language models (SLMs) may be one of the best ways to understand AI. Rather than relying on increasingly capable models that hide complexity, Gwyneth argues that constraints build stronger fundamentals. From prompt engineering and context management to deployment and security, SLMs force learners to think more carefully about how AI actually works.

The conversation extends beyond AI models into learning itself. Gwyneth shares her self-taught journey from growing up on a remote farm in Ecuador with limited internet access to becoming a Microsoft Cloud Advocate and creator of the Learn to Cloud platform. Along the way, the group discusses productive struggle, mentorship, cloud engineering, Python, security, and what educators should prioritize as AI becomes part of every student's learning experience.

The episode closes with a thoughtful discussion about AI dependency, judgment, and whether we would actually flip the switch and turn AI off if given the choice.

Show Notes

Wins of the Week

• Gwyneth celebrates the New York Knicks reaching the NBA Finals after more than 50 years.
• Julian shares that he has accepted a new role as a Fractional CTO.
• Kelly reflects on taking her first real vacation in over a year—and how stepping away from work sparked unexpected ideas.

Small Language Models

• Why SLMs are valuable teaching tools
• Learning prompt engineering through constraints
• Running models locally on everyday hardware
• When local AI makes sense for classrooms
• Understanding tokens, context windows, and model limitations
• Why bigger models can sometimes hide important lessons

Learning Through Constraints

• Learning to drive in an old manual pickup truck as a metaphor for learning AI fundamentals
• Why difficult learning experiences often create lasting understanding
• Building strong habits before relying on more capable tools
• Consistency versus constantly chasing the newest resource

Self-Taught Learning

• Growing up without reliable internet in rural Ecuador
• Downloading YouTube playlists to learn programming offline
• Developing discipline through limited access
• The value of repetition and focused practice
• Why mentorship accelerates learning

Python Journey

• Transitioning from cloud engineering to Python advocacy
• Learning Python beyond scripting
• Discovering what "Pythonic" really means
• Wrestling with list comprehensions and other advanced syntax

• Favorite learning resources:

  • Fluent Python
  • Effective Python

Learn to Cloud

• Building an open-source cloud engineering curriculum
• Hands-on labs and automated verification
• AI-assisted assessment
• Supporting self-taught learners around the world
• Creating accessible technical education

Cloud, AI, and Security

• Deploying AI applications to the cloud
• Containers, virtual machines, and serverless deployments
• Why operations and security deserve more classroom attention
• Introducing secure development practices early
• The importance of authentication, secrets management, and responsible deployment

Teaching in the AI Era

• Helping students understand how AI works instead of simply using it
• Why productive struggle still matters
• The changing role of educators
• Balancing AI assistance with independent thinking
• Preparing students for a future where AI is always available

Final Thoughts

• AI dependency versus capability
• Judgment as the skill that matters most
• Human connection in an AI-driven world
• Would we actually turn AI off?
• Finding balance between technological progress and intentional learning