Stewart Alsop sits down with Teo Ionita, founder of Ideaticon, a data consulting company that builds data platforms for businesses. Their conversation explores how data engineering has evolved from the traditional analyst-engineer-scientist triad into something fundamentally different with the advent of LLMs, which Teo argues are better at working with SQL primitives than humans—essentially turning data work into a management problem where humans need to understand systems theoretically rather than mechanically. They discuss everything from clickstream data and behavioral analytics to the nature of consciousness in AI systems, touching on why the rationalist community's existential risk narratives might be more about regulatory capture than actual danger, and exploring Teo's theory that LLMs are the ultimate expression of "technique"—existing only in the interaction between human and machine, like golems summoned by magic words.
Key Insights1. Data engineering has evolved from a single role into a triad of specialized positions: data analysts who focus on querying clean data and generating insights, analytics engineers who transform and prepare data using tools like DBT, and data engineers who handle data ingestion and pipeline infrastructure. This evolution occurred as companies realized they needed different specialists to handle the growing complexity of data systems, with each role focusing on a distinct part of the data lifecycle from collection through to consumption.
2. LLMs have fundamentally transformed data analytics by replacing traditional dashboard consumption with natural language queries. Instead of carefully crafted SQL queries that require precision like a sniper rifle, LLMs operate like machine guns firing multiple queries simultaneously until they hit the target. Anthropic recently announced Claude will create dashboards directly, marking a significant shift in how people interact with data. However, SQL remains the most efficient language for data retrieval because it was originally designed to be close to natural language for business users.
3. Companies of any size can benefit from data engineering, even at the individual level. The speaker manages data analytics for his own one-person app, using Claude to analyze user activity, subscription data, and engagement patterns. This represents a dramatic shift from knowledge management, which typically only becomes necessary around 500 employees. With LLMs, anyone can perform sophisticated data analysis regardless of company size, democratizing access to insights that previously required dedicated teams.
4. The relationship between humans and LLMs requires a new type of theoretical understanding rather than mechanical coding knowledge. Success with AI agents depends on understanding systems at a high level, knowing what each component should do without necessarily understanding the detailed implementation. This mirrors how managers operate, maintaining a theoretical framework while delegating execution to specialists. The speaker and one colleague completed a massive data migration in six months that would have taken a team of ten people over a year by maintaining this theoretical understanding while letting agents handle implementation details.
5. We are losing the ability to develop fuzzy knowledge and intuition about systems because people no longer write code by hand or go through the empirical learning process. This creates a critical challenge: how do we teach people to develop the theoretical understanding and pattern recognition needed to effectively manage AI agents when they never experience the hands-on work that traditionally built that intuition? This skill of knowing what is relevant and what is not, of understanding the proper shape of a system, remains essential but increasingly difficult to transmit to new practitioners.
6. Clickstream or behavioral data has become crucial for training LLMs to replicate human work patterns. Meta and other companies are now tracking every interaction their employees have with systems, similar to how advertising companies historically tracked user behavior to optimize ad performance. This data helps create models that understand how people actually work, shop, and interact with systems. The speaker specializes in this type of data collection and analysis, which represents the convergence of traditional analytics with the new demands of AI training.
7. LLMs represent the ultimate expression of technique as defined by philosopher Jacques Ellul, existing only in the moment of human-machine interaction. The models are essentially dormant weights stored on servers until activated by human prompts, at which point they become animated by the data and goals provided by users. This relationship suggests we may not escape technique through AI but instead deepen our entanglement with it, as every interaction with these systems represents a new instantiation of the human-machine feedback loop that defines modern technological society.
Timestamps00:00 Tokyo weather discussion and travel recommendations for springtime visits
05:00 Teo explains his data consulting company Ideaticon, building pipelines from ingestion through data warehouses to client applications
10:00 Evolution of data roles from analysts to engineers, how SQL was originally designed for business users but became specialized programming
15:00 Three-part data team structure emerges with analysts, engineers, and analytics engineers handling different pipeline stages
20:00 Scale question: companies need data work earlier than knowledge management, even one-person operations benefit from analytics
25:00 Agency and consciousness in LLMs discussed, challenging rationalist assumptions about autonomous AI systems
30:00 Loop architecture examined, LLMs lack genuine goals beyond tasks, regulatory capture concerns around existential risk narratives
35:00 Relevance realization problem, LLMs better than SQL at finding patterns but lack human judgment on importance
40:00 Teo describes managing agents like managing people, maintaining theoretical understanding while delegating mechanical coding work
45:00 Knowledge transfer crisis: losing ability to develop fuzzy systems understanding without hands-on coding experience
50:00 Clickstream and behavioral data collection, Meta tracking workers to train agents on human interaction patterns
55:00 Technique philosophy, LLMs as ultimate expression of human-machine relationship, consciousness as vital force given through prompts
