Not Every Problem Needs a Sledgehammer

“I spoke to an engineer at one of our suppliers who said that in the last six months, he’s generated a million lines of code.”Michael Frendo, the CTO of New Relic, offered me that as evidence of acceleration, one data point among several. The counterweight came from the start of his own career, at Nortel in Ottawa: “Our telephone switch at the time had the largest code base in the world. It was the largest. It was 14 million lines of code, and it had 2,500 engineers working on it for decades. Now we talk about the engineer delivering a million lines of code in six months. Think about that juxtaposition.”

He’s had a lot of vantage points to think about it from. “It seems like there’s no technology I’m not somehow attracted to, as long as it’s got something to do with tech,” he told me, and the record backs him up: Cisco, because “I recognized there was a shift coming, going from essentially the TDM world to the packet world,” then Polycom, Juniper, Proofpoint, a stretch in long-haul optical. A few months before we spoke he became CTO of New Relic: “It is the company that sort of created the whole observability space. And what I say to people, now we get to recreate it.”

The fast versus the slow

He is not ambivalent about the acceleration. “Many years ago, I worked for a CEO who said, ‘You know what? It’s not about the big versus the small. It’s about the fast versus the slow.’ And in tech, the fast almost always beats the slow.” AI, in his view, is speed you don’t decline: “Any company that’s going forward now that doesn’t embrace AI and understand what it’s capable of, and use the skill sets that are there to accelerate our ability to develop within the boundaries of making sure that it does it correctly, will fall behind.”

The numbers he sees inside his own teams: “depending on the project and depending on the stage that it’s at, two times, three times, five times productivity improvements in certain aspects of the task.” Secondhand, wilder: “I’ve heard anything from conservatively 2X to bold statements around 10 to 15X, and this is from people who are regularly putting technology into production.” And the gains spread beyond the code itself. “Developing technology is not just writing code,” he said, “although writing code is a significant part of it. But AI doesn’t just help with the writing of the code, it helps with developing the spec. It helps with creating the test case. It helps with creating the documentation. It helps with getting you the compliance that you need.”

One example he offered sits upstream of engineering entirely. “There’s always been this interesting sort of tension between product management and engineering, where product management has an idea and they sort of throw it over the wall to engineering,” and the requirements dribble in as everyone figures out what was meant. “Now you put vibe coding in the hands of a product manager and they prototype the whole thing out. It’s not production, but they prototype it out.” They demo it to a customer, collect feedback, and engineering gets “a set of requirements at the beginning that’s so much more complete and so much better defined.” Software, as he put it, is now being developed “at a speed that’s never, ever been seen before.”

True engineering

So when I raised the fear making the rounds, that we won’t need to think anymore, I expected the standard reassurance. What I got was sharper. “I would take some exception to the comment, ‘We don’t need to think anymore.’” His teams use AI everywhere, and “it has not removed the need to do true engineering, to break the problem down into consumable pieces to make sure that those pieces work.”

The failure mode is specific. “If you give an AI coding tool too large a task, it will fail. It doesn’t break it down. It doesn’t necessarily know what’s right and what’s wrong.” Overload it and “you’re pretty much guaranteed it’s gonna hallucinate and make mistakes.” As for why: “Because it gets, I don’t know, confused, I guess, is a way to put it. I don’t know if that’s an accurate way of describing it.” He’s collected the cautionary tales, including “a recent example where AI wiped out the databases of a particular company, which I’ll stay away from names.”

The discipline that survives is old: decompose, give “enough context,” then verify. “You have to make sure that the code it generates is correct, that it is doing what you expect it to do.” Some of that is testing; some is the spec-driven school, where “the spec really becomes the source of truth as opposed to source code.” And none of it holds still: “If you wanna be successful in this environment, you need to keep up with the fact that the models are getting better.”

Ninety-seven-ish percent

Late in the conversation I asked whether this era belongs to the startups or to the incumbents.

“Both. I mean, they’re both true.”

He’s seen this movie. Of the dot-com boom: “It is also true that the biggest companies that exist today came out of that era. But it’s also true that probably ninety-seven-ish percent of the companies disappeared.” Of the current field: “We’ve already seen some of the AI frontier companies have already sort of faded... it’s come down to, in the US at least, mostly about Anthropic and OpenAI,” both “being valued around a trillion dollars, which is kind of amazing as well.”

And then, instead of talking about companies at all, he swerved to the tool everyone is holding. “You have to think about the frontier model as being a sledgehammer. And not every problem needs a sledgehammer. While a frontier model may be applicable to solve a lot of these problems, it’s unlikely to be financially viable to solve a lot of these problems.”

He runs a business inside that asymmetry: “We have about a billion dollars in revenue, which is not small, but it’s not the forty-four billion that our friends at Anthropic are running at today.” He doesn’t expect the tool to get cheaper on its own. When I asked what happens if the labs stop subsidizing usage: “At some point they have to, right? Because they have to be in the business of making money as well. We all remember the early days of the big search engine and everyone saying, ‘Well, it’s never gonna make any money. They can’t figure out how to make money.’ Who knew that clicking on a link was a valuable thing to do?”

So: “If we’re not making decent margins, we won’t be here tomorrow. Doesn’t matter how cool our tech is.” He thinks this discipline is the sorting function for the whole field, the thing that “will separate those who are successful from those who might build something cool but can’t make a business out of it.”

All of which was the setup for the sentence he actually runs his organization by:

“What I tell my team is if we can solve something deterministically, we should, because the cost is lower and because determinism in observability is important.”

Strong signals

The rule has a second half: “If we wanna derive insights and actions and potentially self-healing to that, and we need a broader knowledge base, and the probabilistic nature of it is real, then we will use AI.” Between the two halves sits the decomposition skill from earlier: “As you break the problem down into consumable chunks, you may actually have different models deal with different parts of the problem.” Some chunks get the frontier model. Some get something smaller (”we will likely develop some of our own smaller AI models as well. It just makes sense”). Some get no model at all.

But why determinism, in his industry, beyond cost? “Observability historically has been about pretty strong signals. If a CPU is running at 100%, if memory runs out, if a disk drive fails, if a link fails, these are all very physical things that happen.” Every needle on the dashboards you page on was calibrated for failures like that: loud, physical, threshold-crossing. “Very different from an AI agent hallucinating. That requires a different kind of detection. AI models drift over time, and they start to make mistakes. They are inevitably probabilistic in their nature.” A hallucination pegs no CPU and fills no disk. And the million-line engineers are multiplying the surface area: “There’s just gonna be a lot more to observe. So the market itself will get bigger.”

A deterministic component is one you can make promises about. When your product is the thing that tells everyone else what’s wrong, the promises are the product.

The holy grail

He was specific about where he does want the sledgehammer. Today, when something breaks, a human triages the alerts, potentially across products from multiple companies, and hunts for root cause. With an SRE agent, “you’re going to be able to take more of that toil away from the individual human being who has to do it, and be able to analyze what’s coming in, gather those insights, determine what the action should be, and inevitably get to the holy grail of actually self-healing. That is coming and already happening in some cases.” Broad knowledge, genuine ambiguity, insight and action derived from mess: a problem shaped exactly like the tool. That’s the sledgehammer swung at something that is actually a boulder, and one he’ll pay the token bill for.

The stakes of getting the matching wrong aren’t abstract to him. “The first company I worked for doesn’t exist anymore. They never made that transition.”

The question Michael left me with is cheaper to ask than the alternative. A million lines in six months means the swings are already happening. The next time a problem crosses your team’s desk, the question isn’t whether the model can solve it. It almost certainly can. It’s whether this problem is worth a sledgehammer, and what you’ll be able to promise about the answer.

High Output is brought to you by Maestro AI. Michael’s rule is to solve problems deterministically when you can and save frontier models for the work that actually needs a sledgehammer. But across a real engineering organization, those choices disappear into thousands of conversations between engineers and coding agents. A PR can show you what shipped. It can’t show you which model did the work, where the tokens went, or how much correction and rework happened along the way. That context doesn’t live in tickets or dashboards either. Your Anthropic bill tells you something is happening. Maestro tells you what. Maestro plugs into Claude Code, Codex, and Cursor and turns that hidden work into an honest ledger: model and token spend per person, per category, and per PR, alongside what the work produced. Now you can see whether the sledgehammer earned its cost.

Visit https://getmaestro.ai to see spend by category and output per dollar: an honest ROI ledger for AI-assisted engineering.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit maestroai.substack.com