Matteo works on cybersecurity for Node.js every day. AI has become essential for managing the flood of new vulnerabilities. Still, Fable blocks anything that interacts with a pointer, Anthropic's frontier models are off-limits for security research, and the US government can change the rules at any time. This leaves open source maintainers and security researchers without good options. So Matteo decided to run AI models locally. Luckily, the creator of Redis had already built just what he needed.
In this episode of The Node (and more) Banter, Luca Maraschi and Matteo Collina talk about Dwarf Star IV, Salvatore Sanfilippo's custom setup for running open-weight models locally. They discuss what it really takes to run a 304 billion parameter model on regular hardware. The conversation covers everything from the aggressive quantization technique that fits DeepSeek V4 Flash into 128 gigabytes, to the real challenges of self-hosted AI, like power outages, hot European summers, and NAS units overheating while you're away. It's a straightforward look at what AI sovereignty means in real life.
In this episode, we cover:
✅ How Salvatore's quantization technique keeps the routing layer at high precision while pushing the experts down to two bits, and why that's what makes a 304B model fit on a MacBook Pro or a GB10
✅ Why frontier models are effectively blocked for Node.js security work, and why that may be putting everyone at more risk, not less
✅ The freshman who found a valid, convoluted vulnerability using Kimi, and what it says about AI democratizing security research on both sides of the fence
✅ Why the spike in downloads for Keet, data retention laws, EU chat control, and companies running their own AI hardware are all the same story: infrastructure freedom is winning
The takeaway?
The gap between what frontier models can do and what they'll let you do is becoming a real problem for legitimate security work. Local models close that gap, but the entry cost is still high, and the capability gap is real. What's changing is that the reasons to go local are no longer just technical. They're political.
