A Little Model Goes A Long Way

I was recently having a discussion with someone about AI models. He kept saying things about how, if I'm not using the latest & greatest models like Fable 5 and Sol, that I'm basically wasting my time and that I'll never get anything substantive from AI.

Don't get me wrong. Those are both good models, but they're overpriced and too heavy for most peoples' actual needs.

When I told him I don't use frontier models for my day-to-day, he looked at me like I'd said I still use a flip phone.

So, what do I use?

Well, I stick with open models that I can self-host - or I use Symbient AI, a 7b model I personally trained from scratch back in 2022.

For coding I previously used qwen3.6:27b base, but I recently moved to ThinkingCap-qwen3.6:27b from Bottle Cap AI. In my experience, ThinkingCap is much faster and more reliable than base-qwen. It also uses far less time & tokens during reasoning but still gets the same result.

Here's an actual usage example.

Last night before bed I gave my coding harness, Forage, instructions to create a super simple CAD/CAM app I can use to create toolpaths for my CNCs. Using ThinkingCap, Forage jumped into action. When I woke this morning the app was waiting. A complete app written entirely by AI while I slept.

The best part? A local, self-hosted model did that. Not a frontier model. One of the little guys running on MY hardware.

For non-coding tasks I use gpt-oss:120b or its little brother, gpt-oss:20b. Both are capable of powering my Chief of Staff Agent, and many others that do things like monitor my emails, calendars, CRM, and other tools, provide me with daily briefs, alert me when I've missed something, and save me hours of work every week. Those are hours I can spend on other things. And, again, this is all possible from local, self-hosted models.

I'm not knocking the big models - they're genuinely impressive. But for most of what business owners actually need day to day, they're overkill. Expensive overkill, running on someone else's servers, with someone else's rules.

And that's really the part that gets glossed over. When you're sending your data to frontier providers, you're trusting a lot of things you can't verify - that your data isn't training the next version, that no one internally is looking at your conversations, that access won't change or get restricted, that you won't get boxed into their ecosystem down the road.

Self-hosting flips that. Your data stays yours. Your business keeps running regardless of what any provider decides tomorrow. And honestly, the "small" models are more than capable when you pair them with the right tooling.

Having the ability to self-host models that run my business and do so where I can trust them is priceless. No really - it doesn't cost me anything aside from the electricity to run the hardware.