Why Model Councils Beat Model Hype

There is so much hype around the Fable 5 model being reintroduced, but what so many people have now figured out is that you can get even better performance with other models through proper harnesses and model councils.

If you are an engineer who pays close attention to what the models are actually doing in the background, the Claude models become one of the scariest to use, especially Opus, which is horrible at respecting the instructions and plans.

I know there are a lot of Anthropic fans out there who will say you’re just using it wrong, but I’ve been collecting screenshots of the countless times Claude admits to making huge mistakes by essentially jumping to conclusions and not following our frameworks.

It’s like Claude/Opus is this brilliant kid who panics when something goes wrong and just starts stumbling instead of taking the deep breaths and going back to the deep level of planning that it should.

I’m petrified whenever I hear somebody who’s not an engineer say they vibe coded some mission-critical app, especially if they did it purely with Claude Code. If that’s you, just remember to require “E2E TDD” for all changes, which stands for “end-to-end test-driven development”, which is the only way to ensure that LLMs build software properly.

I still use Claude Code, but I’m using it as part of the model council and having my Codex CLI do most of the heavy lifting.

Codex is much more of a “measure twice, cut once” type of system, so I’ve actually been able to have Codex set up AWS infrastructure perfectly, which is something I would never trust Claude to do.

In the near future, the open-source models will be catching up, which is another reason to focus on proper harnesses rather than trying to have everything run through one model provider’s way.

If your company is trying to implement AI, please make sure the consultants you bring in are actual engineers who understand frameworks and how to think in systems.

I see far too many AI consultants who can do more harm than good. I think, with powerful tools like Claude Cowork and Claude Code, you are one wrongly-worded prompt away from exposing critical documents.