Discussion about this post

User's avatar
Halfcycle's avatar

Step 7's regression suite, past failures frozen into tests, is the part most homemade harnesses skip and the one that compounds. The question we kept running into while building Halfcycle: what about the failures someone else's project already hit? A suite grown from your own traces only knows your own traces, so every team relearns the same lessons. Did any of Decode's benchmark failures come from other people's runs, or all from yours?

Andrii Tretiak's avatar

"Same model, better harness" matches what I saw in my own setup. The biggest jump for me wasn't a new model, it was cutting my agent system from 42 agents to 13, moving the fixed steps into hooks and scripts, and loading skills only when a task needs them. Less stuff in the context, and the agents stopped mixing up rules from different tasks.

5 more comments...

Ready for more?