Discussion about this post

User's avatar
Andrii Tretiak's avatar

The line about agents being fantastic at building what you decide and bad at deciding what to build is exactly my experience. I have a review agent that checks other agents' work against my rules, and when something slips through, I go into the code, trace why it happened and adjust the skill or the agent behind it myself. Your point about not using judges as validation hit too: a reviewer that just scores a broken output lower instead of saying it's broken is the quiet kind of failure that's easy to miss.

Santosh Shinde's avatar

Thank you for admitting the rubric punished your own writing habits. That is such a useful reminder to read the judge prompt, not just the score.

2 more comments...

Ready for more?