Hey fix the price of the models or even better revert it back to how you had it.  Loved the harness when you weren't gouging us on the model costs.   Until you fix that...no one cares about these performance benchmarks.

There's no point in comparing that. As long as you use a model and harness from the same company, it will always be cheaper than relying on middlemen like Microsoft. But the more expensive AI gets, the more people will do Bring Your Own Key or host their own models. In that case, the efficiency of the harness is very important to know.

they wont because they are beat regarding price. not comparing cost is a choice

I like GitHub copilot but it’s too expensive compared to Claude.  Great post but I’d like to see more comparing costs.  We took our usage data pre june 1st and worked out future costs of Claude CLI vs GitHub Copilot CLI and it’s a no brainer for us.  Can you incorporate more in the cost comparison angle?