• Study Scope & Setup:

  • Evaluated 21 model–harness combinations using 7 models across 3 agent harnesses (Claude Code, Codex CLI, and Pi) on two benchmarks: SWE-bench Lite and Terminal-Bench 2.0.
  • Controlled variables by testing on identical sampled tasks, using native high-effort settings, capping attempts at 100 turns, and normalizing direct API token pricing.

• The "Harness Tax" Phenomenon:

  • Changing the harness has minimal effect on task resolution rate (within ±2% on SWE-bench Lite and ~±5% on Terminal-Bench 2.0), but dramatically alters token expenditure (diverging by up to 2×–5× for identical success rates).
  • This overhead is driven by initial context bloat, verbose system prompts, and heavy tool schemas injected into the context window before work begins.

• Minimalist Harnesses Remain Highly Competitive:

  • Pi, a lightweight open-source harness, proved competitive with or superior to heavy proprietary harnesses in both resolution rate and token efficiency.
  • For example, running Claude Fable 5 resolved ~97%–98% of tasks across both Claude Code and Pi, yet Claude Code cost approximately twice as much ($1.33 vs. $0.67 per attempt).

• First-Party Harness Mismatch:

  • Models do not necessarily perform best inside their creator's native harness.
  • In 9 out of 12 head-to-head comparisons across Anthropic and OpenAI models, third-party or alternative harnesses delivered higher success rates than the model provider's default CLI.

• Core Practical Takeaway:

  • Accepting a coding agent's default harness incurs an unnoticed "harness tax" in the form of inflated API bills or premature subscription rate limits.
  • Agent benchmarking should decouple models from scaffolds, treating harness complexity and tooling architecture as separate empirical engineering trade-offs.

Not sure of the truth of the story either @Josh, but thanks for the trip down memory lane. My math teacher gave me that article when I was in the 12th grade because he knew I had been variously killing time in his math classes since 9th grade memorizing the first 8,000 digits of pi and reading for fun.

Good picture (and article) on parts of pi 4

This part needs input from the PI! How do we want to track down the source data and encourage those to be published somewhere?

Actually, he first used π in 1727, and it meant 6.28.., not 3.14..

Best method I’ve seen, so far.

Some people are porting Apple's Swift programming language to the Raspberry Pi. At the time of this post in December 2015, they had the compiler running on RPi 2 with Ubuntu Linux. They did not yet have the Foundation libraries, the Swift Package Manager, or a version for RPi 1, and it was not certain whether it would run on Raspbian Linux.