The company said that in testing, 95 percent of Fable sessions ran entirely on Fable responses, without falling back to Opus 4.8.

We spent days loading the system with hundreds of threads, refining rough edges and polishing corners that developers may never see.

That is called profiling, not performance testing. Performance testing should ensure that a piece of code runs within a desired amount of time, given a certain context, before the new code goes into production.

I would like to make an appeal to core developers: all design decisions involving involuntary session creation MUST be made with a great caution. In case of a high-load project, avoiding to create a session for non-authenticated users is a vital strategy with a critical influence on application performance. It doesn't really make a big difference, whether you use a database backend, or Redis, or whatever else; eventually, your load would be high enough, and scaling further would not help anymore, so that either network access to the session backend or its “INSERT” performance would become a bottleneck. In my case, it's an application with 20-25 ms response time under a 20000-30000 RPM load. Having to create a session for an each session-less request would be critical enough to decide not to upgrade Django, or to fork and rewrite the corresponding components.

Performance Benchmarking What it is: Testing a system under certain reproducible conditions Why do it: To establish a baseline which can be tested against regularly to ensure a system’s performance remains constant, or validate improvements as a result of change Answers the question: “How is my app performing, and how does that compare with the past?”

It is also good practice to make sure that your load testing is functionally correct. Both the performance and functional goals can be codified using thresholds and checks (like asserts).