the inference efficiency improved by nearly three orders of magnitude or 1,000x in less than 2 years
for - stats - AI evolution - Math benchmark - 2022 to 2024
stats - AI evolution - Math benchmark - 2022 to 2024 - 50% increase in accuracy over 2 years - inference accuracy improved 1000x or 3 Orders Of Magnitude (OOM)
there is essentially this Benchmark 00:09:58 called the math benchmark a set of difficult mathematic problems from a high school math competitions and when the Benchmark was released in 2021 gpt3 only got 5%
for - stats - AI - evolution - Math benchmark
stats - AI - evolution - Math benchmark - 2021 - GPT3 scored 5% - 2022 - scored 50% - 2024 - Gemini 1.5 Pro scored 90%