Measuring the field across labs
Epoch AI studies the inputs, capabilities and economic effects of AI development. It maintains databases of models, training compute, hardware and data centers, runs model evaluations and publishes forecasts. Those are different kinds of evidence: a measured benchmark score, an estimated training cost and a forecast about future growth should not be read as equally certain observations. [1]
Its history begins with a volunteer group collecting model data in 2021, followed by a 2022 paper on training-compute trends and the formation of Epoch AI. Its current work makes those comparisons available beyond the companies building the models, so readers can inspect the resources behind a capability claim as well as the claim itself. [1]
FrontierMath and the public record
Epoch introduced FrontierMath on November 8, 2024, after submitting its paper to arXiv the previous day. The benchmark uses original mathematics problems written by specialists, with answers that can be checked computationally. The launch included representative samples while keeping the evaluation problems unpublished to reduce contamination. [3][4]
That design makes FrontierMath useful for comparing mathematical reasoning under a defined evaluation setup. A result on its problems does not by itself establish general intelligence, performance on every kind of mathematical research or the ability to automate an entire scientific job. The paper describes a particular test, with particular tasks and verification methods. [4]
Funding and access matter to interpretation
The announcement page now discloses that OpenAI commissioned and owns 300 FrontierMath questions and had access to their statements and most solutions, with a subset of solutions withheld for evaluation. Epoch retained the right to conduct and publish its own evaluations. Those terms matter when interpreting results from the benchmark; the current disclosure should not be mistaken for the information available at the original launch. [3]
Epoch identifies itself as a charitable nonprofit and publishes grant and collaboration disclosures. Its role as an outside research organization is useful context, but the funding and access terms of each project remain part of the evidence readers need to assess its findings. [2]
Sources
- Making sense of AI
Epoch AI · retrieved Oct 23, 2026
- Transparency
Epoch AI · retrieved Oct 23, 2026
- FrontierMath: a benchmark for evaluating advanced mathematical reasoning in AI
Epoch AI · Nov 8, 2024
- FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
arXiv · Nov 7, 2024