GSM8K overview and key facts (October 2026)

GSM8K is a widely used evaluation dataset of 8,500 English math word problems at grade school level. It checks whether a language model can reason through several calculation steps.

The dataset has 7,473 training and 1,319 test problems in two configurations: main and socratic, which adds guiding sub-questions. Freelance contractors wrote the first problems, and the set was scaled up with the data labeling platform Surge AI. Workers re-solved every problem to catch errors; OpenAI estimates that about 1.7 percent still contain errors or ambiguities. The data and code are released under the MIT license.

• Tests calculation reliabilityTeams that plan to use a model for quotes, invoices or planning tasks can check how reliably it solves multi-step arithmetic. Errors on simple problems are a warning sign for business calculations.
• Trains step-by-step reasoningThe training split with worked solutions can be used to fine-tune smaller models to show their reasoning. This makes answers easier for staff to check. Staff can follow each step and spot where a calculation went wrong.
• Tracks model versionsBecause results on GSM8K are widely reported, teams can compare a new model version with published figures and their own earlier runs. Sudden drops point to changes in the model that deserve a closer look.

• Socratic variant adds guiding sub-questions to each solution.
• Very widely downloaded evaluation set with an MIT license.

GSM8K was created and published by OpenAI in the United States; the problems are in English.

Many current models score very high on GSM8K, and the test problems are likely part of some training data, so the dataset separates weak models better than strong ones. Use it as a basic check and add harder or company-specific tests. The problems are in English only.

No reviews yet. Be the first to share how your team uses it.

Every review shows its verification level. We read each one before it goes live. How we check reviews