Essay accuracy, measured against state examiners
Texas publishes real student essays together with the scores its trained examiners gave them. We graded all 78 of them, grades 3 through 8, with the same engine and the same built-in STAAR rubrics every account uses, and compared our scores with the examiners’. This page is the result: the numbers, how we got them, and what they do not cover.
9 in 10
essays within one point of the examiner
on the 0-5 total score, across all 78 essays
97-99%
criterion scores within one point
Organization and Development 99%, Conventions 97%
0.78
agreement with the official examiners (QWK)
about 10% above the accepted bar for automated marking, 0.70
4 in 10
exact same total score as the examiner
most of the remainder sit exactly one point away
When our score differs from the examiner’s, it is slightly more likely to be a point low than a point high. We would rather err strict than inflate a grade, and either way the teacher has the final say on every score.
How we tested
- The essays are the 78 student responses in the Texas Education Agency’s 2025 released STAAR constructed-response scoring guides: 13 per grade, grades 3-8, each published with the score an official examiner awarded and the examiner’s reasoning.
- Each essay was graded against the same STAAR rubric built into the product, with the reading passage and prompt supplied as context, exactly as a teacher’s upload would be.
- Grading used the default model every account gets, including free accounts. No special configuration, no retries, no discarded runs.
- The engine never saw the official scores. We compared afterwards.
- We publish the aggregate numbers. The student essays are TEA’s and are not reproduced here.
Quadratic weighted kappa is the statistic exam boards use to measure how well two scorers agree; 0.70 is the commonly accepted bar for automated marking systems. The released essays are typed, so this measures scoring rather than handwriting reading. Benchmark run August 2026 on the engine currently in production.
See it on a real essay
Several of the rubric pages show a full essay graded against that exact rubric, with every mark and comment the engine produced. Or upload one of your own.