Essay accuracy, measured against state examiners
Texas publishes real student essays together with the scores its trained examiners gave them. We graded all 78 of them, grades 3 through 8, with the same engine and the same built-in STAAR rubrics every account uses, and compared our scores with the examiners’. This page is the result: the numbers, how we got them, and what they do not cover.
In short: we graded the 78 student essays that the Texas Education Agency released with official examiner scores in its 2025 STAAR constructed-response scoring guides (grades 3 through 8, 13 per grade), using the same engine and the same built-in STAAR rubrics every account uses, and compared our scores with the examiners’ afterwards. Nine in ten essays came back within one point of the examiner on the 0-5 total score, four in ten matched the examiner exactly, and agreement measured as quadratic weighted kappa was 0.78, above the 0.70 bar commonly accepted for automated marking. The essays are typed, so this measures scoring, not handwriting reading, and the teacher can change any score.
9 in 10
essays within one point of the examiner
on the 0-5 total score, across all 78 essays
97-99%
criterion scores within one point
Organization and Development 99%, Conventions 97%
0.78
agreement with the official examiners (QWK)
about 10% above the accepted bar for automated marking, 0.70
4 in 10
exact same total score as the examiner
most of the remainder sit exactly one point away
When our score differs from the examiner’s, it is slightly more likely to be a point low than a point high. We would rather err strict than inflate a grade, and either way the teacher has the final say on every score.
How we tested
- The essays are the 78 student responses in the Texas Education Agency’s 2025 released STAAR constructed-response scoring guides: 13 per grade, grades 3-8, each published with the score an official examiner awarded and the examiner’s reasoning.
- Each essay was graded against the same STAAR rubric built into the product, with the reading passage and prompt supplied as context, exactly as a teacher’s upload would be.
- Grading used the default model every account gets, including free accounts. No special configuration, no retries, no discarded runs.
- The engine never saw the official scores. We compared afterwards.
- We publish the aggregate numbers. The student essays are TEA’s and are not reproduced here.
Quadratic weighted kappa is the statistic exam boards use to measure how well two scorers agree; 0.70 is the commonly accepted bar for automated marking systems. The released essays are typed, so this measures scoring rather than handwriting reading. Benchmark run August 2026 on the engine currently in production.
Common questions
How accurate is the NudgeLearn essay grader?
On 78 real student essays with official Texas examiner scores, nine in ten of our scores were within one point of the examiner on the 0-5 total, four in ten matched exactly, and criterion-level scores were within one point 97-99% of the time. Agreement with the examiners, measured as quadratic weighted kappa, was 0.78, above the 0.70 bar commonly accepted for automated marking.
What was the grader tested on?
The 78 student responses in the Texas Education Agency’s 2025 released STAAR constructed-response scoring guides: 13 per grade, grades 3 through 8, each published with the score an official examiner awarded. Each essay was graded against the STAAR rubric built into the product, with the reading passage and prompt supplied as context, using the default model every account gets. The engine never saw the official scores. Benchmark run August 2026.
What does the accuracy study not cover?
It covers essays only, not math. The released essays are typed, so it measures scoring rather than handwriting reading. The sample is 78 essays from one state’s assessment. When our score differs from the examiner’s, it is slightly more likely to be a point low than a point high, and in every case the teacher reviews the score and can change it before it reaches the student.
See it on a real essay
Several of the rubric pages show a full essay graded against that exact rubric, with every mark and comment the engine produced. Or upload one of your own.