Grading essays and other unstructured responses is a significant challenge. Using human scorers is time-consuming and expensive. Many organizations have huge item banks required for adaptive assessments and formative learning. Use of AI for essay scoring typically needs training on large answer sets and is impractical for many.

A realistic approach is to use generative AI. This session from a technology company leader and independent psychometrician, outlines a large-scale study of such approaches for delivery of tens of millions of essay grades. The study has been used to develop software that uses generative AI to create scores and textual feedback against rubrics for essays, which can be used to grade formative assessments or as input into human-graded higher-stakes assessments.

In this session we will share experience on which LLMs are effective and how generative AI compares against humans and trained models. We will also share initial results on grading short answer questions and with answers like video, audio, drawing and project work.

Join us for insights on the real-life capabilities of generative AI for scoring essays and unstructured answers, and to help to understand the future of grading - will it be human, generative AI, trained AI or something else