1 min read
Reading 10,000 open-ended answers: where AI helps and where people still decide
Language models can sort feedback in hours, not months. The judgement about what it means is still a human job.
The ESS team
Open-ended questions are where survey respondents tell you what you did not think to ask. They are also the part of the dataset that most often goes unread, because reading thousands of answers by hand takes weeks.
What AI does well
Grouping similar answers into themes, even when people use different words.
Working across languages, so an answer in Portuguese and one in Spanish land in the same theme.
Counting how often each theme appears in each group, country or cohort.
What still needs people
A model does not know which finding matters for your program. Evaluators decide which themes to merge, which are noise and which deserve a recommendation. We also read a sample of answers by hand in every project to check that the themes are faithful to what people wrote.
The goal is not to replace reading. It is to make sure every answer is read, and that people spend their time on interpretation.
Used this way, AI turns open-ended questions from a burden into the most useful part of the survey.
Topics
- AI
- Surveys
- Evaluation
