AI how-to guides Customers and sales
How to analyse survey comments with AI
Feed the open-ended answers from your surveys (NPS, CSAT, employee or product surveys) to an AI model that groups them into themes, scores sentiment and links each theme to segments and to the scores people gave. People define the questions, check the themes against real comments and decide what to do. Start with one survey and a theme list you validate by hand.
What AI does
- Reads every comment, in several languages, and groups them into themes and sub-themes.
- Scores sentiment per comment and per theme, not just per survey.
- Links themes to scores, segments, products, regions or channels to show what drives satisfaction.
- Picks representative verbatims for each theme so findings can be checked.
- Spots emerging issues between survey waves and alerts the right team.
What stays with people
- Deciding which questions the analysis must answer and which segments matter.
- Validating and naming themes, and merging or splitting them when the model gets it wrong.
- Turning findings into decisions and owning the follow-up with customers or employees.
How to put it in place
-
Define the questions and the segments
Agree with the people who will act on the results what they need to know: why scores drop, what differs by product or region, what is new since the last wave. Decide which survey fields and customer attributes the analysis can use.
-
Prepare and protect the data
Export the comments with their scores and segment fields, remove duplicates and test answers. Remove or mask personal data people may have typed into free text, and check the consent and purpose under which the survey was collected.
-
Discover themes and build a codebook
Let the model propose themes from a sample, then review them with someone who knows the business: rename, merge and split until they make sense. The result is a codebook with a definition and examples for each theme.
-
Classify all comments and check a sample by hand
Apply the codebook and sentiment scoring to every comment. Have people review a random sample per theme and fix the definitions where the model and the reviewers disagree.
-
Report by theme, segment and score, with verbatims
Show how often each theme appears, its sentiment and its relation to the score, broken down by segment, with real comments as evidence. Route the findings to the team that owns each issue.
-
Repeat every wave and watch for drift
Run the same codebook on each new wave so results are comparable over time. Review it periodically to add new themes and retire the ones that no longer appear.
What you need
- Survey exports with comments, scores and segment fields, from your survey or CX platform.
- A clear legal basis and purpose for analysing the responses, agreed with data protection.
- Someone who knows the business to validate themes and review samples.
- An owner for each type of finding who will act on it.
- Enough comments for themes to be meaningful; very small samples are better read by a person.
When it is not worth it
- A few dozen comments per wave: reading them is faster and more reliable.
- When nobody will act on the findings: fix ownership of the follow-up first.
- Surveys whose closed questions already answer what you need to know.
Risks and how to control them
- Themes that sound right but are not: always show the verbatims behind each theme and review a sample by hand.
- Personal data in free text: mask names, contact details and sensitive information before analysis and limit who sees raw comments.
- Sarcasm, irony and mixed comments misread: score sentiment per theme within a comment and check edge cases.
- Results that change because the model or prompt changed: fix the codebook, version the setup and compare waves with the same method.
How Thinkia does it
We start by mapping surveys, support, social and sales notes to entities and journeys, with privacy in mind. With CX and product we agree the north-star metrics, the segments and who sees what, then analyse themes and sentiment across the whole customer voice, broken down by product, region, cohort or channel.
Insight packs are tested with stakeholders to correct blind spots and definition drift before they become routine. Findings reach the right team with priority and context, and scheduled briefings, alerts and exports fit your planning cycles instead of adding another static dashboard.
The Thinkia solution behind this guide
- AI customer insightsKnow what your customers actually think. Not what the survey says.
- AI customer supportYour customers get instant answers. Your agents handle what actually matters.
Thinkia products involved
Frequently asked questions
Can ChatGPT analyse survey comments?
A general chatbot can summarise a small batch, but results vary between runs and it is hard to trace. For recurring surveys you need a fixed codebook, consistent classification of every comment, links to scores and segments, and control over where the data goes.
What is the difference between sentiment analysis and theme analysis?
Sentiment tells you whether a comment is positive or negative; themes tell you what it is about. You need both, and ideally sentiment per theme, because one comment can praise the staff and complain about the price.
How accurate is AI at coding open-ended answers?
It depends on the comments, the languages and how clear the theme definitions are. Measure it yourself: have people code a sample, compare with the AI and refine the definitions until agreement is acceptable for your use.
Does it work in several languages?
Yes. Current models can read comments in many languages and map them to the same themes, so you can compare countries. Check a sample in each language, because quality is not the same in all of them.
Is it legal to analyse survey comments with AI?
Generally yes, if the analysis fits the purpose for which the data was collected and data protection rules are applied. Mask personal data in free text, control where the data is processed and ask your data protection officer when in doubt.