The challenge
Six weeks after a fall-line redesign, NPS dropped 11 points. The product team needed to know whether the dip reflected a real product problem or a sampling artifact, and which features were driving it.
The marketing team had a generic NPS survey running in a CRM. The single-item NPS told them the mood; it did not tell them what to do. They needed a follow-up survey that could carry diagnostic weight, with reliability evidence and theme-level depth on the open-ended response.
How they designed the survey
The team built a 10-question survey on ReliCheck. The NPS item came first, followed by 8 Likert satisfaction items across four dimensions (fit, fabric, value, and customer service), and one open-ended item asking for the single thing the customer would change.
Three design choices kept the response rate up. The survey was capped at 90 seconds. Customers were sampled randomly from the redesigned line's 9,800 buyers, with no incentive offered (to avoid distorting the NPS estimate). The email subject line said 'Two questions about your recent order' to set expectations honestly.
What the data showed
1,243 customers responded over a 10-day window. NPS came in at 28 (down from 39 pre-redesign). The composite satisfaction score (averaged across the 8 Likert items) tracked NPS at the customer level, r = 0.71, giving the team confidence the two measures were pointing at the same underlying experience.
Item-level breakdowns showed two of the four dimensions (fit and fabric) carrying the dip; value and customer service held steady. AI theme extraction on the 1,243 open-ended responses grouped them into 7 themes; the top three (sleeve length, fabric weight, sizing chart accuracy) accounted for 64% of the responses.
"The reliability number gave us cover internally to act on the survey instead of waiting for another round. The composite was holding together, the dip was real, and the open-ended themes told us exactly what to fix."
At a methods glance
| Sample size | n = 1,243 of 9,800 (12.7% response rate) |
| Instrument | 1 NPS, 8 Likert (4 dimensions), 1 open-ended |
| Reliability | Composite α 0.86; dimension αs 0.78 – 0.86 |
| NPS validation | r(NPS, composite) = 0.71 at the respondent level |
| AI themes | 7 themes from 1,243 free-text responses; top 3 covered 64% |
| Export | PDF for executives, Excel for product, CSV for the data team |
What they did with the result
- The product team revised the sleeve length spec on three top-selling items, swapped the fabric weight on the outerwear category back to the prior season's spec, and rebuilt the size chart with photos at three reference points.
- The customer service team adjusted the post-purchase email to set fabric-weight expectations more clearly for the categories that customers were comparing across seasons.
- The marketing team scheduled a re-survey 90 days after the changes shipped, using the identical instrument, so the next NPS reading could be compared on equal footing rather than as two unrelated snapshots.
Read more about how customer feedback teams use ReliCheck →