The most expensive bugs in survey work are not typos. The expensive ones are the design choices that look fine on the page, get sent to a thousand respondents, and quietly tilt the numbers a few degrees in the wrong direction. The data comes back clean enough to analyze, the report goes to the dean or the executive team, and the decision gets made on a foundation that nobody thought to inspect.
Five patterns account for most of the damage. None requires advanced training to spot. All of them get past first-draft review more often than they should.
1. The double-barreled item
An item that asks about two things at once. Respondents who agree with one half and disagree with the other have no honest response, so they pick the midpoint or the option that feels closest to a compromise. The composite score then averages two unrelated signals.
The fix is the rule "one idea per item." Read each item out loud and listen for the word and in the middle. Most double-barrels are caught at that step.
2. Leading premises baked into the prompt
A premise that prejudges the answer steers respondents toward agreement before they have a chance to evaluate the question. Even small modifiers (award-winning, popular, recent improvements to) shift the response distribution.
The fix is to strip every adjective from the prompt that does not name the object being rated. If the customer service is genuinely award-winning, the satisfaction scores will reflect it. The prompt should not.
3. Midpoint parking from too-wide reference periods
"Over the last year, how often have you..." invites a guess. Respondents who do not remember pick the middle option to be safe. The middle gets crowded with non-answers, and the distribution looks tighter than it is.
Reference periods that match how often the behavior actually happens produce sharper data. Daily behavior fits a one-week window. Quarterly events fit a six-month window. Asking about a year of behavior is rarely useful unless the topic is rare and memorable.
4. Inconsistent stems within one scale
A scale with mixed grammatical frames forces respondents to mentally restart on every item. The cognitive cost looks like measurement error in the data: alpha drops, item-total correlations weaken, and the report flags items that are not actually bad, just inconsistent with their neighbors.
Consistency does not mean monotony. The fix is to choose one frame per scale (an "I feel" stem, a "my [X] is" stem, a "how often do you" stem) and apply it to every item in the block. Different scales can use different frames.
5. Demographics at the top
Asking for race, gender, age, and tenure on the first screen accomplishes two things, both bad. Some respondents bounce because the survey now feels like a marketing form. Others stay but answer the substantive items in a more guarded way, having just been reminded which group they belong to.
The fix is the simplest one on this list: move demographics to the end. The respondents who drop out late have already given you the items that matter. The respondents who finish answer demographic items more honestly because the substantive part is over.
None of these are obscure
None of these mistakes requires obscure expertise to fix. All of them require somebody to read the survey from the respondent's seat once before launch, ideally out loud. A read-aloud test by the survey designer catches most of these in twenty minutes. A round of cognitive interviews with five people from the actual audience catches the rest.
For the longer treatment of these patterns and the design decisions that prevent them, see the Survey design guide.