The challenge
The organization had a triannual needs assessment cycle. Past cycles produced a 60-page funder report with a survey appendix and a separate qualitative report from focus groups, and the two appendices rarely talked to each other.
Funders had started asking pointed methodological questions: how representative was the survey sample, what was the reliability of the construct scales, did the qualitative themes corroborate the quantitative findings or contradict them. The previous report could not answer the third question because the two methods were never integrated.
How they designed the survey
The organization redesigned the cycle around a single mixed-methods question set. The survey carries 24 Likert items across six need domains (housing, food, healthcare, transportation, childcare, employment) plus four open-ended items asking, in plain language, what would make the biggest difference.
Three in-person focus groups (n = 8 each) ran in parallel using the same six domains as a discussion guide. The qualitative team coded transcripts using grounded theory; the survey team ran AI theme extraction on the 1,612 open-ended survey responses. The two coding frames were compared at the end as a corroboration check.
What the data showed
1,612 households responded to the survey (8.9% rate without incentives). The six domain composites all came in above α = 0.83 with KMO above 0.79. AI theme extraction on the 1,612 open-ended responses produced 11 themes; manual review condensed them to 7.
Of the top five priorities the survey identified (housing affordability, public transit gaps, after-school childcare, prescription costs, and full-time employment for caregivers), four also showed up as top themes in the focus group transcripts. The fifth (prescription costs) showed up in two of the three focus groups as a secondary concern. The corroboration table became the centerpiece of the funder report.
"Funders read mixed-methods reports skeptically because the qual and quant rarely line up. Ours did, and the table that showed the convergence was the page reviewers commented on. The survey reliability statistics were what made the table believable."
At a methods glance
| Survey sample | n = 1,612 households (8.9% response rate) |
| Survey instrument | 24 Likert items, 6 domains, 4 open-ended |
| Focus groups | 3 groups, n = 8 each, 90 minutes each, recorded and transcribed |
| Reliability | Domain αs 0.83 – 0.90; KMO 0.79 – 0.86 |
| Theme convergence | 4 of 5 top survey priorities also top focus group themes |
| Export | Word methods appendix, PDF report for funders, Excel domain rollups |
What they did with the result
- The organization restructured its strategic plan around the five corroborated priorities. Each priority now has a multi-year action item set with measurable outcomes the next assessment cycle will track.
- The mixed-methods workflow became the standard for the organization's evaluation work. New assessments start with a single question set used across both modalities, so the corroboration table can be built without retrofitting.
- Two regional foundations expanded their funding after reading the report, citing the methodological transparency and the convergence of qual and quant evidence as the deciding factors.
Read more about how mixed-methods teams use ReliCheck →