Stephen T’s Blog Spot

A common and well-meant step in qualitative research is to take your findings back to the people who gave you the data and ask, does this ring true? It feels like the ultimate check. Who better to confirm an interpretation than the people it is about? This is member checking, also called respondent or participant validation, and it is often treated as the gold standard of qualitative rigor. Return your account to participants, and if they endorse it, it must be accurate, and if they reject it, it must be wrong. The instinct is good; however, the logic in its strong form does not hold. Agreement and accuracy are not the same thing, and neither are disagreement and error.

Consider why a participant might agree with an account that is not faithful to the data. They might agree because it is easier than arguing, because the researcher is an authority, because they do not want to contradict a polished document, or simply because they are being polite. Agreement can be social rather than epistemic. And a study that only checks back with the participants likely to nod along is not validating anything; it is cherry-picking its own confirmation.

Now consider why a participant might reject an account that is accurate. Good qualitative analysis usually produces interpretations that go beyond what any single person would say about themselves. It connects themes across many accounts, names a pattern no one participant lived as a whole, and situates individual stories in a larger frame. A participant may not recognize their own words in that higher-order reading, and non-recognition is not the same as the interpretation being wrong; the analyst’s job was never to simply transcribe each person’s self-understanding. Participants also sometimes reject a finding precisely because it is accurate and unflattering, and people change, so by the time you check back they may reinterpret the experience they first described.

Underneath all of this is a single confusion. Member checking quietly merges two different questions: is this account faithful to the evidence, and does this participant endorse it? Those can come apart in both directions, which is why endorsement cannot serve as a truth test. It is the same lesson an earlier post drew about triangulation, that agreement across sources is not automatically a stamp of validity: a thing can feel confirmed without being correct.

None of this means you should stop. It means you should stop treating it as a verdict and treat it as what it is: a source of data and a relationship, not a vote. A participant’s reaction to your interpretation is itself rich material worth analyzing, and disagreement especially is informative. When someone pushes back on a theme supported by many others, the theme does not fall, but the pushback is a case worth understanding, the qualitative equivalent of hunting for the exception that tests your account. Member checking also catches plain factual errors, and it honors participants by giving them a say in how they are represented, which matters on its own ethical terms.

Doing it well means being clear about what you are asking. Are you checking facts, or seeking reactions? Treat the responses as data to be interpreted, not ballots to be counted. Expect disagreement and welcome it. And never let a participant’s endorsement or objection quietly override the weight of the evidence. What you should not do is write that member checking confirmed the findings, as though that sentence settled the matter.

In evaluation this lands close to home. We routinely take findings back to program staff, participants, and stakeholders, and the pressure is real to read agreement as validation and disagreement as a problem to manage. Both are errors. Those reactions are invaluable, as data, as ethics, and as a route to use, but a program’s staff endorsing a flattering finding does not make it true, and their rejecting a critical one does not make it false. Keep the reaction and the evidence in separate columns.

So here is my question. When you take your findings back to the people they are about, are you testing whether the account is true, or learning how they respond to it, and do you keep those two firmly apart?