# Lesson 1: Foundations of Qualitative Data Analysis *Companion-podcast transcript • Sarah & Kiffer* *Office Hours episode to listen to after working through the lesson* --- **Sarah:** Here's a made-up result. In an invented set of twenty interviews about loneliness, seven people talk about small exchanges with strangers. One man times his walk so he passes the same crossing guard every morning. A woman says the pharmacist is the only person who uses her name all week. Another woman says she always uses the self-checkout. Is that a finding? **Kiffer:** It's a pattern, and a pattern is where analysis starts. It becomes a finding when someone can say why strangers keep turning up in these interviews, and can then test that explanation in every transcript, including the one from the woman at the self-checkout. **Sarah:** Welcome to Office Hours. I'm Sarah. **Kiffer:** And I'm Kiffer. The step from noticing a pattern to explaining it sits right at the centre of Lesson one, Foundations of Qualitative Data Analysis, so let's start there. **Sarah:** Question one. When does a pattern become analysis? Plenty of people accept the definition, patterns plus an explanation of why they're there, and then write results sections that are almost all pattern. **Kiffer:** It helps to know where the idea of thick description comes from. Clifford Geertz used it in 1973, and he borrowed it from the philosopher Gilbert Ryle. Ryle's example was two boys who each rapidly contract the eyelid of the right eye. For one boy it's a twitch. For the other it's a wink to a friend. A camera would record the same movement. The whole difference lies in what the movement means, and a thin description, eyelid contracted, can't capture it. **Sarah:** That's worth hearing, because people often take thick to mean long. A page of quotations can still be thin if it never says what the words are doing for the person who said them. So for our strangers, the thin description is "seven of twenty participants described small talk with strangers." **Kiffer:** Right, and a lot of results sections stop about there, with a theme name and three quotations. The thick version proposes why. Here's an invented line from the woman with the pharmacist. "She says, have a good day, Margaret. Some weeks that's the only time I hear my own name." One possible explanation is that these small exchanges are evidence of being seen. A stranger who greets you is proof that you still count as somebody out in the world. **Sarah:** And that's a hypothesis, which means it could turn out to be wrong. So how would we check it? **Kiffer:** That's where the systematic commitment comes in. You go back to all seven transcripts, and to the thirteen that never mention strangers, and ask whether the explanation fits each one. Which brings us to the woman at the self-checkout. Sarah, would you read her line? **Sarah:** Here's her invented line. "I always use the self-checkout. If the cashier asks how I'm doing, I'm afraid I'll actually tell her." So here's a question for everyone listening. Does that line break the being-seen explanation, or does it refine it? Take a few seconds. Pause the audio if you'd like more time. *(Pause)* **Kiffer:** I'd say it refines it. For her the exchange is about being seen too, and being seen is exactly what she's guarding herself from. She's worried her loneliness will show. So the revised explanation is that small talk with strangers works as a low-stakes test of being seen. Some participants seek it out because it confirms they still count. Others avoid it because it might reveal how much they need it. It's the twitch and the wink turned around. There, one movement carried two meanings. Here, one meaning drives two opposite behaviours. **Sarah:** And we only got to the revision because we went looking for the case that didn't fit. If I'd built the theme from the five most vivid quotations, I'd never have met her. **Kiffer:** That's also why the phrase "themes emerged" causes so much trouble. Braun and Clarke made the point in 2006. Describing themes as emerging, or as being discovered, is a passive account of analysis, and it hides the active role the researcher always plays. A paper that says its themes emerged usually leaves out the very steps a reader would need to check them, like the self-checkout case and the revision it forced. **Sarah:** Question two. What's actually in your dataset? The four-line inventory, with cases, unit, volume and what's missing, looks like bookkeeping. I'll admit I'd be tempted to skip it. **Kiffer:** It's quick, and it catches problems that are expensive to find later. Let's try it on a different kind of data. Imagine, and this is invented, a community health centre that keeps a comment box next to its drop-in program for a year. At the end there are two hundred forty handwritten cards. First line of the inventory. How many cases? **Sarah:** Two hundred forty. **Kiffer:** Two hundred forty cards. How many people wrote them? **Sarah:** Oh. We don't know. The cards are anonymous. One regular could have written thirty of them, and plenty of people none at all. **Kiffer:** So the case is a card, and that matters the moment you count anything. "Forty cards complain about the hours" could mean forty people or three. Second line, the unit of data. Mostly short texts, though some cards will have a drawing or a word underlined three times, so a few are still images as well. Third line, volume. Say the average card runs about twenty-five words. **Sarah:** Then it's two hundred forty times twenty-five, which is six thousand words. That's a small corpus. You could read every card more than once in an afternoon. Which leaves the fourth line, what's missing. **Kiffer:** And that's the one I'd like everyone to try. Whose voices aren't in that box? See if you can name two groups. Take a few seconds. *(Pause)* **Sarah:** People who can't easily write a comment in English, or can't write comfortably at all, because of literacy, eyesight or a shaky hand. And the people who stopped coming. If the program is struggling, their reasons are the ones you most need, and they're the least likely to be standing beside the box. **Kiffer:** Both good. I'd add the people who never came in the first place, and anyone who doubted that an anonymous card would really stay anonymous. So the box is a reasonable source for what regular users think, and a poor source for why people leave. **Sarah:** And why people leave is usually what a centre most wants to know. It's the "why did the program stall" kind of question that qualitative work is supposed to answer, and this box can't reach it. You'd need to go and find the people who stopped coming. **Kiffer:** Right, and the inventory tells you that before you've spent a month coding. Notice too that someone has to type the cards up before anyone codes them, and the drawings and underlining tend to vanish at that step. **Sarah:** Which is transcription, and transcription is an analytic act in its own right. **Kiffer:** It is, and interviews show it even more clearly. Here's an invented moment. A participant says "It's fine," then "I mean," then there's a long pause, and then "it's fine, I've got the cat." In intelligent verbatim, where fillers and false starts are removed, that can come out as "It's fine. I've got the cat." On the page, that reads as reassurance. With the pause and the repetition kept, it reads more like someone deciding whether to say more. **Sarah:** Question three. What kind of claim can a study of twenty interviews support? The four research goals look tidy on paper, exploration, description, comparison and model testing. In a real abstract they blur together. **Kiffer:** Let's start with a claim it can't make. Statistics Canada's Canadian Social Survey, collected in August and September 2021, found that about thirteen percent of people aged fifteen and older always or often felt lonely. That answers a "how common" question about the whole country, and twenty interviews can't answer it. The other kinds of claim are where it gets interesting. **Sarah:** So give me some. **Kiffer:** Here are four sentences from the abstract of an invented study of twenty people, chosen to vary in age and life stage. One. Participants described loneliness in terms of space, a quiet house, rooms nobody uses. Two. Older participants described loneliness in terms of space, while younger participants described it as being left out of plans. Three. Older participants describe loneliness this way because their homes hold the traces of people who are gone. Four. Programs that get older adults out of the house should therefore reduce their loneliness. Sarah, which research goal is each one, and can twenty interviews support it? **Sarah:** Let me work through them. One is description, and twenty interviews can support it. Two is comparison, which a deliberately varied sample is built for. Three explains the comparison, the thick part we were asking for in question one, so I'd call it the strongest sentence in the abstract. And four is model testing. If three is right, four follows from it. **Kiffer:** I'd stop you on two of those. Take four first. What would this study have needed in order to test that? **Sarah:** A hypothesis stated before the analysis, and data that could show it was wrong. And nobody in these interviews took part in a program, so nothing in the data could. Okay, I mislabelled four. It's a hypothesis the study generates, which makes it an output of exploration. It belongs in the discussion, as something a later study could test. **Kiffer:** Good. Now three. Here's one more detail about the invented sample. Every participant over sixty lives alone, and every participant under thirty lives with roommates or family. So here's the question for everyone. What does that do to sentences two and three? Take a few seconds. *(Pause)* **Sarah:** Age and living alone travel together, so the comparison can't pull them apart. The spatial language might come from coming home to a quiet house, at any age. Sentence three gives the credit to age, to a long life with people now gone, and the transcripts would look exactly the same if living alone were doing the work. So the sentence I called the strongest is the one the data support least. **Kiffer:** It's a persuasive sentence, which is part of the problem. A thick explanation reads as depth, and it still has to beat its rivals. If you could read just one more transcript to check it, whose would you choose? **Sarah:** Someone who breaks the link. An older person who doesn't live alone, or a younger person who does. **Kiffer:** Suppose there's one, and she's invented too, a woman of seventy-four who lives with her daughter's family. If she still talks about loneliness in terms of space, say the room at the end of the hall, that leans toward age or life stage. If she talks about being left out of the family's plans, that leans toward living arrangement. One case can't settle it. For this claim, though, it's the most informative transcript you could add. **Sarah:** So the honest abstract keeps each sentence at the goal its data can reach. Two becomes "older participants, all of whom lived alone, described loneliness in terms of space." Three becomes a possible explanation, with living alone named as a rival. And four moves to the discussion as a question for future work. **Kiffer:** That's a version another analyst could check line by line, and that idea of checking is where the next question starts. **Sarah:** Question four. What does replicable mean when two analysts read the same data? The standard from the lesson is coherence. A second competent analyst, following your documented procedure, should land somewhere coherent with you. **Kiffer:** And when a second analyst can't get there, the usual causes are undocumented decisions, idiosyncratic coding, or interpretations that outrun the data. The first is the easiest to fix. Let's stay with the invented study and read three entries from its codebook change log. Entry one, from a first read of five transcripts, before coding began. Created the code quiet house, for loneliness described as silence or empty rooms at home. Entry two, after transcript eleven. Split quiet house into silence as absence and silence as rest, because two participants described quiet as a relief. Entry three, after transcript sixteen. Dropped silence as rest. Too few cases. **Sarah:** That looks like a careful log to me. **Kiffer:** It looks careful. Two of those entries would still leave a second analyst stuck. So here's the question for everyone. Which two, and what's missing from each? Take a few seconds. *(Pause)* **Sarah:** Entry two says when and why, which is good. It doesn't say whether transcripts one to eleven were re-coded under the split. If they weren't, the first eleven only carry the old code, and any comparison across the dataset mixes two schemes. And entry three. "Too few cases" doesn't say how many, or where those excerpts went. If they were folded into silence as absence, the quiet house looks more uniform than it really is. **Kiffer:** Entry three is the one that worries me most. Dropping a code because it's rare can quietly delete the cases that don't fit, the very cases question one sent us looking for. A fuller entry would give the number, say three excerpts from two participants, say where they went, and keep them in a memo so they can be reported as a counterpoint. Then a second analyst can see the decision and argue with it, which is what replicable asks for. **Sarah:** A change log like that is how one analyst makes the work followable. In applied health research, the more common route is an agreement statistic between two coders, and I'm skeptical that it belongs in interpretive work at all. In reflexive approaches, the analyst's interpretation is the instrument. Braun and Clarke argued in 2021 that coding reliability measures make no sense for reflexive thematic analysis, because the researcher's subjectivity is treated as a resource. And under a reliability target, the codes that survive tend to be the literal ones. "Mentions a stranger" is easy to agree on. "A test of being seen" is harder. **Kiffer:** I think that's a real risk, and for one analyst doing reflexive work, I'd agree the statistic adds little. Where I'd push back is applied public health, where a codebook is often applied by several people, sometimes across sites. There, an early agreement check is the cheapest way to learn whether a definition means the same thing to two people. The useful output is the list of disagreements, and each one belongs in a change log like the one we just read, with what the team decided and why. **Sarah:** As long as the number doesn't end up in the abstract as proof that the findings are valid. That's my other worry. A public health reader sees a high agreement figure and stops asking questions, which undercuts the transparency the number was supposed to provide. **Kiffer:** Agreed, and there's a reason the number gets read that way. Lincoln and Guba set out their trustworthiness criteria in 1985 as counterparts to the conventional ones. Credibility answers to internal validity, transferability to external validity, dependability to reliability, and confirmability to objectivity. At most, an agreement statistic speaks to dependability, the consistency side. It says nothing about credibility, and readers with quantitative training often hear it as covering both. **Sarah:** So where do you land? **Kiffer:** I'd use agreement as a diagnostic when a team applies a codebook, report it code by code, and revise definitions wherever coders split. For interpretive analysis by one person, I'd meet the replicability standard through documentation: a positionality memo before coding, a change log and dated memos as the analysis goes, and a colleague whose job is to question each claim. **Sarah:** I'd skip the statistic more often than you would, and I can live with a shared rule. Decide before coding which kind of analysis you're doing, and choose evidence of rigour that fits it. Whichever you choose, show the reader where analysts or colleagues disagreed and what changed because of it. **Kiffer:** That includes disagreements with yourself, which is where the positionality memo earns its place. **Sarah:** A lot of the positionality statements I read are a list of identities and little else. What makes a memo useful? **Kiffer:** A list tells the reader who you are. A useful memo also says what you expect to find and what you might miss. Let's try one. Imagine, and this is invented, an analyst about to code loneliness interviews who's a nurse and has worked nights in long-term care. Sarah, draft a sentence for her memo. **Sarah:** Okay. "Because I've worked nights in long-term care, I expect to hear loneliness as empty rooms and long nights. I may give less weight to participants who describe quiet as a relief." **Kiffer:** That's the kind of sentence that helps. "I'm a nurse in my forties" tells a reader who she is. Your version tells them what to look for in her analysis. Suppose she's the analyst who wrote entry three of that change log. Her own memo predicted the decision to drop silence as rest. **Sarah:** So the memo and the change log check each other. Read side by side, they show a reader, and the analyst, exactly where an expectation may have shaped a decision, and that's worth a dated entry in the audit trail. **Kiffer:** Let's pull it together with three things to take away. **Sarah:** First, a pattern becomes analysis when you propose why it's there and then test the explanation in every transcript, including the one that doesn't fit, like the woman at the self-checkout. **Kiffer:** Second, keep each claim at the research goal its data can reach. A comparison in which two attributes travel together can't tell you which one matters, and a prediction about a program the study never observed is a hypothesis for later work. **Sarah:** And third, replicability rests on decisions a second analyst can see. A change log that says when, why, how many and whether earlier transcripts were re-coded, along with a specific positionality memo, lets someone follow your reading and disagree with it where it matters. **Kiffer:** If you'd like more practice, take the four-sentence abstract from this episode and write one for a pattern you notice in the course's loneliness interviews, with a sentence each for description, comparison, explanation and a hypothesis for later work. For the comparison, check which other attributes travel with the one you're comparing. **Sarah:** Next time, it's Lesson two, Research Questions, Theory and the Literature, where we look at how a qualitative research question takes shape and what theory brings to it. **Kiffer:** Take care, everyone. **Sarah:** See you in Lesson two.