Introduction and Overview

Lessons 1 through 4 gave you the upstream apparatus of a qualitative project: an operational definition of QDA, a research question, a sampling logic, and a data-collection procedure. You now arrive at the central act of the analysis. You have transcripts. You have read them. You sense that something is going on across them. You suspect there are patterns. The question of this lesson is: how do you find those patterns systematically, and how do you record what you find in a form another analyst could follow?

Two activities sit at the heart of analytic work on text: finding themes and building a codebook. These twin moves are the connective tissue of every major qualitative analytic tradition — from thematic analysis (Braun & Clarke, 2006; Braun & Clarke, 2019) through qualitative content analysis (Hsieh & Shannon, 2005) to the systematic coding manuals used in applied health research (Saldaña, 2021). They are tightly coupled but not the same. Finding themes is the discovery phase — the search for what recurs, what surprises, what is missing, what coheres. Building a codebook is the codification phase — turning what you found into operational rules that you (and other analysts) can apply consistently across the rest of the corpus. This lesson works through both, drawing on Chapters 5 and 6 of Bernard, Wutich, and Ryan (2017).

1.1 Themes, Codes, Categories, Concepts — A Precise Vocabulary

The terms theme, code, category, and concept get used interchangeably in much of the published qualitative literature, including in papers that are otherwise methodologically careful. Bernard, Wutich, and Ryan (2017, Ch. 5) are precise where most writers are not. Adopting their vocabulary will save you grief when you write your methods section and will save the reader confusion about what you actually did.

A theme is a recurring abstract idea you identify in the data. It is the analyst's product. “Loneliness as the cost of love” is a theme. “Spatial metaphor for absence” is a theme. Themes are typically expressed as short phrases rather than single words. They sit at a higher level of abstraction than the individual statements that supply evidence for them.

A code is the operational label you attach to a passage of text when you encounter an instance of a theme (or sub-theme). Codes are the working units of analysis: they are what you actually mark up in Taguette or NVivo. A single theme may be supported by several codes; a single passage may receive multiple codes. The code is the marker; the theme is what the markers, when assembled, are about.

A category is a grouping of related codes. In a hierarchical codebook, categories are the parent nodes and codes are the children. “Coping strategies” is a category that might contain codes for “phoning a confidant,” “watching comfort television,” “going to a coffee shop to be around people,” and “cooking food from home.” The category organizes; the codes do the marking.

A concept is the most abstract of the four. A concept is a theoretically meaningful idea that may organize many themes. Liminality is a concept. Embodiment is a concept. Structural exclusion is a concept. Concepts are typically borrowed from theoretical literatures and used to organize themes into something a discipline can argue about.

1.2 Ryan and Bernard's Twelve Techniques for Finding Themes

In a now-classic 2003 paper, Gery Ryan and H. Russell Bernard (2003) catalogued twelve techniques qualitative researchers use to find themes. The paper became foundational because it disaggregated what had previously been described as “immersion” or “reading deeply” into a set of specifiable operations. Bernard, Wutich, and Ryan (Ch. 5) reproduce and updates the list. Each technique answers a slightly different question about the data, and most projects use several in combination. Below we work through all twelve, with examples drawn from the loneliness dataset.

Technique 1: Repetitions

The most obvious technique. What words, phrases, ideas recur across transcripts? If multiple participants reach for the same word to describe something, that word is doing analytic work. In the loneliness corpus, the word chair appears in at least eight transcripts as a stand-in for an absent person: Linda's “Bill's chair,” Frank's chair, Helen's mention of the chair her brother used to sit in when he visited, and others. The word tired recurs across caregiver and bereaved participants in a way that goes beyond ordinary fatigue: it appears to mark a specific exhaustion of grieving-as-work. Repetition is what nearly all theme-finding starts with, and what every other technique builds on.

Technique 2: Indigenous Typologies and Categories (Emic Terms)

What categories do participants themselves use? When a participant reaches for a non-English word, a slang term, or a phrase that operates as a category in their world, the analyst should pay attention. In the loneliness corpus, Amira uses the Arabic word wahda to name something that the English category of “loneliness” cannot fully hold — a loneliness specific to having been the sole survivor of a particular life. Aarav uses ekantam and ekakitatvam as paired Sanskrit-Hindi terms that distinguish chosen solitude from involuntary aloneness. Marcus speaks of “code-switching” loneliness — an experience he names that has no neat one-word English equivalent. These emic categories are gifts; they often become themes that organize an entire section of your eventual paper.

Technique 3: Metaphors and Analogies

People describe abstract experiences (especially feelings) by reaching for concrete images that map onto them. Cataloguing the metaphors a corpus uses is a powerful theme-finding move. The loneliness corpus is dense with spatial metaphors of absence and erosion: Maya feels “hollow” in her chest; Sarah describes loneliness as her “witness-less hours”; Helen describes it as “fading at the edges”; Frank uses imagery of a slow disappearance; Maya again talks about feeling she could “disappear and nobody would notice”; Linda talks about “walking around with that absence.” Read together, these metaphors converge: loneliness is repeatedly figured as a thinning or vanishing of the self. That convergence is a theme that you would never have seen if you had read the metaphors one at a time.

Technique 4: Transitions

What does the participant move to right after they say what they say? Transitions are turn-taking shifts and topic changes. They tell you what feels related in the participant's mind. In Linda's transcript, every passage about Bill's chair is followed by a passage about Rufus the dog: “I haven't moved it... the dog is what keeps me up in the morning.” The transition from the empty chair to the dog suggests an analytic linkage you might otherwise have missed: the dog is functioning as the affective replacement for the chair's absence. In Maya's transcript, the loneliness topic transitions repeatedly to her phone — phone, food, TV — signalling that her coping repertoire is digitally mediated.

Technique 5: Similarities and Differences (Constant Comparison)

Read two passages side by side. What is the same? What is different? This is the constant-comparative move that Glaser and Strauss made foundational to grounded theory and that you will meet again in Lesson 7. As a theme-finding technique, it is most useful when you have already identified candidate themes and want to test whether they hold up across subgroups. In the loneliness corpus, the bereaved-spouse loneliness of Linda (age 67, widow of three years) and Frank (age 81, widower of one year) share most features but differ on duration of mourning and the role of children: Linda's adult sons are present (David in Toronto, Michael in Calgary), while Frank's are estranged. The comparison sharpens the theme rather than dissolving it.

Technique 6: Linguistic Connectors

Words like because, since, as a result, therefore, that's why, and so are causal connectives. They mark places where the participant is explaining a relationship between events or states. Searching for them is a fast way to find passages where causal accounts of loneliness appear. Maya's transcript contains: “I just — I felt like everyone's life is still happening and I'm just here. Like I left and life closed up where I used to be.” The connective like here is not strictly causal but it is doing relational work. Linda's transcript: “If you love deeply for a long time, you will, eventually, be lonely deeply for a long time. The two are connected.” That explicit linkage of love and loneliness is a participant's own causal model, surfaced by attention to a connective.

Technique 7: Missing Data — What People Don't Say

What the corpus does not contain is as analytically informative as what it does. If you asked every participant about coping and three avoided answering, that pattern of avoidance is itself data. In the loneliness corpus, several participants conspicuously do not use the word “lonely” about themselves until very late in the interview, even though the interview is explicitly about loneliness. Marcus repeatedly substitutes “disconnected” or “invisible.” Older men in the corpus avoid the word in a way younger women do not. That gendered pattern of avoidance is a finding. Missing data is also useful at the level of what the interview guide asked but participants deflected: questions about professional help, in particular, are routinely deflected.

Technique 8: Theory-Related Material

Bernard, Wutich, and Ryan call this “looking through a theoretical lens.” If you bring a specific theory to the data — Cacioppo and Patrick's loneliness-versus-aloneness distinction, Weiss's social-emotional loneliness typology, structural/situational/existential models — you can deliberately scan the corpus for passages that confirm, complicate, or contradict the theory. This is more deductive than the previous seven techniques (which start from the data) and we will return to it when we distinguish inductive from deductive coding in Section 2. As an example: Cacioppo and Patrick (2008) argue that loneliness and being alone are dissociable states. The loneliness corpus is full of explicit articulations of that distinction (Maya: “loneliness is different than being alone”; Helen: “I have lived alone all my adult life… the loneliness I have now is different from the solitude I had at 50”). The theoretical lens both helps you see the pattern and gives you a way to write about it.

Technique 9: Cutting and Sorting (The Pile-Sort)

A physical, tactile method. Print out striking quotes from your corpus, one per index card. Spread them out on a table. Group cards that feel related. Move cards around as your sense of groupings evolves. Give each group a name. This technique externalizes the analytic work and uses spatial cognition to find patterns that screen-based reading misses. It is especially useful when you have 20+ candidate themes and need to consolidate them into a manageable set. Bernard, Wutich, and Ryan recommend actual cutting-and-sorting (literal scissors) at least once per project; the tactile experience is what makes it work. We will do a digital version of this in the Section 4 workflow.

Technique 10: Word Lists and KWIC (Keyword-in-Context)

Generate a frequency list of all words in the corpus and inspect the high-frequency content words (after removing stopwords like “the” and “and”). For words that catch your eye, generate a KWIC concordance: every occurrence of the word with a few words of context on either side. KWIC concordances are how you check whether participants are using a word the same way. If “chair” appears 23 times across the corpus, are all 23 instances chairs-as-stand-ins-for-people, or are some just literal pieces of furniture? The KWIC concordance answers that quickly. This is also the natural bridge to computational text analysis (Module 12); we will use quanteda in Section 4 of this lesson to do a small word-list and KWIC exploration.

Technique 11: Co-occurrence

Which codes (or words) appear together more often than chance would predict? Co-occurrence is the first move toward axial coding (covered in Section 3) and toward concept-mapping. If your code chair-absent-spouse co-occurs in nearly every transcript with codes for pet-as-companion or volunteer-coping, that co-occurrence is a finding. Co-occurrence is also how you build the network displays you will meet in Module 12. In R, co-occurrence is straightforward once your coded extracts are in a long-format data frame.

Technique 12: Metacoding (Codes About Codes)

Once you have a working set of codes, you can step up a level and label the codes themselves. Are some of your codes affective (about feelings) and others behavioural (about coping actions)? Are some emic (using participants' own words) and others etic (using your analyst-imposed terminology)? Sorting your codes by type is metacoding, and it often reveals structural patterns in your codebook that you would not have seen by staring at the codes one at a time. In the loneliness corpus, a useful metacoding move is to sort all codes into four buckets: experiential (what loneliness feels like), causal (what triggers it), responsive (what people do about it), and interpretive (what people think it means). The four-bucket frame becomes a candidate structure for the findings section of your eventual paper.

1.3 Working an Example Through Four Techniques

Take three passages from the loneliness corpus and run them through four of the techniques to see how theme-finding actually works.

Technique 1 (Repetitions): All three passages reach for embodied descriptions of loneliness. The words physical, chest, weight, body, voice, real recur. Loneliness is a bodily experience for these participants, not just a cognitive one.

Technique 3 (Metaphors): Three different metaphors, all converging on the same image. Maya: hunger, hollow, ache (interior absence). Linda: weight, half a life not there, walking around with absence (carrying something missing). Helen: fading, less real, voice strange (thinning of the self). The metaphors are not the same, but the analytic abstraction over them is: loneliness as the somatic register of an absence.

Technique 5 (Similarities and Differences): The three passages share embodiment but differ on what is absent. For Maya it is a future not yet built; for Linda it is a specific deceased person; for Helen it is the simple act of social contact. A theme that holds across the differences: the body registers absence as presence (you feel something missing as something there).

Technique 7 (Missing Data): Notice what none of the three say. None invokes psychiatric vocabulary. None mentions a therapist or a medication. None reaches for the word “depression” even though the descriptions are compatible with depressive symptomatology. The absence of clinical framing is itself a finding: these participants are describing loneliness as a normal embodied condition, not as a disorder. That has implications for how an intervention should be framed.

Four techniques applied to three passages have already given you the beginnings of a theme: loneliness as embodied absence, narrated outside clinical vocabularies. That theme can now become a code (or several related codes) in your codebook.

Introduction and Overview

Theme-finding (Section 1) is the discovery phase. Coding is what you do once you have themes. Coding turns themes into operational rules and applies them across the corpus consistently — the bridge between recognition and analysis that Boyatzis (1998) and Braun and Clarke (2006) describe as the heart of thematic analytic rigor. The two big design questions for coding are: where do the codes come from (inductive, deductive, or hybrid), and what does a defensible codebook look like (the architecture). This section addresses both.

2.1 Inductive Coding (Data-Up)

In inductive coding, you start with the data and let the codes develop from what you find. You read transcripts, mark passages that strike you, label the marks with provisional codes, and refine the labels as you read more. After three or four transcripts, you have a working set of perhaps 30–50 codes; you consolidate, merge, and rename until you have a coherent codebook of 8–15 codes you can apply across the remaining transcripts.

Inductive coding is the default for exploratory studies, for under-described phenomena, and for projects in the grounded-theory tradition (which we will meet in Lesson 7). Its virtue is that it stays close to the participants' own categories and avoids forcing the data into pre-existing analyst frames. Its risk is that, without theoretical anchoring, it can produce codebooks that are descriptive but not analytically interesting — what Charmaz (2014) warns against as “coding too close to the data.”

For most of the loneliness capstone work in this course, inductive coding is the appropriate starting point. The dataset is rich, the phenomenon is contested, and the participants speak in distinct vocabularies. Starting from their language is the right move.

2.2 Deductive Coding (Theory-Down)

In deductive coding, you bring a codebook to the data. The codes come from theory, from prior literature, or from a stakeholder framework (e.g., the WHO determinants of health, the CFIR implementation framework, the Cacioppo-and-Patrick loneliness model). You read each transcript and tag passages that instantiate the pre-specified codes. Codes that are not present in the data are recorded as absent.

Deductive coding is the right move when you are testing or extending an existing framework, when you are working in a confirmatory mode, or when your study is part of a multi-site collaboration that needs a shared coding scheme. Its virtue is that it produces results comparable across studies. Its risk is that you may miss things the data are saying that the framework was not designed to see.

An applied example: if you were coding the loneliness transcripts deductively using Weiss's (1973) social-emotional loneliness typology, you would have two pre-specified codes — social loneliness (deficits in a social network) and emotional loneliness (absence of a close attachment figure) — and you would tag each passage as one, the other, both, or neither. You would learn the distribution of the two types across the corpus and would have framework-comparable results. You would also miss the embodiment theme we developed in Section 1, because Weiss's framework does not contain it.

2.3 Hybrid Coding (the Practical Default)

Most contemporary qualitative health research uses a hybrid approach. You begin with a small set of deductive codes drawn from theory or prior literature (a provisional codebook), apply them to the first few transcripts, and let new codes emerge inductively as you read. The codebook grows from the bottom up while keeping its theoretical anchor.

The Fereday and Muir-Cochrane (2006) hybrid framework is a widely cited operationalization of this approach, and Hsieh and Shannon's (2005) "directed" qualitative content analysis is a closely related variant. Bernard, Wutich, and Ryan (Ch. 6) endorse hybrid coding as the practical default for applied health research because it preserves the inductive openness that makes qualitative work valuable while keeping the deductive anchoring that makes it interpretable by quantitatively trained reviewers.

The capstone milestone at the end of this lesson assumes a hybrid strategy: you will start with two or three theoretically motivated codes (perhaps experiential loneliness, causal accounts, and coping strategies, derived from the interview guide structure) and let the rest develop inductively from the first three to five transcripts you code.

2.4 The Anatomy of a Codebook Entry

A codebook is a structured document. Each entry describes one code with enough specificity that another analyst could apply it consistently (MacQueen, McLellan, Kay, & Milstein, 1998; DeCuir-Gunby, Marshall, & McCulloch, 2011). Bernard, Wutich, and Ryan (Ch. 6) recommend seven elements per entry. All seven matter; cutting any of them is the most common reason why intercoder reliability later turns out to be low.

1. Code name. Short, mnemonic, unique. Use hyphens or underscores, not spaces. Example: chair-absent-spouse.
2. Brief definition. One sentence. Example: “A physical object (chair, side of bed, photograph) that the participant marks as standing in for the absence of a deceased or departed partner.”
3. Full definition. A paragraph. When to apply the code; what range of cases it covers; how it relates to neighbouring codes. The full definition is what an analyst reads when in doubt.
4. Inclusion criteria. Bullet-pointed. What features must be present for the code to apply. Example: the participant explicitly names a physical object; the object is associated with an absent person; the participant gives the object affective weight.
5. Exclusion criteria. Bullet-pointed. What looks similar but does not count. Example: mention of a physical object without affective weight (“the chair in the corner”) does not count; mention of an absent person without an associated object does not count.
6. Positive example. A direct quote from the corpus that clearly fits. Example: Linda P05: “Bill sat in that chair every evening for thirty-some years. And it's still there. I haven't moved it. I haven't sat in it. I haven't given it away. It's just there. And every evening I look at it and it's empty.”
7. Negative example. A near-miss quote that does not fit. Example: Helen P11: “I have this walker because my hip gave out.” A physical object is named, but it is not associated with an absent person.

2.5 A Worked Codebook Entry

Here is a complete codebook entry for a code drawn from the loneliness corpus. We will return to this entry in Section 4 when we walk through the Taguette workflow.

2.6 The Codebook as a Living Document

Bernard, Wutich, and Ryan are clear that a codebook is not built once and frozen. You will revise it. The standard expectation is that revisions are documented in an audit trail that records: (a) the date of revision, (b) which codes changed, (c) what they changed to, and (d) the justification. When a codebook is revised midway through a project, the earlier transcripts must be re-coded under the new scheme — not the old one — or the codebook becomes inconsistent across the corpus.

The practical implication is that you should not commit to your final codebook until you have read most of the transcripts you intend to code. The Week 5 capstone milestone asks you to develop a preliminary codebook on 3–5 transcripts; the codebook you submit at Week 5 is not the codebook you will submit at Week 12. The Week 12 codebook will be the result of the Week 5 codebook revised at least twice on the basis of what the rest of the corpus reveals.

Introduction and Overview

Sections 1 and 2 covered the conceptual side of coding: what themes are, where codes come from, what a codebook looks like. This section addresses two operational matters that determine whether your coding is methodologically defensible. The first is coding mechanics: how passages get marked up, how codes relate to each other, how the analyst handles complications. The second is intercoder reliability: when a second analyst applies your codebook to the same passages, how much agreement should you expect, how do you measure it, and what does the resulting number mean?

3.1 Hierarchical Codes (Parent and Child)

A codebook is rarely flat. Codes nest under broader codes, which nest under categories, which sit under the codebook root. The hierarchy is what makes a large codebook navigable and what allows you to aggregate findings at different levels. In the loneliness corpus, a plausible partial hierarchy looks like this:

LONELINESS_EXPERIENCE/
   somatic-absence
   affective-loneliness
   social-invisibility
   temporal-fading
CAUSAL_ACCOUNTS/
   bereavement-onset
   migration-onset
   life-stage-transition
   structural-isolation
COPING_STRATEGIES/
   coping-pet
   coping-volunteer
   coping-phone-confidant
   coping-comfort-media
   coping-cooking-home-food
INTERPRETIVE_FRAMES/
   loneliness-as-cost-of-love
   loneliness-as-failure
   loneliness-as-rebuilding
   loneliness-as-invisible-to-society

The four parent categories at the top are the metacoding bins from Section 1 (Technique 12). Each child code can be applied independently to passages. When you report your findings, you can report at the category level (“coping strategies appeared in all 20 transcripts”) or at the code level (“coping-pet appeared in 6 of 20 transcripts”), depending on what your argument needs.

3.2 Multiple Codes Per Passage

A single passage may instantiate more than one code. This is normal and expected. Linda's chair passage simultaneously instantiates chair-absent-spouse (a specific code), somatic-absence (the absence as carried), and loneliness-as-cost-of-love (the interpretive frame she later articulates). All three codes are applied to overlapping or identical text spans. Taguette and the major QDA packages handle multi-coding natively.

The implication for analysis is that when you later count code occurrences, you are counting passage-code pairs, not unique passages. A transcript with 80 unique coded passages might have 130 code applications because many passages got two or three codes. Both numbers are meaningful; report whichever supports your argument and be clear about which you are reporting.

3.3 Axial Coding (Relationships Between Codes)

The term axial coding comes from Strauss and Corbin's (1990) grounded-theory tradition; for an exhaustive catalog of coding methods including in vivo and axial styles, see Saldaña (2021). It refers to a second pass over the data, after initial coding, in which the analyst attends to the relationships between codes rather than to the codes themselves. Axial-coding questions include: which codes co-occur? Which codes appear in sequence (and in which order)? Which codes seem to be causes of which others, in participants' own accounts? Which codes are mutually exclusive in practice?

Axial coding is what turns a flat codebook into a model. In the loneliness corpus, axial coding might reveal that bereavement-onset codes are nearly always followed in the same transcript by coping-pet codes; that migration-onset codes co-occur with cooking-home-food codes; that loneliness-as-cost-of-love appears only in transcripts that also contain somatic-absence. These relationships are not in the codes themselves; they are in the pattern of co-occurrence. Section 4 will show how to compute co-occurrence in R.

3.4 In Vivo Codes (Using Participants' Exact Words)

An in vivo code is a code whose name is taken verbatim from a participant's speech. The convention is to set in vivo codes in quotation marks in the codebook to mark their origin. In the loneliness corpus, defensible in vivo codes include: “wahda” (Amira's word for a refugee-specific loneliness), “witness-less hours” (Sarah's phrase for the loneliness of being unobserved), “the cost of love” (Linda's interpretation), “code-switching loneliness” (Marcus's articulation of the loneliness of moving between cultural registers), and “fading at the edges” (Helen's metaphor).

In vivo codes do two analytic things at once. First, they keep the participant's voice in the codebook, which protects against analyst over-abstraction. Second, they give the eventual paper memorable language — reviewers and readers remember “wahda” in a way they do not remember refugee-specific-loneliness. Most well-written qualitative findings sections have at least three or four section headers built from in vivo codes. We will use Amira's “wahda” as a section header in the worked findings exercise of Lesson 11.

3.5 Intercoder Reliability — Why It Matters

Once you have a codebook, the central question is: can someone else apply it the way you intended? Intercoder reliability is the operational answer to that question. Two (or more) analysts independently code the same passages using the same codebook; you compute the agreement; you decide whether the agreement is good enough.

Bernard, Wutich, and Ryan (Ch. 6) frame intercoder reliability as the most common operational standard for the third of the three methodological commitments (replicability, from Lesson 1). It is not the only test of replicability, but it is the one most often expected by quantitatively trained reviewers in public-health journals. Methodologically, it serves three purposes: it forces the codebook to be specific enough to be applied consistently; it reveals which codes are unclear and need refinement; and it gives you a defensible number to report in your methods section.

3.6 Percent Agreement (Simple but Flawed)

The most intuitive measure of agreement: count the passages on which two coders agree, divide by the total number of passages coded, multiply by 100. If coders agree on 85 of 100 passages, percent agreement is 85%.

The problem with percent agreement is that it does not adjust for the agreement you would expect by chance. If two coders are using a codebook with only two codes (apply / do not apply), and one of the codes is used 90% of the time, two random coders would agree about 82% of the time by accident. An 85% percent-agreement score in that setting reflects only a few percentage points of real agreement above chance. The chance-adjusted measures below are designed to fix this problem.

3.7 Cohen's Kappa (Two Coders, Nominal Categories)

Cohen's kappa (Cohen, 1960) is the chance-corrected agreement measure used most often in qualitative health research. The formula is:

κ = (p_o − p_e) / (1 − p_e)

Where p_o is the observed proportion of agreement (the percent agreement, expressed as a decimal) and p_e is the proportion of agreement expected by chance, given the marginal distributions of each coder's codings. Kappa ranges from −1 (perfect disagreement) through 0 (chance-level agreement) to +1 (perfect agreement).

Landis and Koch (1977) proposed interpretive thresholds for kappa that have become the field standard. They are guidance, not law — Bernard, Wutich, and Ryan are clear that the appropriate threshold depends on the stakes of the coding and the nature of the categories.

The convention in applied health research is that kappa ≥ 0.60 (substantial) is a defensible threshold for publication, and kappa ≥ 0.80 (almost perfect) is excellent. If your kappa is below 0.60 on a given code, the standard response is to refine the codebook entry for that code (usually by tightening the inclusion or exclusion criteria) and re-code the disputed passages.

3.8 Krippendorff's Alpha (the Preferred Measure)

Cohen's kappa has three limitations: it handles only two coders, it does not gracefully handle missing data, and it assumes the codes are nominal categories (mutually exclusive and unordered). For projects with more than two coders, with intermittent missingness, or with codes that are ordered (e.g., severity levels), Cohen's kappa is the wrong tool.

Krippendorff's alpha (Krippendorff, 2018) generalizes the kappa idea to handle all three limitations. It accommodates any number of coders, any pattern of missing codings, and any level of measurement (nominal, ordinal, interval, ratio). It is the measure most contemporary methodologists recommend, including Bernard, Wutich, and Ryan.

The formula is more complex than kappa's — alpha is built on the difference between observed and expected disagreement, computed from a coincidence matrix — but you will rarely compute it by hand. The R package irr contains kripp.alpha(), which we will use in Section 4. Interpretively, alpha values follow a similar convention to kappa: ≥ 0.80 is acceptable for tentative conclusions, ≥ 0.667 (two-thirds) is the often-cited minimum for any kind of claim. Below 0.667, the codebook is not yet reliable and needs revision.

3.9 When Intercoder Reliability Is the Wrong Measure

Not all qualitative work asks for intercoder reliability. Bernard, Wutich, and Ryan are explicit about this and so are many methodologists writing in the interpretivist tradition. In two situations, the reliability framing is actually misleading.

The first is interpretivist or constructivist work. In Charmaz-style constructivist grounded theory (Charmaz, 2014), the analyst's interpretation is understood to be partly constitutive of what the data mean. Two competent analysts may legitimately reach different but defensible interpretations of the same passages. The standard of evaluation is not identity (the kappa standard) but coherence — whether each interpretation is internally consistent, evidence-based, and methodologically transparent. For this kind of work, the equivalent quality check is investigator triangulation (multiple analysts compare interpretations and document where they differ and why) rather than a kappa score.

The second is narrative and discourse-analytic work (Lessons 9 and 10). Narrative analysis attends to the structure of a single telling; discourse analysis attends to the rhetorical work an utterance performs. Neither tradition typically treats codings as nominal categories applied independently to passages, so a kappa is not the right object. The methods sections of credible papers in these traditions usually report on member-checking, on theoretical sensitivity, and on the writing trail rather than on kappa.

For your capstone — which adopts Bernard, Wutich, and Ryan's pragmatist-positivist stance and uses inductive-deductive hybrid coding — intercoder reliability is the right measure. We will compute it in Section 4. But you should be aware that there are defensible qualitative methodologies for which it is the wrong measure, so that you do not later read an interpretivist paper and mistake the absence of a kappa for a methodological failure.

3.10 The QDA Software Landscape

Before we move into the workflow in Section 4, a quick orientation to the software landscape. There are four commercial options and a growing free-and-open-source ecosystem. None of them does anything you cannot do by hand on a small corpus; what they buy you is speed, consistency, and the ability to scale.

We chose Taguette for HSCI 841 because it is free (every student in the world can use it), it is open-source (your project is not held hostage by a licence), and its export format is standard (CSV and HTML, which any analysis tool can read). The features it lacks — advanced visualizations, sophisticated network views — you will build in R, which is also free and open-source. The combination is a complete, transferable workflow.

Introduction and Overview

The first three sections of this lesson laid out the conceptual apparatus: themes versus codes, the twelve theme-finding techniques, inductive versus deductive coding, codebook architecture, coding mechanics, intercoder reliability. This section turns operational. You will see, end to end, how to find candidate themes in the loneliness corpus using R, how to build a codebook for them, how to apply the codebook in Taguette, how to export the coded extracts back into R for analysis, and how to compute Krippendorff's alpha on a small two-coder reliability check. The section ends with the Week 5 capstone milestone.

4.1 Step 1 — Import the Corpus into R

The loneliness transcripts live in ../term projects/HSCI_841/transcripts/ as plain-text files (P01_Maya.txt through P20_Frank.txt). Each transcript opens with a metadata header (participant ID, age, gender, occupation, etc.) followed by the interview proper. We will read all 20 into a single R object using the readtext package.

# Load the stack you installed in Lesson 1
library(tidyverse)
library(quanteda)
library(readtext)

# Point at the transcripts folder
transcript_dir <- "term projects/HSCI_841/transcripts"

# Read all 20 transcripts at once into a readtext data frame
loneliness_rt <- readtext(
  file.path(transcript_dir, "P*.txt"),
  docvarsfrom = "filenames",
  docvarnames = c("participant_id", "pseudonym"),
  dvsep = "_"
)

# Convert to a quanteda corpus (this is the object the rest of the workflow uses)
loneliness_corpus <- corpus(loneliness_rt)

# Sanity checks
summary(loneliness_corpus, n = 5)  # first five transcripts: tokens, types, sentences
ndoc(loneliness_corpus)         # number of documents — should be 20
docvars(loneliness_corpus)      # participant_id and pseudonym columns

4.2 Step 2 — Repetitions and KWIC as Theme-Finding Aids

Theme-finding technique 1 (Repetitions) and technique 10 (Word lists and KWIC) are computationally tractable. Once your transcripts are in a quanteda corpus, you can compute word frequencies in seconds and pull keyword-in-context concordances for any word that catches your eye. The result is not the end of theme-finding; it is the front edge of it. The patterns the computer surfaces are then read closely by you, in their original transcripts, to decide whether they support a theme.

# Tokenize: split each transcript into words, drop punctuation, lowercase
loneliness_tokens <- tokens(
  loneliness_corpus,
  remove_punct = TRUE,
  remove_numbers = TRUE
) |> tokens_tolower()

# Remove stopwords (the, and, of, etc.) — what's left is content words
loneliness_tokens <- tokens_remove(loneliness_tokens, stopwords("en"))

# Build a document-feature matrix and compute global word frequencies
loneliness_dfm <- dfm(loneliness_tokens)
freq <- textstat_frequency(loneliness_dfm, n = 40)
print(freq)

# KWIC: look at every occurrence of "chair" with 5 words on either side
chair_kwic <- kwic(loneliness_tokens, pattern = "chair*", window = 5)
print(chair_kwic)

# Repeat for words that emerged as candidate themes in Section 1
kwic(loneliness_tokens, pattern = "tired", window = 5)
kwic(loneliness_tokens, pattern = "hollow", window = 5)
kwic(loneliness_tokens, pattern = "fading", window = 5)
kwic(loneliness_tokens, pattern = c("because", "since"), window = 6)  # Technique 6: linguistic connectors

4.3 Step 3 — Set Up the Taguette Project

R surfaces patterns; Taguette is where you mark passages with codes. The two work together: you use R to find what to look for, and Taguette to record where you found it. The workflow below assumes you set up Taguette in Lesson 1; if not, return to Section 4.5 of Lesson 1 first.

4.4 Step 4 — Re-import Coded Extracts into R for Analysis

Taguette's CSV export gives you a long-format data frame: one row per (passage, code) pair. Multi-coded passages appear in multiple rows. This is the format you want for almost any quantitative analysis of your qualitative coding — code frequencies, co-occurrence, code-by-participant matrices, comparison across subgroups.

# Read the Taguette export
codings <- read_csv("term projects/HSCI_841/taguette_export_week5.csv")

# What the export contains (columns vary slightly by Taguette version)
glimpse(codings)
# Typical columns: tag (= code), content (= passage), document (= transcript filename)

# Code frequencies: how many times each code was applied across the corpus
code_freq <- codings |>
  count(tag, sort = TRUE)
print(code_freq)

# Code frequencies by participant: which codes appeared in which transcripts
code_by_participant <- codings |>
  count(document, tag) |>
  pivot_wider(names_from = tag, values_from = n, values_fill = 0)
print(code_by_participant)

# Simple co-occurrence: how often each pair of codes appears in the same transcript
# (transcript-level co-occurrence, not passage-level — passage-level is also computable)
co_occurrence <- codings |>
  distinct(document, tag) |>
  inner_join(distinct(codings, document, tag), by = "document") |>
  filter(tag.x < tag.y) |>
  count(tag.x, tag.y, sort = TRUE)
print(head(co_occurrence, 20))

4.5 Step 5 — Compute Krippendorff's Alpha for a Two-Coder Reliability Check

For the intercoder reliability portion of your Week 5 work, you and a peer partner will independently code one shared transcript using the same provisional codebook. You then compute Krippendorff's alpha (or Cohen's kappa) on the resulting codings. The R code below assumes you have produced a wide-format matrix where rows are passages and columns are coders, with the cell value being the code each coder assigned.

library(irr)

# Simulated example: 12 passages, 2 coders, codes encoded as integers
# In practice you will build this matrix from your own and your partner's Taguette exports
# Rows = passages; Columns = coders
codings_matrix <- matrix(c(
  # Coder A    Coder B
       1,         1,   # Passage 1: both said somatic-absence
       1,         1,   # Passage 2: both said somatic-absence
       2,         2,   # Passage 3: both said coping-pet
       1,         3,   # Passage 4: A said somatic-absence, B said affective-loneliness  ← disagree
       3,         3,   # Passage 5: both said affective-loneliness
       4,         4,   # Passage 6: both said loneliness-as-cost-of-love
       2,         2,   # Passage 7
       1,         1,   # Passage 8
       3,         1,   # Passage 9: disagree
       2,         2,   # Passage 10
       4,         4,   # Passage 11
       1,         1    # Passage 12
), nrow = 12, byrow = TRUE)

# Krippendorff's alpha (nominal codes)
kripp.alpha(t(codings_matrix), method = "nominal")

# Compare to Cohen's kappa (two coders, nominal)
kappa2(codings_matrix, weight = "unweighted")

# And simple percent agreement (for reference — note how it overstates agreement)
agree(codings_matrix)

4.6 The Iterative Cycle

The five steps above describe one pass through the workflow. In practice you will iterate. After computing reliability on a first transcript, you will revise the codebook for codes that scored poorly, re-code the disputed passages, and only then move on to the remaining transcripts. The audit trail (Section 2.6) records each iteration. By the end of Week 12, your capstone will have gone through three or four iterations of this cycle, each documented, each producing a better codebook than the one before.

4.7 The Week 5 Capstone Milestone

The Week 5 milestone is the first piece of analytic work in the capstone arc that produces a real deliverable from the loneliness data. The positionality memo (Week 1), the research question (Week 2), the sampling rationale (Week 3), and the data-collection critique (Week 4) were design pieces. From Week 5 onward, you are doing analysis on transcripts.

Bringing It All Together

Lesson 5 is the heaviest analytic lesson in the first half of HSCI 841. It is the lesson that converts the conceptual apparatus of the first four lessons (definitions, design, sampling, collection) into operational coding moves. The twelve theme-finding techniques (Section 1), the inductive/deductive/hybrid distinction (Section 2), the seven-element codebook entry (Section 2), the four coding mechanics (Section 3), and the intercoder reliability statistics (Section 3) are tools you will use, in some combination, in every qualitative project for the rest of your career.

The Week 5 capstone milestone is the first piece of real analytic work on the loneliness dataset. The preliminary codebook you build this week will be revised in Lesson 6 (when we add a conceptual model), expanded in Lesson 7 (when we add constant-comparative analysis across subgroups), and stress-tested in Lessons 8–11 (when we run content, narrative, discourse, and analytic-induction passes on the same coded extracts). The codebook is the spine of your capstone paper, and Lesson 5 is where it is born.

The final reflection below asks you to take stock of one specific decision you will make in your own coding work. Concrete, specific, defensible — that is the standard the rest of the capstone will be evaluated against.

This glossary collects the key concepts, people, and methodological terms introduced in Lesson 5. Use it as a reference while you work through the material, or as a review before the final assessment. Type in the search box to filter entries.