This glossary collects the key concepts, people, and ideas you will meet in this lesson. Use it as a reference while you work through the material, or as a review before assessments. Type in the search box to filter entries.

Introduction and Overview

Lessons 1–10 covered how to design, conduct, analyse, and synthesize epidemiologic studies. Lesson 11 turns to validity — the property that determines whether all that work produces a defensible answer. The premise that well-conducted observational studies can deliver effect estimates comparable to randomised trials when bias is properly addressed is the long-standing claim of Concato, Shah, & Horwitz (2000); the equally enduring counter-claim that most published findings are false unless biases are explicitly quantified comes from Ioannidis (2005). The four content sections walk through the bias inventory you first met in HSCI 230, now consolidated for HSCI 341 as a systematic appraisal framework. Section 1 defines validity and introduces selection bias; Section 2 works through canonical examples and reduction strategies for selection bias; Section 3 covers information bias and misclassification; Section 4 closes with validation studies, measurement error, and how information bias affects required sample size. By the end, you should be able to read any observational study and decide where its validity is at risk.

11.1 Introduction to Validity

An awareness of the key features of study design, implementation, and analysis should help ensure we obtain valid results from research. The term validity relates to the absence of a systematic bias in results — a valid measure of association in the study group will have the same value as the true measure in the source population (except for variation due to sampling error).

To the extent that the study group and source population measures differ systematically, the result is said to be biased.

The Three Major Types of Bias

Click each card to learn more about the three major categories of bias that threaten the validity of observational studies:

11.2 Selection Bias

Selection bias results from the fact that the composition of the study group differs from that in the source population, and this biases the association observed between the exposure and the outcome of interest. Selection bias can affect study results significantly.

From a sampling and study-design perspective, each study will have an objective that relates to a defined target population. Ideally, the study group would completely reflect the source population, which in turn would reflect the target population. In practice, this is rarely the case.

Bias Variables and DAGs

Bias variables influence participation in a study in a way that causes the initial or final composition of the study group to differ from the source population, thus biasing the observed association.

The basic conditions for selection bias can be shown using directed acyclic graphs (DAGs):

In Scenario 1, both E and D independently affect selection. When we condition on selection (study only the responders), E and D become associated even though they are independent in the source population. In Scenario 2, a bias variable related to both exposure and disease directly affects selection, creating a spurious association. This unified causal-graph framing of selection bias as conditioning on a common effect was crystallised by Hernán, Hernández-Díaz, & Robins (2004); Greenland (2003) had earlier quantified how collider-stratification bias compares in magnitude with classical confounding, and Greenland, Pearl, & Robins (1999) provide the foundational DAG primer for epidemiologic use.

Sampling Fractions & Sampling Odds

We can understand selection bias by examining sampling fractions. The source population and study group follow the structure shown below:

The four sampling fractions (sf) represent the proportion selected from each cell:

If subjects were selected by random sampling, all four fractions would be equal — no selection bias. If the sampling fractions are equal, the OR of the sampling fractions (OR_sf) equals 1, and there is no bias in the observed OR.

Sampling Odds

In practice, sampling odds may be easier to conceptualise than individual sampling fractions. For a cohort study, we compare the sampling odds of disease among exposed versus non-exposed subjects:

If these selection odds are equal, there is no bias. If the ratio of sampling odds is greater than 1, bias is away from the null; if less than 1, bias is toward the null.

Introduction and Overview

Section 1 defined selection bias in the abstract. Section 2 is the case-study half of the topic. The examples below are the canonical ones — healthy worker effect (McMichael, 1976), Berkson's bias (Berkson, 1946/2014), and others — that you should be able to recognize in any observational study you read going forward. The reduction strategies that follow them are practical tools you can apply at the design stage.

11.3 Examples of Selection Bias

Selection bias can manifest in many ways across different study designs. Understanding these patterns helps researchers anticipate and prevent bias during the design phase.

11.3.1 Choice of Comparison Groups

A general principle is that study groups should be selected from the same source population. In cohort studies, it is important that the non-exposed group be comparable with respect to other risk factors for the outcome. In case-control studies, the control group should reflect either the prevalence of exposure in the ‘non-case’ members of the population from which the cases arose.

Types of Selection Bias

Click each card to explore different types of selection bias:

11.4 Reducing Selection Bias

Introduction and Overview

Sections 1 and 2 covered selection bias — errors arising from who ends up in the study. Section 3 turns to information bias: errors arising from how exposure or outcome is measured once participants are in. The misclassification framework that follows distinguishes nondifferential and differential errors and quantifies their predictable consequences for effect estimates.

11.5 Information Bias

The previous discussion was concerned with whether study subjects had the same exposure-disease association as that in the source population. Now we review the effects of incorrectly classifying or measuring the study subjects’ exposure, extraneous factors, and/or outcome status.

11.6 Bias from Misclassification

Misclassification bias results from a rearrangement of study individuals into incorrect categories because of errors in classifying exposure, outcome, or both.

11.6.1 Non-Differential Misclassification of Exposure

If misclassification of the exposure and outcome are independent (i.e., errors in classifying exposure are the same in diseased and non-diseased subjects, and vice versa), the misclassification is called non-differential.

With dichotomous exposures and outcomes, non-differential errors will bias measures of association toward the null (given Se_E + Sp_E > 1). The observed cell values are a mixture of correctly and incorrectly classified subjects:

11.6.2 Evaluating Non-Differential Exposure Misclassification

If the most likely values of Se_E and Sp_E are known, we can correct the observed classifications. Since b₁′ + b₀′ = b₁ + b₀ = m₀, we can solve for the true cell values:

11.6.3 Non-Differential Misclassification of Disease

11.6.5 Misclassification of Both Exposure and Disease

When both exposure and disease are misclassified, we need to pay close attention to reducing these errors whenever possible. Most researchers prefer to evaluate errors for the more important one first, conducting a “what if?” analysis one set of errors at a time.

11.6.6 Differential Misclassification

If the errors in exposure classification are related to the status of the outcome under study, the errors are called differential:

The resulting bias may be in any direction — either exaggerating or underestimating the true association. In case-control studies, recall bias is one common illustration: ‘affected’ subjects (cases) may have increased sensitivity, and perhaps lower specificity, than non-affected subjects in recalling previous exposures.

11.6.7 Reducing Misclassification Errors

Introduction and Overview

Sections 2 and 3 named the bias types and described their predictable effects. Section 4 turns to the practical question of what an investigator can do about them. Validation sub-studies, formal measurement-error models, surrogate-measure adjustments, and sample-size adjustments for misclassification all give the working epidemiologist tools for converting awareness of bias into corrected estimates — the suite of methods now grouped under quantitative bias analysis, summarised in good-practice form by Lash, Fox, MacLehose, Greenland, Maclure, & Poole (2014) building on Greenland's (1996) basic methods for sensitivity analysis of biases and his multiple-bias modelling framework (Greenland, 2005).

11.7 Validation Studies to Correct Misclassification

A thorough review of validation studies to correct misclassification identified four main approaches: regression calibration, maximum likelihood, semi-parametric, and Bayesian methods. One key finding is that the more advanced methods are not user-friendly, while ‘simple’ approaches have important limitations.

11.8 Measurement Error

Errors in measuring quantitative factors can lead to biased measures of association. The bias can arise because the variable is not measured accurately (systematic bias) or due to a lack of precision (variability).

Regression Calibration Estimate (RCE)

To introduce the concepts of correcting measurement errors, suppose we have 2 quantitative exposure factors (X₁ and X₂) and a binary or continuous outcome (Y). The uncorrected ‘naive’ model is:

where the subscript ‘u’ indicates the coefficients are biased because the predictor variables (X′) are measured with error. The regression calibration estimate (RCE) involves:

11.9 Errors in Surrogate Measures of Exposure

Often, epidemiologists study the effects of a complex exposure using surrogate measures. For example, in air pollution studies, what is the ‘appropriate’ measure? It could be a complex mixture of agents, doses, and durations.

11.10 Impact of Information Bias on Sample Size

Classification and measurement errors can have a serious impact on measures of association. With non-differential misclassification, measures are biased toward the null; with classical measurement error models, the same is true for continuous variables. This leads to an important conclusion:

Summary: Types of Information Bias

# PART A -- E-values for unmeasured confounding
library(EValue)

# Study reports OR = 2.4 (95% CI 1.6, 3.5); treat as rare-disease RR
evalues.OR(est = 2.4, lo = 1.6, hi = 3.5, rare = TRUE)

# Same idea for a hazard ratio
evalues.HR(est = 1.8, lo = 1.3, hi = 2.5, rare = FALSE)

# PART B -- Greenland's simple bias adjustment for misclassification
correct_OR <- function(OR_obs, Se = 0.95, Sp = 0.95, P_E_unexp = 0.30) {
  numer <- OR_obs * (Sp + Se - 1)
  denom <- (Sp + Se - 1) - (1 - Sp)*OR_obs - (1 - Se)
  numer / denom
}
correct_OR(OR_obs = 2.4, Se = 0.85, Sp = 0.95)

Bringing It All Together

Lesson 11 turned the analytic machinery of HSCI 341 toward the question of whether the answer it produces is actually trustworthy. The arc moved from a definition of validity, through selection bias and its canonical patterns, into information bias and the predictable directionality of nondifferential and differential misclassification, and finally to the practical tools (validation sub-studies, regression calibration, sample-size adjustment) that convert awareness of bias into corrected estimates. When you read or report an observational study, three appraisal frameworks codify this material: the STROBE reporting guideline (von Elm et al., 2007), the ROBINS-I risk-of-bias tool (Sterne et al., 2016), and the GRADE system for rating quality of evidence (Guyatt et al., 2008).

The final assessment below asks you to integrate across all four sections: distinguishing the three major bias types, recognizing canonical examples in described studies, predicting the direction of bias under given misclassification structures, and identifying the right correction strategy. Each question maps to a problem you will encounter when you appraise a published study or design your own.

What you take away here sets up Lesson 12's confounding-and-causal-inference content directly: with selection and information bias named and bounded, confounding is the third leg of the bias triangle, and the methods of Lesson 12 (matching, stratification, multivariable adjustment, propensity scores — Rosenbaum & Rubin, 1983) are how you handle it. The unifying causal-inference framework that ties selection, information, and confounding bias together as conditional-independence questions on a causal graph is articulated by Rothman & Greenland (2005).

Lesson 11 Comprehensive Assessment

This final assessment covers all topics from Lesson 11: Validity in Observational Studies. You must score 100% to complete this lesson. Review the feedback for any incorrect answers before retrying.