Introduction to Observational Studies
Evaluating Epidemiological Research
Kiffer G. Card, PhD, Faculty of Health Sciences, Simon Fraser University
Learning objectives for this lesson:
- Differentiate between descriptive and explanatory studies
- Differentiate between experimental and observational studies
- Describe the three main elements of the unified approach to observational study design
- Describe the advantages and limitations of case reports, case-series reports, and surveys
- Design a cross-sectional study accounting for its strengths and weaknesses
- Identify circumstances where a cross-sectional study is appropriate
- List three approaches for obtaining incidence estimates from cross-sectional prevalence data
- Differentiate between repeated cross-sectional studies and following a cohort in a longitudinal study
- Apply the STROBE checklist to reporting a cross-sectional study
Dohoo, I. R., Martin, S. W., & Stryhn, H. (2012). Methods in Epidemiologic Research. VER Inc.
Study Classification & Design Framework
Descriptive vs. Explanatory Studies
Epidemiologic studies can be classified into two major categories: descriptive and explanatory (analytic). This classification reflects both the study’s objectives and its ability to support causal inference.
Descriptive studies include case reports, case-series reports, and surveys. They are designed solely to describe the nature and distribution of outcome events such as health-related phenomena. They describe the who, what, when, and where of disease occurrence.
Although a descriptive survey is not designed to assess hypotheses about manipulatable causes of the outcome event, the frequency of the outcome is usually described in categories of age, race, sex, season, and space.
Explanatory studies (also called analytic studies) are designed to make comparisons and contrasts between subgroups of study subjects based on exposure or outcome status. They allow the investigator to identify statistical associations between exposures and outcomes.
Explanatory studies can be further subdivided into experimental and observational studies, depending on whether the investigator controls the allocation of study subjects to exposure groups.
Experimental vs. Observational Studies
In experimental studies, the investigator controls (usually through randomisation) the allocation of study subjects to exposure groups. In contrast, in observational studies, the investigators try not to influence the natural course of events for the study subjects.
In experimental studies, we try to reduce variation from all sources through selection and control of the experimental setting. In observational studies, we embrace the presence of natural variation in order to identify important interactions among key variables and the exposure–disease association.
The price paid through the use of observational studies is that considerable efforts are required to prevent confounding (bias) of the exposure–disease association. Experiments are the preferred choice when the treatment is straightforward and easily manipulated, such as a vaccine or a specific therapeutic agent. The major advantage of the experimental approach is the ability to control potential confounders through the process of randomisation.
A Unified Approach to Study Design
Hernan (2005) stressed that when considering an observational study design, we should think about the design of a field experiment to accomplish the same objective. This approach, reinforced by Rubin (2007), emphasises that ‘design trumps analysis’ and that all elements of the study design should be completed before seeing any outcome data.
As a first step in considering an epidemiological study, a ‘thought experiment’ can be accomplished and should specify the key elements of study group, its selection, assignment to exposure, procedures for follow-up, and detecting the outcome. The important part is that formal randomisation would ensure ‘exchangeability’—the groups being compared are so similar that it does not matter which group was assigned to exposure.
All design features are completed before anyone has seen the outcome data. This includes subject exclusion, selection criteria, and control of confounding. Rubin formalises the process through propensity scores (the probability of exposure given the covariates) in the exposed and non-exposed groups. Unless these are virtually equal, some degree of confounding is possible.
After completing the initial design, we project forward to the presentation of study results under 3 different scenarios: (1) the exposure appears to increase risk; (2) the exposure appears to decrease risk; or (3) the exposure does not appear to be associated. For each scenario, we must defend the proposed design. This process helps identify potential weaknesses.
Hierarchy of Evidence for Causal Inference
From the perspective of drawing causal inferences, experimental studies are generally referred to as the gold standard. The hierarchy of causal evidence (from strongest to weakest) is typically:
| Study Type | Difficulty | Investigator Control | Causal Evidence | Relevance |
|---|---|---|---|---|
| Laboratory trial | Moderate | Very high | Very high | Low |
| Controlled field trial (RCT) | Moderate | High | Very high | High |
| Cohort study | Difficult | High | High | High |
| Case-control study | Moderate | Moderate | Moderate | High |
| Cross-sectional study | Moderate | Low | Low | Moderate |
| Survey | Moderate | Moderate | Not applicable | High |
| Case series | Easy | Very low | Not applicable | Low to high |
| Case report | Very easy | Very low | Not applicable | Low to high |
Reflection
Think about a health outcome you are interested in studying. Would an experimental or observational approach be more appropriate, and why? Consider ethical, practical, and scientific factors in your answer.
Section 1 Knowledge Check
1. Which of the following BEST distinguishes explanatory from descriptive studies?
2. What is the major advantage of experimental studies over observational studies for causal inference?
3. The ‘unified approach’ to observational study design includes the thought experiment, completing design features before seeing data, and:
Descriptive Studies: Case Reports, Case Series & Surveys
Descriptive studies are used to describe the main features of a disease or health-related outcome. Although they are not designed to evaluate associations between exposures and outcomes, the observations made in a descriptive study can form the basis of hypotheses which are then further investigated in analytic studies. Three forms of descriptive studies are case reports, case-series reports, and surveys.
Key Characteristics of Study Types
Descriptive studies differ from analytic observational studies in important ways. The following comparison highlights these differences:
A common feature of both case reports and case-series reports is the absence of a comparison group. Without a comparison group, it is impossible to draw valid conclusions about causal associations. This is why descriptive studies are considered hypothesis-generating rather than hypothesis-testing.
From Survey to Analytic Study
Kalsbeek and Heiss (2000), and Speybroeck et al (2003) have described the appropriate analysis of surveys bearing in mind the study design. If the survey is designed to collect information about both an outcome of interest and potential exposures (risk factors) beyond the categories of people, place, and time, it then becomes a cross-sectional analytic study and as such, can be used to evaluate associations between exposures and outcomes.
Leenen et al (2008) conducted a survey of the prevalence of hypertension in Ontario. The sampling frame consisted of municipalities and dissemination areas. From 6,436 eligible dwellings, contact was made with 4,559 potential participants. Hypertension prevalence was found to be 21.3% of the population overall. This survey combined both prevalence estimation and risk factor analysis, making it a cross-sectional analytic study.
Reflection
Can you think of a disease or health condition for which a case-series report might be the most appropriate initial study design? What hypothesis might it generate for future analytic studies?
Section 2 Knowledge Check
1. What is the primary limitation shared by both case reports and case-series reports?
2. A survey becomes a cross-sectional analytic study when it:
3. A case-series report documenting 50 patients with a rare autoimmune condition would be classified as:
Cross-Sectional Studies: Design & Implementation
Observational Studies Overview
Observational studies (a subgroup of analytic or explanatory studies) have an explicit formal contrast as part of their design: the prevalence of the outcome by exposure category groups is the central foundation. They differ from descriptive studies in that the comparison of two or more groups is central, and from experiments in that the researcher has no control over the allocation of study subjects to the exposure groups.
Prospective vs. Retrospective Designs
Observational studies can also be classified as prospective or retrospective. In prospective studies, the disease or outcome has not occurred at the time the study starts. In retrospective studies, both the exposure and the outcome have occurred when the study begins—hence cross-sectional studies are inherently retrospective in nature.
Sampling Drives the Design
The choices of observational analytic study design have traditionally been among 3 approaches based on how study subjects are selected:
- Cross-sectional study: A sample is obtained from the source population, and the prevalence of both disease and exposure is determined at the time of subject selection.
- Cohort study: A sample of study subjects from a source population with heterogeneous exposure levels is obtained, and the incidence of the outcome in the follow-up period is determined.
- Case-control study: Subjects with the outcome (cases) are identified and their exposure history is contrasted with the exposure history of a sample of non-case subjects (controls).
Cross-Sectional Study Design
The defining feature of a cross-sectional study is that it is an observational study whose outcome frequency measure is prevalence. The basis of the cross-sectional design is that a sample, or census, of subjects is obtained from the source population and the presence or absence of the outcome is ascertained at that point.
If the researcher wants to make inferences about the frequency of the outcome in a target population, then study subjects should be obtained by a formal random sampling procedure. The source population is the listing (real or implied) of potential study subjects from which the study group is obtained. The study group is that set of subjects who agree to take part in the study.
Exposure and other covariate status, such as demographic data, are obtained at the time of study subject selection or first contact/examination. Because the outcome measure is prevalence, it is sometimes difficult to know the appropriate time frame in which the exposure, if time-varying, might cause the outcome. Studying currently (prevalent) exposed subjects can also lead to bias when interpreting the impact of these exposures.
It is important to clearly define the outcome/disease of interest. In general, great care should be used if the outcome is a surrogate for a clinically important event. It is also important that widely accepted diagnostic criteria be used to identify the disease or outcome of interest.
The two main approaches used to prevent bias from factors associated with the outcome and whose distribution differs between exposure groups (confounders) are exclusion (restricted sampling) and analytic (statistical) control. Matching to prevent confounding cannot be applied in cross-sectional studies. Analytic control requires the use of a multivariable model.
Lanes et al (2011) conducted a cross-sectional study of postpartum depression (PPDS) among Canadian women. The survey used the Edinburgh Postnatal Depression Scale (EPDS) as the outcome measure. Potential risk factors included socioeconomic status, demographic factors, and maternal characteristics. Of 8,542 selected women, 6,421 responded. The national prevalence of minor/major and major PPDS was found to be 8.46% and 8.69% respectively. The mother’s stress level during pregnancy and prior depression had the strongest associations.
Reflection
In the postpartum depression study described above, the exposure ‘stress during pregnancy’ was measured retrospectively at the same time as the outcome. What challenges does this create for causal inference? How might you address these challenges?
Section 3 Knowledge Check
1. The defining feature of a cross-sectional study is that its outcome frequency measure is:
2. Cross-sectional studies are inherently:
3. Which approach to controlling confounding CANNOT be applied in cross-sectional studies?
Limitations, Incidence Estimation & Reporting
Inferential Limitations of Cross-Sectional Studies
By its nature, a cross-sectional study design measures prevalence, which is a function of both incidence and duration of the disease. Consequently, it is often difficult to disentangle factors associated with persistence of the outcome from factors associated with developing the outcome in the first instance (i.e., becoming a new incident case).
When the exposure factors are time-varying, it is often very difficult to differentiate cause and effect. For example, if one is studying the relationship between dog ownership and blood pressure, and the association is negative, one cannot differentiate between people that obtained a dog because they had low blood pressure from those whose lifestyle changed, consequently lowering their blood pressure after obtaining a dog. The more changeable the exposure, the worse this issue becomes.
Cross-sectional studies are best suited for time-invariant exposures such as race or sex. In these instances, the investigator can be certain that the exposure preceded, or at least was not caused by, the outcome.
Estimating Incidence from Cross-Sectional Studies
Although cross-sectional studies directly measure prevalence, there are approaches for estimating incidence from prevalence data. This is often desirable because incidence data are more useful for causal inference.
A simple way to obtain population-level incidence data is to perform two cross-sectional studies, one before and one after an event of interest. For example, Miller et al (2010) performed two cross-sectional studies before and after the 2009 H1N1 epidemic in England, giving a population-based estimate of incidence.
Other approaches include using two different tests—one that detects early immune response and one that detects long-lasting immunity. People who test negatively to the less sensitive test are followed forward for a defined time period to ascertain how many become positive. This approach has been refined for HIV studies.
Rajan and Sokal (2011) describe how to estimate age-specific incidence from prevalence data. Their general approach uses two prevalence estimates at different time points. The incidence rate at year ‘a’ is:
where ‘n’ is the time between the two prevalence estimates (Pa and Pa+n) in the cross-sectional survey.
Repeated Cross-Sectional vs. Cohort Studies
Sometimes it is desirable to follow a population over time. Two options exist: repeated cross-sectional samplings of the population, or a longitudinal study of the initial study subjects (a cohort approach). Each has distinct advantages:
Reporting Observational Studies: The STROBE Statement
In 2004, a network of methodologists, researchers, and journal editors established what we now know as the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement. It provides a checklist of 22 items considered essential for good reporting of observational studies.
The STROBE checklist covers: Title & Abstract (indicate study design), Introduction (background, objectives, hypotheses), Methods (study design, setting, participants, variables, data sources, bias, sample size, statistical methods), Results (participants, descriptive data, outcome data, main results), and Discussion (key results, limitations, interpretation, generalisability).
Reflection
Consider a cross-sectional study that finds an association between pet ownership and lower blood pressure. Explain why this finding cannot be interpreted as causal evidence that pet ownership lowers blood pressure. What study design would be more appropriate?
Section 4 Knowledge Check
1. The primary reason cross-sectional studies have limited ability to support causal inference is:
2. Cross-sectional studies are best suited for studying exposures that are:
3. The STROBE statement provides:
Lesson 8 — Final Review & Assessment
Final Reflection
Reflect on the full range of study designs discussed in this lesson. If you were asked to investigate the relationship between a novel environmental exposure and a chronic health outcome, what type of study would you begin with and why? How might your study design evolve as evidence accumulates?
Lesson 8 Comprehensive Assessment
This assessment covers all sections of Lesson 8. You must answer all 15 questions correctly to complete the lesson. Review the feedback after each attempt.
1. Which study type is designed to make comparisons between subgroups based on exposure or outcome status?
2. The key difference between experimental and observational studies is:
3. The ‘thought experiment’ in the unified approach to study design involves:
4. Case reports are considered useful primarily because they:
5. A case-series report should document which of the following?
6. In the study classification hierarchy, which observational study design provides the strongest evidence for causal inference?
7. In a cross-sectional study, subjects are selected from the source population based on:
8. The natural measure of association in a cross-sectional study of a binary outcome is:
9. A major limitation of cross-sectional studies when studying time-varying exposures is:
10. Prevalence is a function of:
11. Repeated cross-sectional studies are preferred over cohort studies when:
12. Which of the following is NOT a component of the STROBE checklist?
13. Propensity scores in observational study design are used to:
14. A purposive non-random sample in a cross-sectional study primarily threatens:
15. A study that samples subjects from the general population, measures their current blood pressure and dietary habits simultaneously, and compares blood pressure between high-salt and low-salt diet groups is: