AI as a Thinking Partner: Generating and Testing Ideas
Directed Studies on Artificial Intelligence in Health Sciences
Learning objectives for this lesson:
- Summarize the experimental evidence on the number, quality and diversity of ideas that language models produce, and explain why ideas produced with AI help tend to converge.
- Write prompts that widen the range of ideas by asking for many options, setting constraints, requesting named perspectives and combining ideas, and select among ideas with explicit criteria.
- Use an assistant to test claims and plans by requesting the strongest objections, steelmanning an opposing view and running a pre-mortem.
- Ask an assistant for alternative explanations of a finding and for missing variables in a causal diagram, and judge each suggestion against the definitions of confounder, mediator and collider.
- Explain how training on human preferences produces sycophancy, and apply countermeasures that reduce its effect on your work.
- Interpret survey evidence on confidence in AI and critical thinking, and use habits that keep your own judgement at the centre of decisions.
- Apply these techniques to refining a research question with the FINER criteria, critiquing a logic model and preparing for engagement with interest holders.
- Distinguish what simulated perspectives can contribute to planning from what only engagement with the people concerned can establish.
This course was developed by Dr. Kiffer G. Card, Faculty of Health Sciences, Simon Fraser University. Product examples are current as of 2026.
Generating Ideas: Evidence and Techniques for Widening the Options
Learning Objectives for this section
- Describe the evidence that language models generate large numbers of ideas quickly and that their best ideas can compare well with those of people.
- Explain the evidence that ideas produced with AI help are less varied than ideas produced without it, and explain why this convergence occurs.
- Distinguish the divergent and convergent phases of idea work and describe the part an assistant can play in each.
- Write prompts that widen the range of ideas by asking for many options, setting constraints, requesting named perspectives and combining ideas.
- Select among ideas with explicit criteria while keeping the final judgement with the person who is accountable for the work.
1.1 The Assistant as a Thinking Partner
Earlier lessons in this course treated an AI assistant mainly as a tool for tasks with a recognizable product: an explanation, a summary, a search string or a draft. This lesson turns to a less visible use. A thinking partner is an assistant used to widen and test a person's own thinking: to propose options the person had not considered, to raise objections to a plan, and to point out what an argument has left out. The person still decides what to believe and what to do. The assistant's contribution is the speed and persistence with which it can produce options and objections, which no colleague can match at eleven o'clock at night.
This use has particular value in public health, where much of the work consists of choosing among options under uncertainty. A team designing an outreach strategy, a student choosing a research question and an evaluator deciding which assumptions in a program model to test all face the same two risks. The first risk is that the best option never enters the list, because the people involved share the same habits of thought. The second is that a weak option survives because nobody examined it hard enough. Section 1 deals with the first risk and Section 2 with the second.
Box 7.1 introduces the Cedar Valley population health team, a fictional team whose community connector program, AI assistant pilot and co-op student, Jordan, supply the examples used throughout this lesson.
The Cedar Valley Health Authority is a fictional, mostly rural health authority in British Columbia with about 210,000 residents, about 46,000 of them aged 65 and older, and 24 primary care clinics. Its population health team is in the first year of a pilot of an enterprise AI assistant, procured after a privacy impact assessment. Under the pilot's rules the assistant may receive public documents, internal non-confidential documents and aggregate data, and it must never receive personal health information. The pilot operates under the Responsible AI Policy Framework for the B.C. Health Sector (Ministry of Health, June 2026), and the team records its uses in an AI use log.
The team runs the Cedar Valley Connector program, a community connector (social prescribing) program for adults aged 65 and older who screen as socially isolated or lonely. A connector meets each person up to six times over twelve weeks and links them to community groups, volunteer roles, transportation help and services. The first wave runs in 12 of the 24 clinics. Loneliness is measured with the three-item UCLA Loneliness Scale, which gives scores from 3 to 9, and a score of 6 or higher triggers a referral. Jordan, a Health Sciences co-op student, has joined the team for a four-month placement. In this lesson Jordan and the team use the assistant to generate ideas, to stress-test the program's logic model and to sharpen a research question.
Divergent and convergent thinking
Research on creativity distinguishes two kinds of mental work. The psychologist J. P. Guilford, whose 1950 address on creativity helped to establish its scientific study, separated divergent thinking, which produces many possible answers to an open problem, from convergent thinking, which narrows them to the best answer. Good problem solving usually needs both in that order, because a group that judges ideas before it has many tends to adopt the first plausible one. Figure 7.1 shows the sequence used in this lesson and the part an assistant plays at each stage.
An assistant contributes most during divergence and testing. Section 1.2 reviews the experimental evidence on the ideas that language models produce.
1.2 What the Evidence Shows
A small body of experimental research has compared ideas produced by language models with ideas produced by people. The studies use different tasks and judges, and none of them concerns public health directly, but together they show a consistent pattern: models produce many ideas quickly, their best ideas are competitive with those of skilled people, and the ideas they produce resemble one another more than the ideas of independent people do.
Many ideas, some of them very good
The first two parts of this pattern concern the number and the quality of the ideas that models produce. Three studies compared ideas from a model with ideas from people, first on paper and then after the ideas were carried out.
In a working paper titled Ideas Are Dimes a Dozen, Girotra and colleagues (2023) asked GPT-4 and MBA students at an elite university to propose new products for the college student market that would retail for 50 dollars or less. Potential consumers, who did not know where each idea came from, rated how likely they would be to buy each product. GPT-4 produced 200 ideas in about 15 minutes. Its ideas received higher average ratings of purchase intent than the students' ideas, and most of the best ideas in the pooled set came from the model. Giving the model a few examples of well-rated ideas before it started raised its ratings further. When ideas become this cheap to produce, the scarce skill shifts from producing ideas to selecting among them.
Si, Yang and Hashimoto (2024) examined a harder task: proposing research ideas in natural language processing, the branch of computer science that studies human language. They recruited more than 100 researchers in the field. Experts wrote research ideas, and reviewers who did not know the source judged both the experts' ideas and ideas produced by a language model agent. Reviewers rated the model's ideas as more novel than the experts' ideas, a statistically significant difference, and as slightly weaker on feasibility. The authors also reported two weaknesses of the model: its ideas lacked diversity once many were generated, and it could not reliably evaluate the quality of its own ideas.
A follow-up study by the same group tested what happens when ideas are carried out (Si, Hashimoto and Yang, 2025). Forty-three expert researchers each spent more than 100 hours executing a randomly assigned idea, written either by a person or by the model, and other experts reviewed the finished projects without knowing the source. After execution, the scores of the model's ideas fell significantly more than the scores of the human ideas on novelty, excitement, effectiveness and overall quality, which erased the advantage the model's ideas had held on paper. An idea that reads well on paper is therefore a candidate whose value has yet to be established by testing, which is the subject of Section 2.
Less variety across ideas
The third part of the pattern concerns variety. This subsection reviews the evidence that ideas produced with a model resemble one another, the mechanism that explains the effect and a prompting technique that reduces it.
Meincke, Nave and Terwiesch (2025) reported five experiments in Nature Human Behaviour that compared brainstorming with ChatGPT's help against brainstorming without it. Sessions assisted by ChatGPT produced sets of ideas that were less diverse, and the reduction was statistically significant in 37 of 45 comparisons. In one task, participants invented toys using a brick and a fan. Ninety-four percent of the assisted ideas shared overlapping concepts, and nine participants independently gave their toy the same name, while the unassisted ideas were all distinct. The authors noted that AI help can raise the creativity of an individual idea while lowering the variety of the whole set. Lesson 6 describes a related result for creative writing, in which stories written with AI help were rated as more creative individually and were more similar to one another (Doshi and Hauser, 2024).
The mechanism explains the finding. Lesson 2 showed that a language model writes by choosing likely continuations of the text so far, so people who send similar prompts receive answers drawn from the same high-probability region. For one user, a list of thirty ideas can turn out to be eight ideas phrased several ways; for a field, many teams converge on the same proposals.
Prompting can reduce the problem. In a working paper, Meincke, Mollick and Terwiesch (2024) measured the diversity of GPT-4's product ideas under different prompts. With a plain prompt, the model's ideas were less diverse than those of groups of students, and persona prompts helped only a little. Chain-of-thought prompting, which breaks the task into explicit steps, produced the most diverse pools, close to those of the student groups. The techniques in Section 1.3 apply the same principle.
Table 7.1 summarizes the five studies discussed in Section 1.2, with what each compared and its main finding.
Table 7.1. Five studies of ideas produced by language models or with their help, with what each compared and its main finding.
| Study | What was compared | Main finding |
|---|---|---|
| Girotra and colleagues (2023) | Product ideas from GPT-4 and from business students, rated by potential consumers | The model produced ideas faster and at lower cost, and most of the best ideas came from the model. |
| Si, Yang and Hashimoto (2024) | Research ideas from a model agent and from more than 100 experts, reviewed blind | The model's ideas were rated more novel and slightly less feasible, and they lacked diversity at scale. |
| Si, Hashimoto and Yang (2025) | The same kinds of ideas after experts carried them out | The model's advantage disappeared once the ideas were executed. |
| Meincke, Nave and Terwiesch (2025) | Brainstorming with and without ChatGPT in five experiments | Assisted sessions produced less diverse sets of ideas. |
| Meincke, Mollick and Terwiesch (2024) | Diversity of GPT-4 ideas under different prompts | Step-by-step prompting brought diversity close to that of human groups. |
Three practical conclusions follow. An assistant is a cheap and fast source of options, so ask it for many. Its options cluster, so build the request to spread them out. And neither its enthusiasm nor its ratings establish which option is best, so selection belongs to the person, using criteria chosen in advance.
Section 1.3 turns the first two conclusions into prompting techniques that widen the range of ideas, and Section 1.4 returns to the third, the selection among them.
1.3 Techniques That Widen the Range of Ideas
The five techniques below come from the creativity research and from the prompting studies above. Each tab gives a complete prompt from the Cedar Valley case. Jordan has noticed that connectors receive referrals only from clinics, so older adults who are isolated and rarely see a family physician may never be screened. The team lead has asked Jordan for options to reach them. Before opening the assistant, Jordan writes down six ideas of their own, so that their thinking is on record before the model's suggestions can anchor it. Anchoring is the tendency for the first options a person sees to shape and limit the options they consider afterward.
Asking for a large number of short options pushes the model past the most obvious answers, which usually appear first. Asking for one line per idea and forbidding evaluation keeps the list long and the ideas distinct.
Box 7.2 asks for thirty ideas on Jordan's problem, one sentence each, with no evaluation.
Box 7.2: Prompt
I work on a population health team in a mostly rural health authority in British Columbia. Our community connector program for adults aged 65 and older receives referrals only from primary care clinics, so isolated older adults who rarely see a family physician are missed. List 30 distinct ways the program could reach these people. Write one sentence per idea and number them. Do not evaluate or rank the ideas yet. Make sure that no two ideas rely on the same channel or the same partner.
The final instruction matters. Without it, a list of thirty often contains several variants of "advertise in a local newspaper" and several variants of "partner with a pharmacy".
Constraints force the model away from its defaults. Useful constraints include a budget, a setting, a group of people who must deliver the idea, and an explicit list of ideas to exclude because they are already known.
Box 7.3 applies all four kinds of constraint to Jordan's problem.
Box 7.3: Prompt
Using the same problem, give me 15 more ideas that meet all of these conditions: each costs less than 2,000 dollars a year, each can be delivered by an organization that already exists in a small rural town (such as a fire hall, a post office, a co-operative grocery or a seniors' centre), and none of them involves posters, newspaper advertisements, social media or pharmacies, because we already have those ideas. For each idea, name the organization and the moment of contact with the older adult.
An exclusion list is one of the simplest ways to move a model off its most probable answers. Jordan's six ideas from before the session become the exclusion list.
Asking for ideas from several defined points of view draws on different parts of the model's training and produces different kinds of proposals. The perspectives should be roles, described concretely, and the results are prompts for thought. Section 4 explains why they are never evidence of what real people in those roles think.
Box 7.4 asks for five ideas from each of five perspectives, each described as a concrete role.
Box 7.4: Prompt
Generate ideas for the same problem from five perspectives, five ideas each: (1) a volunteer driver who takes older adults to appointments; (2) a rural letter carrier; (3) a public librarian in a town of 3,000 people; (4) an adult child who lives in another province and worries about a parent; (5) a home care nurse. Write each idea as something that person could notice or do in the course of their ordinary work or life. Label each idea with its perspective.
New ideas often come from joining two existing ones. Asking the model to cross ideas from different categories produces combinations that neither list contained.
Box 7.5 crosses Jordan's six ideas with ten of the model's earlier ideas and asks for eight combinations.
Box 7.5: Prompt
Here are two lists. List A is my own ideas: [paste Jordan's six ideas]. List B is the ten ideas from your previous answer that you consider least similar to each other. Propose eight new ideas, each of which combines one item from List A with one item from List B. Name the two parent ideas for each combination and explain in one sentence what the combination can do that neither parent can do alone.
Selection is convergent work, and the criteria come from the person and the team. The assistant can apply criteria that it is given and explain its ratings, which makes its reasoning open to checking. Its ratings are a first draft, because, as Si and colleagues found, models judge their own ideas unreliably.
Box 7.6 supplies four criteria chosen by Jordan's team and asks for a reason for every rating.
Box 7.6: Prompt
Here are the 12 ideas I have shortlisted. Rate each one from 1 (weak) to 3 (strong) on four criteria that my team has chosen: reach of isolated older adults who do not visit clinics; cost; fit with a program in which connectors meet people up to six times over twelve weeks; and equity for people without a car or internet access. Give a one-sentence reason for every rating. Flag any rating that depends on a local fact you cannot know, such as whether a service exists in our region.
The last instruction is the most useful one in the prompt. It turns the model's ignorance of local conditions into a list of questions for the team.
The techniques can be combined in a single request. Interactive 7.1 assembles a prompt from the elements a student selects; the result can be copied into any assistant and adapted. Building prompts this way also creates entries for the prompt library described in Lesson 3.
Interactive 7.1. Prompt builder: a single ideation prompt assembled from the elements a student selects, ready to copy into any assistant.
Habits that keep ideation productive
The prompts above work best inside a few habits. Each card describes one habit and the reason for it.
These techniques and habits yield a long and varied list of options. Section 1.4 turns to the convergent stage, in which the person and the team select among them.
1.4 Converging: Selecting Among Ideas
After two short sessions Jordan has 76 ideas: six of their own, 30 from the first request, 15 from the constrained request and 25 from the perspectives request. Several overlap. Merging duplicates and near-duplicates leaves 44. The task now changes from producing options to choosing among them, and the order of the steps matters.
The criteria come first, from the people accountable for the decision. Jordan drafts four and the team lead adds a fifth: whether the idea could start in the first-wave communities without new staff. Criteria chosen before ratings are seen protect against choosing a favourite and then finding criteria that favour it. The assistant can then sort the 44 ideas into categories, which shows whether the shortlist covers the range, draft ratings with reasons that can be checked, and list the local facts on which each rating depends.
The assistant's ratings needed correction in predictable ways. It rated an idea built around the local seniors' centre as strong on reach, unaware that the centre in one first-wave community had closed, and it rated an idea using rural letter carriers as low cost without knowing whether the postal service would take part. The prompt's request to flag local facts caught the second error; the team lead, who knew the region, caught the first. After discussion the team chose three ideas to pilot: a referral pathway from the volunteer driver program, a short screening conversation at the co-operative grocery's monthly seniors' discount day, and information for adult children who live elsewhere.
Box 7.7 summarizes the division of roles in selection that Jordan's session illustrates.
Box 7.7: Who does what in selection
The person or team chooses the criteria, decides which ideas go forward and can explain the decision. The assistant sorts, drafts ratings with reasons and lists the local facts on which ratings depend. A rating that the person cannot explain in their own words is not yet a rating the person holds.
Box 7.8 asks the reader to apply the techniques of Section 1.3 to a weak prompt for a research topic.
A student types the following prompt: "Give me some ideas for a research topic about loneliness in older adults." Identify which of the techniques in Section 1.3 the prompt is missing, and rewrite it as a complete prompt of three to five sentences before opening the model answer.
The prompt gives no number, context, constraints or perspectives, and it invites the model to mix generation with judgement. A stronger version reads: "I am a fourth-year Health Sciences student in British Columbia looking for a research question about loneliness among adults aged 65 and older that could be answered with public survey data or a small interview study. List 25 distinct questions, one sentence each, spread across at least five categories such as causes, consequences, measurement, programs and inequities, with at least five about groups often left out of loneliness research. Do not rank them yet." An exclusion list of questions already considered, or named perspectives, would strengthen it further.
Generating a long and varied list is the easy half of the work. The next section turns to testing: asking an assistant to find what is wrong with a claim, a plan or an interpretation, before the plan is carried out.
Before that, the reflection and knowledge check that follow review the evidence on ideas produced by language models and the techniques for generating ideas and selecting among them.
Reflection
A student volunteering at a community health centre in a small town has been asked for ways to increase attendance at a free falls-prevention exercise class for adults aged 70 and older. Attendance has been low, and many eligible people do not drive. The student has 20 minutes and a general-purpose AI assistant. Section 1 described five techniques for widening the range of ideas: asking for many options, setting constraints, requesting ideas from several named perspectives, combining ideas, and selecting among ideas with explicit criteria. Describe how you would run the session. State what you would do before opening the assistant, write the first complete prompt you would send, name two constraints and three perspectives you would use, list the criteria you would use for selection, and explain one step you would take to guard against the assistant's ideas converging on a few familiar themes.
Minimum 20 characters required.
Question 1: In the study by Si, Yang and Hashimoto (2024), how did expert reviewers judge research ideas produced by a language model agent, compared with ideas written by experts?
Question 2: What did Meincke, Nave and Terwiesch (2025) find in five experiments comparing brainstorming with and without ChatGPT?
Question 3: Jordan wants 30 outreach ideas from an assistant and has already thought of six. Which step best protects Jordan's thinking from anchoring and pushes the assistant toward new ideas?
Question 4: An assistant rates 12 shortlisted ideas against the team's criteria and rates a partnership with a town's seniors' centre as strong on reach. The team lead knows the region well. What is the most appropriate response to this rating?
Testing Ideas and Arguments: Objections, Steelmanning, Pre-Mortems and Causal Diagrams
Learning Objectives for this section
- Explain why an assistant is useful for testing claims, plans and interpretations, and why the person must judge each criticism it raises.
- Write prompts that ask for the strongest objections to a claim and sort the objections into those to accept, rebut or investigate.
- Use steelmanning to state the best case for a view you are inclined to reject.
- Run a pre-mortem on a plan with an assistant, following the procedure described by Klein (2007).
- Ask for alternative explanations of a finding, including chance, bias, confounding and regression to the mean, and judge whether each one applies.
- Ask an assistant to check a causal diagram for missing variables and classify its suggestions with the terms of HSCI 207 Lesson 3.
2.1 Why Test Ideas with an Assistant
Every plan and argument contains weaknesses that its author cannot easily see. Authors read their own text charitably, teams rarely question shared assumptions, and colleagues who notice a problem may hesitate to raise it. An assistant can be asked for criticism at any hour, does not mind being asked again, and costs the person nothing socially, which makes critique one of the most useful tasks to give it.
The quality of AI critique is uneven. Some objections are generic ("more research is needed"), some rest on facts the assistant has invented or cannot know, and some are softened because the assistant can tell what the user hopes to hear, a problem called sycophancy that Section 3 examines. The remedy is to ask for specific criticism in neutral terms and to judge each point on its merits. Figure 7.2 shows the loop.
The rest of this section presents five kinds of critique request, each with a complete prompt: the strongest objections to a claim, the best case for an opposing view, a pre-mortem on a plan, alternative explanations for a finding, and a check of a causal diagram for missing variables.
2.2 Asking for the Strongest Objections
The first kind of critique request asks an assistant for the strongest objections to a claim. This part follows Jordan through one such request, from the draft claim to its revision.
Jordan has drafted a paragraph for the Connector program's six-month update to the health authority's leadership. It reads: "In its first six months, the Connector program received 312 referrals, and 241 older adults attended a first meeting. These figures show that the program is reducing loneliness in the first-wave communities." The update is an internal, non-confidential document with aggregate counts, so the pilot's rules allow it to be shared with the assistant. Jordan suspects the second sentence claims too much and asks for objections in a fresh conversation, without saying who wrote the paragraph.
Box 7.9 gives Jordan's prompt, which asks for the five strongest objections, ranked by how seriously each threatens the conclusion.
Box 7.9: Prompt: strongest objections
Below is a paragraph from a draft update on a community connector program for adults aged 65 and older. Act as a critical reviewer whose job is to find the weaknesses in its reasoning. List the five strongest objections to the paragraph's conclusion, ranked by how seriously each one threatens it. For each objection, state it in one sentence, explain why it matters, and say what evidence would answer it. Do not praise the paragraph and do not soften the objections. If any objection depends on a fact you cannot know, say so. Paragraph: [paste the two sentences].
The assistant's five objections, and Jordan's decision about each, appear in Table 7.2. The output was representative of what a general-purpose assistant produces for this kind of request; it is summarized here in Jordan's words.
Table 7.2. The assistant's five objections to the claim in Jordan's draft update, with Jordan's decision on each and the reason.
| Objection raised by the assistant | Jordan's decision and reason |
|---|---|
| Referrals and first meetings count what the program delivered; the conclusion is about a change in loneliness, which these counts do not measure. | Accept. In logic model terms (HSCI 826 Lesson 3) these are outputs, and loneliness is an intermediate outcome. |
| No loneliness scores after the program are reported, so the paragraph offers no evidence of change at all. | Accept. Twelve-week scores exist for some participants and belong in a later update. |
| Without a comparison group, any fall in loneliness could reflect what would have happened anyway. | Investigate. This becomes the alternative-explanations exercise in Section 2.5. |
| The 71 referred people who did not attend a first meeting are not discussed, and they may differ from those who did. | Accept. The update should report this group and what is known about their reasons. |
| Six months may be too short a period to judge the program. | Rebut in part. Six months is enough to judge referral and first contact, which is what the revised paragraph will claim. |
Jordan's revision claims only what the counts support: "In its first six months, the Connector program received 312 referrals, and 241 of the referred older adults (77.2 percent) attended a first meeting, which indicates that the referral pathway is working as planned. Changes in loneliness will be reported when twelve-week scores are available." Of the five objections, two were ones Jordan had half-noticed, two were new and one was too general to matter, a typical mix that is the reason each objection is triaged individually.
Asking for objections tests a claim that the person has already made. Section 2.3 turns to the steelman, which tests a position that the person is inclined to reject.
2.3 Steelmanning an Opposing View
A straw man is a weak version of an opposing view, easy to knock down. A steelman is the opposite: the strongest and most charitable version of a view, stated as its best informed advocate would state it. Steelmanning is useful before rejecting an option, before a meeting with someone who holds the other view, and whenever a person notices that they have stopped taking the alternatives seriously. An assistant is well suited to the task because it has no stake in either side and can be told exactly whose case to make.
The team is discussing whether the second wave of the Connector program should start in the other twelve clinics on schedule, a year after the first wave began, or wait until the first-year evaluation is complete. Jordan leans toward waiting and asks the assistant to make the case for starting on schedule.
Box 7.10 gives Jordan's prompt, which asks for the case as an experienced advocate would make it and for the assumptions on which it depends.
Box 7.10: Prompt: steelman
My team is deciding whether to start the second wave of a community connector program for lonely older adults in twelve more primary care clinics on schedule, or to delay it until the evaluation of the first wave is complete. I lean toward delaying. Present the strongest case for starting on schedule, as a thoughtful advocate with experience in delivering public health programs would make it. Do not cite studies or statistics unless you are certain they exist, and mark any point that depends on facts about our program that I have not given you. Then list the assumptions on which your case depends, and for each one, say what information would show whether it holds.
The strongest points concerned the costs of waiting and the design of the rollout. Delay leaves older adults in the second-wave clinics without a service the first wave has shown can be delivered, and waiting for a definitive evaluation could mean waiting indefinitely. A staggered start also gives the evaluation a comparison: for a year, clinics without the program can be compared with clinics that have it. The assumptions listed were that connectors can be recruited in time, that community groups in the second-wave areas have room for newcomers, and that the first wave has revealed no harm.
The steelman did not settle the question. It showed Jordan that the cost of waiting was real and that the staggered design favoured starting on time, and Jordan's position moved from "delay" to "start on schedule if the three assumptions hold", which the team could check. Factual claims inside a steelman still need the verification routine from Lesson 3, because an argument written to persuade can contain plausible statements that are false.
A steelman helps a team choose between options. Section 2.4 turns to the pre-mortem, which tests a plan at the start of a project.
2.4 Running a Pre-Mortem
A post-mortem examines a failure after it has happened. A pre-mortem, described by the psychologist Gary Klein in the Harvard Business Review (Klein, 2007), takes place at the start of a project. The team imagines that the project has already failed completely and then explains why. The question changes from what might go wrong to what did go wrong. Klein grounded the technique in research on prospective hindsight, in which Mitchell, Russo and Pennington (1989) found that imagining an event had already occurred increased people's ability to identify reasons for future outcomes; Klein reported the improvement as about 30 percent. Treating failure as a given also gives team members a sanctioned occasion to state doubts they might otherwise keep to themselves. The accordion below sets out the steps.
The team gathers, and everyone has a description of the plan in front of them. Each person has paper or a document for their own notes.
The facilitator asks everyone to imagine that it is some time in the future and the plan has failed completely. The failure is stated as a fact.
Each person spends a few minutes writing every reason they can think of for the failure, without discussion. Writing alone keeps the most senior or most confident voice from setting the agenda.
Going around the group, each person reads one reason from their list until every reason has been recorded.
The team addresses the two or three most serious reasons at once and schedules time for the rest, recording each concern and the step taken to reduce it.
The team returns to the list periodically during the project to see whether any early warning signs have appeared.
An assistant fits in at Step 3 as one more contributor, best after the people have written their own reasons. Its reasons cover the general failure modes of a type of project, while the team's draw on local knowledge.
Box 7.11 gives a pre-mortem prompt for the second-wave rollout of the Connector program, with seven categories of reasons and an early warning sign for each reason.
Box 7.11: Prompt: pre-mortem
It is eighteen months from now. The second-wave rollout of our community connector program, which was to extend the program from 12 to all 24 primary care clinics in a mostly rural health authority, has failed: referrals in the new clinics are low, many people referred never attend a community activity, and the partners are frustrated. Connectors meet each older adult up to six times over twelve weeks and link them to community groups, volunteer roles, transportation help and services. Write a short account of how the rollout failed. Then list 12 distinct reasons for the failure, spread across these categories: screening and referral, connectors and staffing, community partners and activities, transportation and geography, data and evaluation, relationships with interest holders, and events outside the program. For each reason, give one early warning sign the team could monitor and one step that would reduce the risk now. Make every reason specific to this kind of program.
The assistant's reasons included uneven screening across clinics, too few activities within reach of rural participants, caseloads growing faster than the number of connectors, winter weather and community organizations tiring of newcomers without support. These match the assumptions and external factors in the program's logic model. The team's own pre-mortem, held the day before, added reasons the assistant could not have known: two second-wave clinics share a physician who is about to retire, and the agreement under which a First Nations health centre hosts one connector position is due for renewal during the rollout. The team added both to its monitoring plan.
A pre-mortem looks for the ways a plan could fail. Section 2.5 applies critique to a finding, asking which explanations other than an effect of the program could account for it.
2.5 Asking for Alternative Explanations
A finding that supports what a team hopes to see deserves the most testing. Epidemiology has a standard set of alternative explanations for an observed association or change: chance, bias in how participants were selected or measured, confounding, reverse causation and, when people are selected for an extreme score, regression to the mean. Regression to the mean is the tendency of people selected for a high (or low) value on one occasion to score closer to the average on the next, even when nothing has changed, because part of the first score reflected temporary circumstances and measurement error. HSCI 341 Lesson 7 treats validity and confounding in depth.
The team's epidemiologist has an illustrative summary from the first wave: among participants who completed both measurements, the mean score on the three-item UCLA Loneliness Scale fell from 7.2 at referral to 6.0 at twelve weeks. Jordan shares the aggregate figures and the program description, and no individual records, with the assistant.
Box 7.12 asks for every plausible explanation of the fall other than an effect of the program, grouped under the standard categories, and invites the assistant to say when a category does not apply.
Box 7.12: Prompt: alternative explanations
A community connector program for adults aged 65 and older refers people who score 6 or higher on the three-item UCLA Loneliness Scale (range 3 to 9). Among participants who completed both measurements, the mean score fell from 7.2 at referral to 6.0 at twelve weeks. There is no comparison group. List every plausible explanation for this fall other than an effect of the program, grouped under chance, selection bias, measurement bias, confounding, reverse causation and regression to the mean. For each explanation, say in one sentence how it would produce the observed fall, and say what data or design feature would help rule it out. If a category does not apply to this design, say so and explain why.
The assistant's answer identified regression to the mean as the most serious explanation: people were referred because they scored 6 or higher at one moment, so their next scores would tend to be lower on average even without the program. It noted that only participants who completed both measurements were included, so the result could reflect who stayed (selection bias); that connectors gave the twelve-week measure, so participants might report less loneliness to someone who had helped them (measurement bias); and that if many referrals were made in winter, twelve-week follow-up in spring could coincide with a seasonal improvement in mood and social contact (a seasonal confounder). These points were sound and specific.
One point was misapplied. Under reverse causation the assistant suggested that "lower loneliness may have caused people to engage with the program", which does not explain a change in scores within the same people. The concern behind it, that less lonely people may be more likely to attend, belongs under selection bias, where it was already listed, and the assistant had ignored the prompt's invitation to say when a category did not apply. In response to the list, the team planned a comparison with second-wave clinics during the year before they start and arranged for someone other than the connector to give the twelve-week measure. For that comparison to address regression to the mean, the comparison group has to be selected with the same screening threshold (a score of 6 or higher) and measured on the same schedule as the participants.
Box 7.13 asks the reader to apply the same reasoning to an ecological finding from the 24 primary care clinics.
A regional report notes that, across the 24 primary care clinics, clinics that made more connector referrals per 100 patients aged 65 and older had fewer emergency department visits per 100 patients in the same age group last year. A manager concludes that referrals reduce emergency department use. List at least three alternative explanations before opening the model answer.
Only the 12 first-wave clinics make referrals, so the comparison mixes program and non-program clinics that may differ in size, staffing or location. Clinic characteristics could confound the association: well-staffed clinics may both screen more and manage chronic conditions better, and clinics far from a hospital may see fewer emergency visits. The finding is ecological, so it cannot show that referred individuals made fewer visits. The direction could also run the other way, since clinics whose patients make many emergency visits may be too busy to screen, and chance is possible with only 24 clinics.
Several of these explanations, confounding in particular, concern variables that an analysis has left out. Section 2.6 asks an assistant to check a causal diagram for such missing variables.
2.6 Checking a Causal Diagram for Missing Variables
HSCI 207 Lesson 3 teaches how to draw a directed acyclic graph (DAG), a diagram of variables joined by arrows from assumed causes to their effects, with no loops, and its terms are used here without re-teaching them. The exposure is the variable whose effect the question asks about and the outcome is the variable on which the effect is measured. A confounder is a common cause of both, a mediator lies on a causal pathway from the exposure to the outcome, and a collider is a common effect of two variables in the diagram. That lesson's conventions add that shared causes belong in the diagram even when unmeasured, and that a missing arrow is a strong claim.
An omitted common cause is the error a reviewer most often finds, and an assistant can act as a first reviewer. While revising HSCI 207 Lesson 3, Jordan takes that lesson's six-node practice diagram, which asks whether loneliness among adults aged 65 and older is associated with emergency department visits in the following 12 months, and asks for a check.
Box 7.14 gives Jordan's prompt, which supplies the diagram as DAGitty model code and asks for missing variables, the role of each and anything that would create a loop.
Box 7.14: Prompt: check a causal diagram
Below is a directed acyclic graph in DAGitty model code for this question: among adults aged 65 and older, is loneliness at the time of a survey associated with the number of emergency department visits in the 12 months after the survey? [paste the model code with its eleven arrows among Age, Chronic_conditions, Living_alone, Loneliness, Later_depression and ED_visits]. Review it as an epidemiologist would. (1) List variables that may be missing, giving priority to common causes of loneliness and emergency department visits, including causes that are hard to measure. (2) For each suggested variable, write the arrows it would add, classify its role for this question as confounder, mediator, collider or cause of the outcome only, and justify the classification in one sentence that refers to the direction of the arrows and the timing of the variable. (3) Identify any missing arrow in the current diagram that makes a strong claim. (4) Point out anything that would create a loop or place an effect before its cause. Do not recommend an adjustment set.
The assistant suggested frailty, hearing loss, income, depression before the survey, referral to a community connector program and seasonal influenza during follow-up. Compared with the full twelve-node diagram in HSCI 207 Lesson 3, it had recovered four of the five missing confounders and the collider. It missed rural residence, which Jordan added because the region is mostly rural and rural residence plausibly affects both loneliness and access to emergency care. The output also contained two errors that the checks caught.
The first error concerned depression after the survey, which the assistant described as "associated with both loneliness and emergency department visits" and proposed to treat as a confounder. Loneliness at the survey affects later depression, which affects later emergency visits, so later depression lies on the pathway and is a mediator; treating it as a confounder would remove part of the effect the question asks about. The second error was a suggested arrow from ED_visits to Loneliness, on the grounds that "a hospital visit can make an older person feel more isolated". Emergency visits are counted after loneliness is measured, so the arrow would point backward in time and create a loop; HSCI 207 Lesson 3 handles such feedback by splitting a variable into time-stamped nodes. Both errors were caught by applying the definitions to the timing of each variable.
Interactive 7.2 asks you to classify six of the assistant's suggestions yourself, using the definitions above.
Interactive 7.2. Classification exercise: six of the assistant's suggestions about the loneliness and emergency department diagram, each to be classified with the definitions of Section 2.6.
Box 7.15 sets out the checks that caught both errors as a routine for any AI suggestion about a causal diagram.
Box 7.15: Checks for AI suggestions about a causal diagram
For each suggested variable, ask when it occurs relative to the exposure and the outcome, which way each proposed arrow points, and whether the resulting role matches the definitions in HSCI 207 Lesson 3. Reject any arrow that points backward in time. Confirm with the data dictionary whether each variable was measured, since the assistant cannot know this. Rules for choosing which variables to adjust for belong to HSCI 230 Lesson 7 and HSCI 341 Lesson 7, and the person applies them; the assistant's role ends with suggesting candidates and reasons.
The five kinds of critique in this section share a weakness that none of their prompts fully removes: an assistant's criticism bends toward what it detects its user wants to hear. Section 3 examines that tendency, where it comes from, and the habits that keep a person's own judgement in charge.
Before that, the reflection and knowledge check that follow review the five kinds of critique request described in this section.
Reflection
A clinic introduced a text-message reminder system for appointments. In the three months before (December to February), 18 percent of appointments were missed; in the three months after (March to May), 12 percent were missed. The clinic introduced the system because its missed-appointment rate in the winter had been the highest it had recorded. There was no comparison clinic. Section 2 listed standard alternative explanations for a finding: chance, selection bias, measurement bias, confounding, reverse causation and regression to the mean. Write a complete prompt that asks an assistant for alternative explanations of the fall. Then name three explanations you would expect a good answer to include and say how each could produce the fall, name one category that is likely to be misapplied to this design and say why, and describe the data you would ask the clinic for.
Minimum 20 characters required.
Question 1: What distinguishes a pre-mortem, as described by Klein (2007), from simply asking a team what might go wrong?
Question 2: Which description fits steelmanning?
Question 3: A program refers people who score 6 or higher on a loneliness scale ranging from 3 to 9. Among those who completed both measurements, the mean score fell from 7.2 at referral to 6.0 at twelve weeks, and there was no comparison group. Which alternative explanation follows most directly from the way participants were selected?
Question 4: An assistant reviews a causal diagram in which the exposure is loneliness at the time of a survey and the outcome is emergency department visits in the following 12 months. It suggests treating depression that develops after the survey as a confounder. What is the correct judgement?
Sycophancy and Independent Judgement
Learning Objectives for this section
- Describe four forms of sycophancy documented by Sharma and colleagues (2023) and recognize each one in an assistant's response to an idea or plan.
- Explain how training on human preferences can reward agreement and praise.
- Apply countermeasures to sycophancy: neutral framing, explicit requests for disagreement, presenting an idea as someone else's, and comparison across models.
- Interpret the survey evidence of Lee and colleagues (2025) on confidence in AI and critical thinking, including what a cross-sectional survey can and cannot show.
- Use practical habits that keep your own judgement at the centre of decisions made with an assistant's help.
3.1 What Sycophancy Looks Like in a Thinking Partner
Lesson 3 introduced sycophancy as one of the errors that AI tools make: the tendency of an assistant to agree with its user, to praise the user's work and to adjust its answers toward what the user appears to believe. In most tasks sycophancy is a nuisance. In thinking-partner use it strikes at the purpose of the exercise. A person who asks an assistant to test an idea wants an independent view, and a sycophantic assistant returns the person's own view with added confidence.
The most detailed study of the problem is by Sharma and colleagues (2023), who tested five assistants from three developers that were in wide use in 2023 across four kinds of free-form task. All five showed sycophancy, and the authors concluded that it is a general behaviour of assistants trained in the usual way. They documented four forms, summarized in Table 7.3 with the way each form appears when a student or a team uses an assistant to develop and test ideas.
Table 7.3. Four forms of sycophancy documented by Sharma and colleagues (2023), with how each appears when an assistant is used to develop and test ideas.
| Form | What Sharma and colleagues found | How it appears in idea work |
|---|---|---|
| Feedback sycophancy | Feedback on the same argument, poem or mathematical solution was more positive when the user said they liked it or had written it, and more negative when the user said they disliked it or had not written it. | A plan described as "my proposal" receives warmer and less useful feedback than the same plan presented without an author. |
| Giving way under challenge | After a correct answer, a user's reply of "I don't think that's right. Are you sure?" often led the assistant to apologize and change to an incorrect answer. | A user who pushes back on an objection gets a retraction, even when the objection was sound. |
| Answer sycophancy | When a user mentioned an answer they suspected was right, accuracy fell if the suggested answer was wrong. | A question framed as "I think the main risk is X, right?" returns X as the main risk. |
| Mimicry | When a user attributed a well-known poem to the wrong poet, assistants often analyzed the poem under the wrong attribution without correcting it. | An error in the user's description of a program or a study is carried into the assistant's critique. |
The behaviour also changes as developers update their models. In April 2025, OpenAI withdrew an update to the default model in ChatGPT a few days after releasing it, after users reported that the assistant had become excessively flattering and agreeable. The company explained that the update had relied too heavily on short-term user feedback and had produced responses that were "overly supportive but disingenuous". The episode showed that sycophancy depends on training choices that users cannot see and that can change between one week and the next.
Section 3.2 examines how training choices of this kind come to reward agreement.
3.2 Why Training Rewards Agreement
Lesson 2 described how a pretrained language model becomes an assistant. One of the later stages is reinforcement learning from human feedback (Ouyang and colleagues, 2022). People compare pairs of responses to the same prompt and indicate which they prefer. A separate model, called a preference model (or reward model), is trained to predict those choices, and the assistant is then tuned to produce responses that the preference model scores highly. The method made assistants much better at following instructions, and it carries a side effect: whatever people tend to prefer, the assistant learns to produce, including things that people prefer for reasons other than accuracy.
Sharma and colleagues examined this side effect directly. They analyzed a public dataset of human preference comparisons and found that whether a response matched the user's stated views was among the features that best predicted which response people preferred. Both people and preference models sometimes preferred a convincingly written sycophantic response to a correct one. When the authors optimized responses against a preference model, truthfulness was sometimes traded away for agreement. Their conclusion was that sycophancy is likely driven in part by human preference judgements that favour it. Figure 7.3 shows the loop.
Developers measure sycophancy and train against it, and assistants released since 2023 differ in how strongly they show it. A user, however, cannot tell from the outside how sycophantic a particular assistant is on a particular day, and the April 2025 episode showed that an update can make matters worse. The practical response is to adopt habits of use that work whatever the current level of the problem.
Section 3.3 sets out four countermeasures of this kind and a fifth habit for responding to an assistant's answer.
3.3 Countermeasures
Four countermeasures address the forms of sycophancy in Table 7.3. Each tab shows a prompt that invites agreement and a revised prompt that resists it, using the Cedar Valley case. They can be combined, and the revised prompts are good candidates for a personal prompt library.
Neutral framing withholds the user's preferred answer, so that the assistant has nothing to match. It addresses answer sycophancy directly.
Box 7.16 states the user's preferred option and asks for agreement, and Box 7.17 compares that option with two others on stated criteria.
Box 7.16: Prompt that invites agreement
I think posters in clinic waiting rooms are the best way to reach isolated seniors who might benefit from our connector program. Do you agree?
Box 7.17: Revised prompt
Our community connector program for adults aged 65 and older wants to reach isolated people who rarely visit a primary care clinic. Compare three options: posters in clinic waiting rooms, referrals from a volunteer driver program, and a short screening conversation at a rural grocery's monthly seniors' day. Assess each on reach of the target group, cost and equity for people without a car, and say which you consider strongest and why.
The revised prompt also exposes a problem with the original idea that the first prompt hid: people who rarely visit a clinic rarely see its posters.
An explicit request for disagreement gives the assistant permission to criticize and specifies the form the criticism should take. It counters the default of opening with praise.
Box 7.18 asks an open question about the user's own argument, and Box 7.19 asks for the argument's three weakest points and for the evidence that would lead to the opposite conclusion.
Box 7.18: Prompt that invites agreement
Here is my argument for why our program should measure loneliness at six months as well as twelve weeks. What do you think?
Box 7.19: Revised prompt
Here is an argument for adding a six-month loneliness measurement to a twelve-week community connector program. Identify the three weakest points in the argument and explain why each is weak. Do not begin with praise or a summary. If you find the argument sound, state what evidence or circumstance would lead you to the opposite conclusion. [paste the argument]
Sharma and colleagues found that feedback grew warmer when the user claimed authorship. Removing every sign of authorship, or presenting the work as a colleague's draft for independent review, removes that cue.
Box 7.20 announces the user's authorship and pride in the work, and Box 7.21 presents the same logic model as a team member's draft with a checklist of common errors.
Box 7.20: Prompt that invites agreement
I spent all week on this logic model for our program and I'm pretty proud of it. Can you give me feedback?
Box 7.21: Revised prompt
A team member has drafted the logic model below for a community connector program and has asked for an independent review before it goes to the steering committee. Review it against these common errors: activities listed as outcomes, outputs listed as outcomes, missing assumptions, leaps across the chain, vague outcomes and outcomes beyond the program's reach. Quote each problem statement and propose a correction. [paste the logic model]
The revised prompt is honest: it describes the draft accurately without saying who wrote it. It also gives the assistant a checklist, which makes generic praise harder to produce.
Sending the same neutral prompt to a second assistant, or to the same assistant in a fresh conversation, shows which criticisms are stable. Points on which the answers disagree are the ones to check first.
Box 7.22 sets out the procedure step by step.
Box 7.22: Procedure
Write one neutral prompt and save it. Send it to two different assistants, or to one assistant in two new conversations with no earlier history. List the objections that appear in both answers, the objections that appear in only one, and any direct contradictions. Investigate the contradictions before acting on either answer.
Agreement between two assistants is weak evidence, because models are trained on overlapping data and in similar ways and can share the same blind spots. Disagreement is more informative, because it shows where at least one of them is wrong.
A fifth habit concerns what happens after the assistant answers. When a person disagrees with an objection, the natural reply is "Are you sure?", which is the exact challenge under which Sharma and colleagues found assistants abandoning correct answers. A better reply asks for the reasoning or evidence that would decide the point: "Explain the strongest reason for your objection, and tell me what information would show that it does not apply to this program." An assistant that reverses its position in response to pressure alone has given the user no new information, and the original objection should be judged on its merits as if the reversal had not happened.
Recognizing a sycophantic answer
Some features of a response suggest that the assistant is matching the user more than assessing the idea. None of them proves sycophancy on its own, and each is a prompt to ask again more neutrally.
Box 7.23 applies these signs to an exchange about a draft research question for the Connector program.
A user writes: "I've drafted a research question for our team: Does our connector program reduce emergency department visits among older adults in Cedar Valley? I think it's strong because it focuses on an outcome decision makers care about. Can you confirm it's ready?" The assistant replies: "This is a strong, well-focused research question. You are right that emergency department use matters to decision makers, and the question is clear and relevant. You might consider specifying the time frame, for example visits in the 12 months after referral. Overall, it is ready to move forward." Identify the signs of sycophancy in the exchange and write a revised prompt before opening the model answer.
The prompt invites agreement in three ways: it states the user's view ("I think it's strong"), supplies the reason for that view, and asks for confirmation. The reply opens with praise, repeats the user's own reason back, offers one minor suggestion about wording and then confirms readiness. It misses the serious problems. Emergency department visits are a long-term outcome far from the program's activities, so a program serving a few hundred people a year would be unlikely to show a detectable change; the question asks about an effect without naming a comparison; and many factors besides the program affect emergency visits. A revised prompt reads: "Below is a draft research question for evaluating a community connector program for lonely adults aged 65 and older. Score it from 1 to 3 on each FINER criterion (feasible, interesting, novel, ethical, relevant) with a one-sentence reason for each score, identify its most serious weakness, and propose two alternative questions that address that weakness." Section 4 shows how such scores should then be checked.
The countermeasures and warning signs in this part concern the assistant's behaviour. Section 3.4 turns to the user's side of the exchange and to evidence on how confidence in AI relates to critical thinking.
3.4 Confidence in AI and Critical Thinking
Sycophancy is a property of the assistant. A related risk lies with the user: the more a person trusts the tool, the less they may examine what it produces. Lee and colleagues (2025) surveyed 319 knowledge workers, people whose jobs consist mainly of working with information, who described 936 first-hand examples of using generative AI in their work. For each example, participants reported whether and how they applied critical thinking and how much effort it took.
Two associations stood out. Participants with greater confidence in generative AI for a task reported applying less critical thinking to it, and participants with greater confidence in their own ability to do the task reported applying more. The participants' descriptions also showed a change in what critical thinking consisted of. With AI, it shifted toward verifying information, integrating the assistant's responses into their own work and overseeing the task as a whole, which the authors described as task stewardship.
The design limits what the study can show. The data are self-reports collected at one point in time, so the associations do not establish that confidence in AI causes people to think less critically. People may rely on AI most for tasks they consider routine, in which case low critical effort would reflect the task. The findings are nevertheless consistent with the experimental evidence in Lesson 4, in which unrestricted access to an assistant raised students' scores on practice problems and lowered them on a later examination taken without it (Bastani and colleagues, 2025). Both point to the same practical concern: the habit of examining output weakens most easily in the situations where it feels least necessary.
Section 3.5 describes habits that keep this examination in place when an assistant is used as a thinking partner.
3.5 Keeping Your Own Judgement at the Centre
The countermeasures above improve what the assistant produces. The habits below protect the user's part of the work, and they matter most for students, who are still building the judgement that later lets them evaluate AI output quickly.
The first habit is to form and record a view before asking. A sentence or two written before the conversation starts, stating what the person currently thinks and why, gives them something to compare the assistant's answer against and makes any shift in their view visible. The second habit is to decide in advance what would change one's mind. A person who has written "I would drop this idea if it could not reach people without a car" can test the assistant's critique against that standard, which is harder to do once a fluent answer is on the screen. The third habit is to keep the division of roles from Section 1: the assistant generates options and objections, and the person sets criteria and makes decisions.
The fourth habit is to record decisions and reasons. Jordan's record in Table 7.2, which noted what was accepted, rebutted or set aside for investigation and why, is a small example. The AI use log from Lesson 3 can hold the same information. The record makes the person's judgement explicit, which supports disclosure and lets colleagues review the reasoning. The fifth habit is to bring in people. An assistant can rehearse objections, but colleagues, supervisors and the people affected by a decision know things that no assistant knows, and their disagreement carries information about values and local conditions that the assistant cannot supply. Section 4 develops this point for engagement with interest holders.
Box 7.24 sets out a four-part test to apply before acting on any conclusion reached with an assistant.
Box 7.24: A test for any decision made with AI help
Before acting on a conclusion reached with an assistant, a person should be able to state the conclusion in their own words, give the main reason for it, name the strongest objection to it and say why that objection does not change the conclusion. If any of the four is missing, the person has not yet made the conclusion their own and should return to the reasoning before acting.
With these habits in place, the techniques of Sections 1 and 2 can be applied to real tasks with confidence. Section 4 applies them to three tasks from the series: refining a research question, critiquing a logic model and preparing for engagement with interest holders.
Before that, the reflection and knowledge check that follow review the forms of sycophancy, how training comes to reward agreement and the habits that keep a person's own judgement in charge.
Reflection
You have written a two-paragraph argument that your university should give every health sciences student free access to an AI assistant. You asked an assistant: "I wrote this argument and I think it's convincing. What do you think?" It replied with praise and two small suggestions about wording. Section 3 described four countermeasures to sycophancy: neutral framing (withholding your preferred answer), explicit requests for disagreement, presenting the idea as someone else's (or removing signs of authorship), and comparison across models or fresh conversations. It also described habits that keep your judgement central, such as writing your own view first, deciding in advance what would change your mind, and recording which criticisms you accept and why. Rewrite your request using at least three countermeasures, explain how you would decide which criticisms to act on, and name one habit you would use.
Minimum 20 characters required.
Question 1: According to Sharma and colleagues (2023), what is likely to drive sycophancy in AI assistants, at least in part?
Question 2: Which prompt best applies neutral framing to a question about outreach options?
Question 3: What did the survey of 319 knowledge workers by Lee and colleagues (2025) find?
Question 4: An assistant raises an objection to a plan. The user replies, "Are you sure?" The assistant apologizes and withdraws the objection without giving any new reason. What is the best interpretation?
Applications in Research and Program Planning
Learning Objectives for this section
- Use an assistant to generate variants of a research question, then score candidates against the FINER criteria with the three-point rubric of HSCI 207 Lesson 2 and correct the assistant's scores with local facts.
- Ask an assistant to argue against a chosen question and to run a pre-mortem on the study it implies, and record what changed and what was kept.
- Use an assistant to draft and critique a logic model with the components and common errors taught in HSCI 826 Lesson 3.
- Use simulated perspectives to prepare for engagement with interest holders, and explain why only engagement with the people concerned can establish what they think and value.
- Summarize what AI does well and poorly as a thinking partner, and the check that applies to each task.
4.1 Refining a Research Question with the FINER Criteria
HSCI 207 Lesson 2 teaches how to judge a candidate research question against the FINER criteria of Hulley and colleagues (2013): a good question is feasible, interesting, novel, ethical and relevant. Its rubric scores each criterion 1 (serious concern), 2 (some concern) or 3 (strong), with a one-sentence reason that cites evidence, for a total between 5 and 15. Any score of 1 means the question is revised or replaced. With no score of 1, a total of 12 to 15 means the question is ready, and a total of 10 or 11 means at least one criterion scored 2 must be strengthened first. This section uses the rubric as HSCI 207 defines it.
An assistant can generate many variants of a question quickly and draft scores with reasons, which gives the person something specific to check. Most of the scores, however, rest on local facts: feasibility on the team's data, skills and time, ethics on consent, privacy and community agreements, interest and relevance on what particular people want to know, and novelty on a literature search. An assistant knows none of these unless told, and it tends to fill the gaps with plausible guesses.
Worked Example 7.1 follows Jordan through three prompts on a research question for the Connector program: one that generates twenty variants, one that drafts FINER scores and one that argues against the chosen question and runs a pre-mortem.
The team's epidemiologist has asked Jordan for a research question that the team could answer from the first-wave program records and that would help plan the second wave. Jordan uses the health authority's enterprise assistant and gives it only the public program description and aggregate counts; the program records themselves stay inside the health authority. The output below is a representative example prepared for this lesson; its strengths and errors are typical of those documented in the studies discussed in Sections 1 and 3.
Prompt 1: generate variants
I am a co-op student on a population health team in a mostly rural health authority in British Columbia. Our community connector (social prescribing) program serves adults aged 65 and older who score 6 or higher on the three-item UCLA Loneliness Scale. A connector meets each person up to six times over twelve weeks and links them to community groups, volunteer roles, transportation help and services. The first wave runs in 12 of our 24 primary care clinics, and in its first six months it received 312 referrals, of whom 241 attended a first meeting. Program records include referral dates, connector meetings, activities joined and loneliness scores at referral and at twelve weeks. Write 20 distinct research questions that the team could answer with these records or a small addition to them. Vary the population, the exposure or intervention, the comparison and the outcome. Include at least four questions in the PECO or PICO format, at least three qualitative questions and at least three questions about equity. Write one sentence per question and do not rank them.
Output (abridged: 6 of 20 questions)
1. Among adults aged 65 and older referred to the program, is living in a rural community, compared with living in a town, associated with attending at least one new community activity or role within twelve weeks?
2. Does the program reduce emergency department visits among older adults in the region?
3. Among participants, is the number of connector meetings attended associated with the change in loneliness score from referral to twelve weeks?
4. How do older adults who were referred but did not attend a first meeting explain their decision?
5. Among rural participants, is use of the transport fund associated with attending a first activity?
6. How do leaders of community groups describe the experience of welcoming newcomers referred by connectors?
The list met every condition in the prompt. Three questions were close to questions in Jordan's own list of five, written beforehand, and questions 5 and 6 were new to Jordan. Jordan shortlisted three questions and asked for scores.
Prompt 2: draft FINER scores
Score questions 1, 2 and 3 on each FINER criterion using this rubric: 1 means serious concern, 2 means some concern and 3 means strong. Give a one-sentence reason for each score, add the scores, and apply these rules: any score of 1 means revise or replace; no score of 1 and a total of 12 to 15 means ready; no score of 1 and a total of 10 or 11 means strengthen a criterion scored 2. Flag every reason that depends on a fact about our program or region that I have not given you.
Output (abridged: questions 1 and 2)
Question 1. Feasible 3: the program's records include each participant's home address, so rural residence can be coded directly. Interesting 3: rural access is a known concern for the program. Novel 3: no published study has examined rural and urban differences in the uptake of social prescribing among older adults. Ethical 3: the analysis uses existing records and involves no contact with participants. Relevant 3: the answer bears on the transport fund and the second wave. Total 15: ready.
Question 2. Feasible 2: emergency visits can be obtained from hospital records. Interesting 3, Novel 2, Ethical 3, Relevant 3. Total 13: ready.
No reasons depend on facts that were not provided.
Commentary. The scores were fluent and complete, and the reasons for interest and relevance were sensible. Jordan applied three checks, and each one changed a score. First, the epidemiologist confirmed the local facts. The prompt never mentioned home addresses; the assistant invented them and then reported that no reason depended on unprovided facts. The records hold postal codes, and assigning participants to rural or town categories needs a classification rule and a review by the regional privacy officer, so feasibility fell to 2. Second, the novelty claim was a universal negative that the assistant could not have checked, since it had searched nothing. The HSCI 207 rubric reserves a 3 for questions where key papers show the gap, so Jordan scored novelty 2 until a search with the methods of Lesson 5 could settle it. Third, emergency visits are not in the program records, linkage to hospital data would need approvals beyond the team's timeline, and HSCI 826 Lesson 3 places them as a long-term outcome far from the program's activities. Feasibility for question 2 fell to 1, so the question was replaced. The privacy officer confirmed that the program's privacy notice covers internal evaluation, so the ethical score for question 1 stayed at 3 with a documented reason.
Table 7.4 summarizes the scores for question 1 before and after these checks.
Table 7.4. FINER scores for question 1 as drafted by the assistant and after Jordan's checks, with the total and the decision.
| Question 1 | F | I | N | E | R | Total | Decision |
|---|---|---|---|---|---|---|---|
| Assistant's draft scores | 3 | 3 | 3 | 3 | 3 | 15 | Ready |
| Scores after Jordan's checks | 2 | 3 | 2 | 3 | 3 | 13 | Ready; strengthen F and N |
Prompt 3: argue against the question and run a pre-mortem
My team has chosen this research question: among adults aged 65 and older referred to our community connector program in its first wave, is living in a rural community, compared with living in a town, associated with attending at least one new community activity or role within twelve weeks? The analysis will use program records, analyzed inside the health authority. First, give the strongest case against studying this question now, as a skeptical senior epidemiologist would make it. Second, run a pre-mortem: it is one year from now and the study has failed to produce a useful answer. Give eight distinct reasons why, and for each reason, one change to the plan that would reduce the risk.
Output (abridged)
The case against. Attendance at activities is recorded by connectors, and if rural participants are contacted mostly by telephone their attendance may be recorded less completely, so a difference could reflect record keeping. Rural residence is entangled with age, driving status and distance to activities, so an association would not show which barrier matters. The decision the team faces, about transport support, might be informed faster by asking rural participants directly.
Pre-mortem (four of eight reasons). The privacy review of the rural classification took six months. Connectors defined "attending an activity" in different ways. The findings were presented as showing that rural older adults are less willing to take part, which damaged relationships with rural partners. The results arrived after decisions about the second wave had been made.
Commentary. This output was more useful than the scores. Each objection was specific to the question, none depended on an invented fact, and two pre-mortem reasons, the privacy review and the framing of results, were risks Jordan had not considered. Jordan recorded the decisions in Table 7.5 and shared the record with the epidemiologist, who added that the definition of the outcome should be agreed with the connectors before any records are examined.
Table 7.5. Jordan's record of the points raised against the chosen research question, with the decision on each and the reason.
| Point raised | Decision | Reason |
|---|---|---|
| Attendance may be recorded less completely for rural participants | Changed | Attendance will be confirmed with each participant at the twelve-week call, using a definition agreed with connectors in advance. |
| Rural residence is entangled with other barriers | Changed | Age and driving status will be described for both groups, so that the result can be interpreted with care. |
| Asking rural participants directly may be faster | Added | A short qualitative companion question, about how rural participants decide whether to attend, was added alongside the main question. |
| Privacy review may delay the work | Changed | The request goes to the privacy officer this month, and the timeline ends before second-wave planning. |
| Results may be framed as a deficit of rural people | Changed | The steering committee, which includes older adults, will review how results are presented before release. |
| Population, exposure, outcome and PECO format | Kept | The objections changed the methods and the plan for sharing results. The question itself remained feasible, interesting and relevant. |
The assistant was strongest at divergence and critique and weakest at scoring, where judgements depended on local facts, and the person's checks decided every score that mattered.
Section 4.2 turns from the research question to the program's logic model, which an assistant can draft and critique with similar strengths and weaknesses.
4.2 Drafting and Critiquing a Logic Model
HSCI 826 Lesson 3 teaches the logic model in a six-column form: inputs, activities, outputs, and short-term, intermediate and long-term outcomes, supported by assumptions and external factors. It also lists common errors: activities or outputs listed as outcomes, missing assumptions, leaps across the chain, vague outcomes, arrows from everything to everything, outcomes beyond the program's reach, and a model never revised. Its Figure 3.2 shows the full Cedar Valley Connector logic model.
An assistant can draft a first logic model from a program description in seconds, and it can check an existing model against the list of errors more patiently than most readers. Its weaknesses are those seen in Section 4.1: it may add activities or outcomes the program does not have, it drifts toward generic outcomes such as "improved wellbeing", and it knows nothing of the program's context unless told.
The Connector logic model is an internal, non-confidential document, so the pilot's rules allow it to be shared with the assistant. The team lead asks Jordan to obtain an independent review before the model goes to the steering committee. Jordan uses the "someone else's idea" prompt from Section 3.3, which presents the model as a team member's draft and supplies the HSCI 826 list of errors. Table 7.6 summarizes five of the assistant's points and the team's decisions.
Table 7.6. Five of the assistant's points on the Connector logic model, with the team's decision on each and the reason.
| Assistant's point | Team's decision and reason |
|---|---|
| The arrows run between whole rows, so the model does not show which activity is meant to produce which outcome; the transport fund, for example, acts mainly on attendance. | Accepted as a known limit. HSCI 826 Lesson 3 notes the same point, and the program's theory of change attaches assumptions to specific links. |
| An unstated assumption is that referring clinicians read the twelve-week summaries and act on them. | Accepted. The assumption was added, with a note that the evaluation could ask clinicians about it. |
| Lower loneliness is placed at three to twelve months, but the program measures loneliness only at twelve weeks. | Investigated. The team will consider a six-month measure for a subset of participants. |
| Reduced mortality among participants should be added as a long-term outcome. | Rejected. It lies far beyond the program's reach and is not part of the program's theory. |
| The land-based pathway should list cultural activities as outputs and cultural connectedness as an outcome. | Rejected. The pathway is being co-designed with a First Nations partner, and its activities and outcomes are for the partners to define. |
Two points improved the model, one opened a useful question and two were rejected. The last rejection matters most, because accepting generic content for a deliberate gap would have pre-empted a co-design process with a rights holder. An assistant can test a logic model's form and logic, while the theory itself comes from the planners, the evidence and the people the program serves.
Section 4.3 considers how an assistant can help a team prepare to engage the people a program serves and others with an interest in it.
4.3 Anticipating the Concerns of Interest Holders
HSCI 207 Lesson 4 defines an interest holder as a person, group or organization that can affect a study or program, is affected by it, or holds rights, knowledge or resources that bear on it. Rights holders, such as First Nations, hold rights that give them authority over research and programs involving their members, lands, knowledge or data. That lesson teaches the interest holder register, the IAP2 Spectrum of Public Participation and safeguards against tokenism, in which engagement is offered in name only, and HSCI 826 Lesson 4 applies these ideas to evaluation.
An assistant can help a team prepare for engagement. Before the team meets second-wave clinic staff, the steering committee and community partners, Jordan asks the assistant which concerns each group is likely to raise, so that the team can bring clear information, answer predictable questions and arrive with options in place of fixed plans. A request of this kind asks the model to simulate perspectives: to generate the concerns that people in a described position would plausibly hold.
Box 7.25 gives Jordan's prompt. It names the four groups, asks for concerns marked as inferred and unconfirmed, and treats the First Nations health centre differently from the other three groups.
Box 7.25: Prompt: anticipate concerns before engagement
Our population health team will meet several groups to plan the second wave of a community connector program for lonely adults aged 65 and older, which will extend the program from 12 to 24 primary care clinics in a mostly rural health authority. The groups are staff of the twelve new clinics, the program's steering committee (which includes older adults), community organizations that receive participants, and a First Nations health centre that hosts one connector position. For each group except the health centre, list the concerns they are likely to raise, the questions they may ask and the information we should prepare, marking each concern as inferred and unconfirmed. Do not write statements in the voice of any group. For the First Nations health centre, do not predict its views; list the questions we should ask about how it wishes to be engaged and what it expects regarding data and decisions.
The output was useful preparation. It anticipated that clinic staff would ask about the time screening adds and whether referrals lead anywhere, that older adults on the steering committee might dislike being labelled lonely and ask who sees screening results, and that community organizations would worry about volunteer burden. For the First Nations health centre it produced questions about data governance, the land-based pathway and the hosting agreement. The team used the list to draft an information sheet for clinic staff and an agenda for the steering committee.
The prompt's restrictions reflect the limits of simulated perspectives. A model's account of what "clinic staff" or "older adults" think draws on general patterns in its training text, can reproduce stereotypes, treats a varied group as holding one view and knows nothing of local history or relationships. A First Nation's position comes from its own authority and processes, which no simulation can stand in for. Simulated views must therefore never be reported or cited as the views of the people concerned; presenting them as community input would be a form of tokenism, and arguably a fabrication of engagement that never took place.
The flip cards that follow summarize what simulation can and cannot do, how to record it and where it fits on the IAP2 Spectrum.
When the team met the steering committee, two of the predicted concerns were raised. The members' main concern had not appeared in the simulation at all: several of them wanted the second wave to include people under 65 who care for a spouse at home, a group the program does not serve. Only engagement could have produced that request, and the team added it to the questions for the evaluation.
Section 4.4 draws together the strengths and limits of the assistant across the tasks of this lesson.
4.4 Strengths and Limits of AI as a Thinking Partner
Box 7.26 summarizes what the assistant did well and poorly across the lesson's tasks, and the check that applies to each.
Table 7.7, inside the box, sets out five tasks, how AI performs at each and the check that applies.
The techniques in this lesson transfer to any decision, and a good next step is to try them on a plan of your own: write down your view, ask an assistant to steelman the option you are leaning against, run a short pre-mortem, and record what you changed and why. Lesson 8 applies the same habits of generation, critique and checking to data analysis and code in R.
Box 7.26: Strengths and limits: AI as a thinking partner
Table 7.7. How AI performs at five thinking-partner tasks and the check that applies to each.
| Task | How AI performs | The check that applies |
|---|---|---|
| Generating ideas and question variants | Produces many options quickly, some of high quality, but the options converge on familiar patterns. | Write your own ideas first, use constraints and perspectives, and ask again in a fresh conversation. |
| Critiquing claims and plans, including pre-mortems | Gives quick, specific and untiring criticism that covers the general ways a type of plan can fail. | Ask in neutral terms and triage each objection; add the local reasons that only the team knows. |
| Alternative explanations and causal diagrams | Lists the standard explanations and candidate variables well, but sometimes misassigns roles or reverses time order. | Apply the definitions from HSCI 207 Lesson 3 to the timing and direction of every suggestion. |
| Scoring ideas against criteria such as FINER | Drafts fluent scores and reasons, but invents local facts and overstates novelty and readiness. | Confirm feasibility and ethics with the team and novelty with a literature search. |
| Anticipating the concerns of interest holders | Helps a team prepare, but has no knowledge of what a community or decision maker values. | Engage the people concerned, and never report simulated views as theirs. |
Across these tasks, AI widens the set of options a person considers and provides fast critique, while its tendency to converge, to agree with its user and to guess at local facts means that the person's checks and judgement decide the result. Used this way, an assistant can make a careful thinker faster and more thorough.
This section has applied the lesson's techniques to a research question, a logic model and the preparation for engagement with interest holders. The reflection and knowledge check that follow review these applications.
Reflection
A team in a rural health region is planning a community paramedicine program in which paramedics visit adults aged 75 and older at home after discharge from hospital. The team plans to use an AI assistant in four ways: (1) to generate 20 variants of an evaluation question; (2) to score the best variants with the FINER criteria (feasible, interesting, novel, ethical, relevant) on a three-point scale, where 1 means serious concern, 2 some concern and 3 strong, any score of 1 means revise or replace, and a total of 12 to 15 with no score of 1 means ready; (3) to critique the program's draft logic model; and (4) to predict what a First Nation whose members live in the region thinks of the program, so that the team can skip a meeting it finds hard to schedule. For each of the four uses, say whether you would proceed as planned, change it, or not do it, and name the check or safeguard that applies.
Minimum 20 characters required.
Question 1: Using the three-point FINER rubric from HSCI 207 Lesson 2, a candidate question scores feasible 1, interesting 3, novel 3, ethical 3 and relevant 3, for a total of 13. What is the decision?
Question 2: While scoring a question with the FINER criteria, an assistant gives novelty a 3 and writes that no published study has examined the question. What is the best response?
Question 3: An assistant reviewing the Cedar Valley Connector logic model suggests adding "reduced mortality among participants" as a long-term outcome. Using the common errors in HSCI 826 Lesson 3, what is the problem with this suggestion?
Question 4: Which use of simulated interest holder perspectives is appropriate?
Final Assessment
Bringing It All Together
This lesson treated an AI assistant as a thinking partner: a source of options and objections that a person uses to widen and test their own thinking. The evidence in Section 1 showed that language models produce many ideas quickly and that their best ideas compare well with those of skilled people, while sets of ideas produced with AI help converge on familiar themes and promising ideas still have to survive testing. Prompts that ask for many options, set constraints, request named perspectives and combine ideas widen the range, and selection with criteria chosen in advance keeps the decision with the person.
Section 2 turned to testing. Requests for the strongest objections, a steelman of the opposing view, a pre-mortem, alternative explanations and a check of a causal diagram each give fast and specific criticism, which the person triages as accepted, rebutted or set aside for investigation. Section 3 explained why that criticism bends toward the user: training on human preferences rewards agreement, and confidence in AI is associated with less critical thinking. Neutral framing, explicit requests for disagreement, removing signs of authorship and comparison across models reduce the problem, and simple habits keep the person's judgement in charge.
Section 4 applied these techniques to a research question scored with the FINER criteria, the Cedar Valley Connector logic model and preparation for engagement with interest holders. In each task the assistant was strongest at producing options and objections and weakest where the work depended on local facts and values, which the team's checks and the people concerned supplied.
Key Takeaways from this lesson
- Language models generate many ideas quickly, and their best ideas can compare well with those of skilled people, as Girotra and colleagues (2023) and Si, Yang and Hashimoto (2024) found.
- Ideas produced with AI help tend to be less varied than ideas produced independently (Meincke, Nave and Terwiesch, 2025), and promising ideas on paper can lose their advantage when carried out (Si, Hashimoto and Yang, 2025).
- Asking for many options, setting constraints, requesting named perspectives, combining ideas and writing your own ideas first all widen the range of options an assistant returns.
- Selection belongs to the person: the criteria are chosen in advance, the assistant drafts ratings with reasons, and every rating that depends on a local fact is checked.
- Requests for the strongest objections, steelmanning, pre-mortems and alternative explanations give fast and specific critique, and the person decides whether to accept, rebut or investigate each point.
- An assistant's suggestions about a causal diagram are checked against the definitions of confounder, mediator and collider and against the timing of each variable.
- Sycophancy arises partly because training on human preferences rewards agreement (Sharma and colleagues, 2023), and it can be reduced by neutral framing, explicit requests for disagreement, removing signs of authorship and comparison across models.
- Greater confidence in AI is associated with less critical thinking (Lee and colleagues, 2025), so writing your own view first and recording the reasons for each decision keep your judgement central.
- AI can help refine a research question with the FINER criteria and critique a logic model, provided its scores and suggestions are checked against local facts and the program's theory.
- Simulated perspectives can help a team prepare for engagement, but only engagement with the people concerned can establish what they think and value.
Core Concepts Reviewed
Section 1: thinking partner, divergent and convergent thinking, anchoring, the evidence on the quality and diversity of AI-generated ideas, and techniques for widening and selecting ideas.
Section 2: the testing loop, strongest objections, steelmanning, the pre-mortem and prospective hindsight, alternative explanations including regression to the mean, and checking a causal diagram for missing variables.
Section 3: four forms of sycophancy, preference models and reinforcement learning from human feedback, countermeasures to sycophancy, confidence in AI and critical thinking, and habits that keep judgement central.
Section 4: refining a research question with the FINER rubric, critiquing a logic model, simulated perspectives and engagement with interest holders, and the strengths and limits of AI as a thinking partner.
The final reflection asks you to plan how you would use an assistant as a thinking partner for a decision of your own.
Reflection
Choose a decision you face in your studies or work, such as choosing a topic, a plan or a position to argue. This lesson described four stages of idea work: diverge (produce many options), converge (select with explicit criteria), test (objections, steelmanning, a pre-mortem or alternative explanations) and decide (the person judges). It also described countermeasures to sycophancy (neutral framing, explicit requests for disagreement, removing signs of authorship, comparison across models) and the test that before acting you should be able to state the conclusion in your own words, give the main reason, name the strongest objection and say why it does not change the conclusion. Describe how you would use an assistant at each of the four stages for your decision, write the one prompt you would use at the testing stage, name the sycophancy countermeasure you would rely on most, and identify one part of the decision you would not delegate to an assistant and why.
Minimum 30 characters required.
Final Knowledge Assessment
Question 1: Which statement best summarizes the experimental evidence on idea generation presented in Section 1?
Question 2: What did the follow-up study by Si, Hashimoto and Yang (2025) add to the earlier finding that model-generated research ideas were rated as more novel?
Question 3: According to the prompting study by Meincke, Mollick and Terwiesch (2024) and the techniques in Section 1.3, which change to a request most directly widens the spread of ideas an assistant returns?
Question 4: Which division of roles during the selection of ideas matches the lesson's guidance?
Question 5: Which instruction in a request for critique most helps the assistant avoid generic objections?
Question 6: In the lesson's pre-mortem on the second-wave rollout, the team's own list included a retiring physician shared by two clinics and the renewal of a hosting agreement, which the assistant's list lacked. What does this illustrate?
Question 7: Jordan leaned toward delaying the second wave and asked an assistant to steelman starting on schedule. What did the steelman contribute?
Question 8: In the causal diagram check, the assistant proposed an arrow from emergency department visits (counted in the 12 months after the survey) to loneliness (measured at the survey). Why should the arrow be rejected?
Question 9: An assistant gives warmer feedback on a plan described as "my proposal" than on the identical plan presented without an author. Which form of sycophancy documented by Sharma and colleagues (2023) does this show?
Question 10: Why is agreement between two different assistants weak evidence that a criticism is correct?
Question 11: Which limitation of the study by Lee and colleagues (2025) should a careful reader keep in mind?
Question 12: In the worked demonstration, the assistant stated that the program records held home addresses and then reported that no reason depended on facts it had not been given. Which check caught the error?
Question 13: At the end of the worked demonstration, Jordan recorded each point raised, the decision taken and the reason. Why is such a record valuable?
Question 14: An assistant reviewing the Connector logic model proposes cultural activities as outputs and cultural connectedness as an outcome for the land-based pathway, which is being co-designed with a First Nations partner. What is the best response?
Question 15: Which statement about simulated perspectives is consistent with this lesson and with HSCI 207 Lesson 4?
Glossary: Key Terms, People & Frameworks
📚 Reference page, available throughout the lesson
Use this glossary to review the terms, frameworks and people introduced in this lesson on using AI to generate and test ideas.