15 Internal Validity Examples: A Beginner’s Guide

Have you ever wondered if a new diet really caused weight loss, or if a new teaching method truly improved test scores? Asking these questions gets to the heart of internal validity. It’s a crucial concept in research, especially when we want to know if one thing genuinely causes another.

Understanding internal validity helps us trust research findings. It tells us if the researchers did a good job controlling for other factors that could have influenced the results. Without it, we might draw the wrong conclusions.

This article will make internal validity super simple. We’ll explain what internal validity is (in plain English), why it’s so important, the difference between internal and external validity. We will provide 15 clear internal validity examples, illustrating common threats and their respective solutions.

What is Internal Validity?

Download the above infographic in PDF

Internal validity is about confidence. It’s the level of confidence we have that a study accurately shows a cause-and-effect relationship between two variables.

Think of it like being a detective at a crime scene. The detective wants to know for sure who committed the crime (the cause) and rule out other suspects (alternative explanations).

In research terms:

• Cause: The treatment, intervention, or factor being studied (the independent variable).
• Effect: The outcome or result being measured (the dependent variable).
• Internal Validity: How sure are we that the independent variable actually caused the change we see in the dependent variable within the specific context of this study?

High internal validity means we can confidently say, “Yes, in this study, X likely caused Y.” We’ve done a good job ruling out other possible explanations.

Low internal validity means we’re uncertain. We might say, “Well, Y changed after X, but it could have been because of Z, or maybe something else entirely.” There are plausible alternative explanations (called confounding variables) for the results.

The Core Question of Internal Validity: Did the treatment really make the difference, or was it something else?

Why is Internal Validity Important?

Internal validity isn’t just academic jargon; it’s fundamental to reliable knowledge. Here’s why it matters:

1. Establishing True Cause-and-Effect: The main goal of many experiments is to find out what causes what. High internal validity is essential to achieve this goal.
2. Making Sound Decisions: If a study with low internal validity suggests a new drug works, doctors might prescribe it mistakenly. If a flawed study suggests a new policy is effective, governments might waste resources implementing it. We need valid results to make good choices.
3. Building Reliable Knowledge: Science progresses by building on previous findings. If the foundational studies have low internal validity, the entire structure of knowledge becomes shaky.
4. Avoiding Misleading Conclusions: Poor internal validity can lead researchers and the public to believe things that aren’t true, potentially causing harm or preventing effective solutions from being recognized.
5. Evaluating Research Critically: Understanding internal validity allows you (whether you’re a student, professional, or just curious) to assess the quality of research you encounter.

In short, internal validity is the bedrock upon which trustworthy causal conclusions are built.

Internal vs. External Validity: A Quick Comparison

People often confuse internal validity with external validity. They are both important but measure different things.

Downlaod the above infographic in PDF

Think of it this way:

• Internal Validity: Is the study’s conclusion correct for the sample studied?
• External Validity: Is the study’s conclusion relevant beyond the specific sample and setting? For greater clarity, see our article for external validity examples.

Ideally, researchers aim for a balance, but sometimes strengthening one can weaken the other. For example, a highly controlled lab setting (good for internal validity) might not reflect the real world (potentially lower external validity).

15 Examples Illustrating Threats to Internal Validity

Download the above infographic in PDF

The best way to understand internal validity is to see what happens when it’s threatened. These “threats” are specific factors or events that can provide an alternative explanation for the results, making us doubt the cause-and-effect link.

Here are 15 common threats with simple examples:

Threat 1: History

• What it is: An external event occurs during the study that affects the outcome (dependent variable), unrelated to the treatment (independent variable). This event happens between measurements.
• Example:
  • Study: A company implements a new wellness program (IV) to reduce employee stress (DV), measured before and after the 6-month program.
  • Threat: During the 6 months, the company unexpectedly announces major layoffs.
  • Problem: At the end of the study, stress levels might be higher, lower, or unchanged. If they changed, was it because of the wellness program, or the layoff announcement (history threat)? We can’t be sure.
• How to Mitigate: Use a control group that doesn’t get the program but experiences the same historical events. Compare the change in stress between the groups.

Threat 2: Maturation

• What it is: Natural changes occurring in participants over time (growing older, getting more experienced, becoming tired or bored) that affect the outcome, independent of the treatment.
• Example:
  • Study: A researcher tests a new reading program (IV) designed to improve reading skills (DV) in first graders over a full school year. Skills are measured at the start and end of the year.
  • Threat: First graders naturally improve their reading skills significantly throughout the year just by being in school and growing older.
  • Problem: If reading skills improve, how much was due to the special program versus normal development (maturation)?
• How to Mitigate: Use a control group of first graders who don’t use the new program but are measured at the same times. Compare the improvement rates.

Threat 3: Testing

• What it is: The act of taking a test itself influences participants’ scores on subsequent tests (e.g., they remember questions, become more test-savvy, or get anxious).
• Example:
  • Study: A psychologist gives employees a problem-solving test (pre-test), then provides a critical thinking workshop (IV), and finally gives the same problem-solving test again (post-test) to measure improvement (DV).
  • Threat: Simply taking the first test might make employees think more about problem-solving strategies or remember specific questions, leading to higher scores on the second test, even without the workshop.
  • Problem: Was the improvement due to the workshop or the practice effect from the pre-test (testing threat)?
• How to Mitigate: Use a control group that takes both tests but doesn’t attend the workshop. Or, use different but equivalent versions of the test. Consider study designs without a pre-test (like the Solomon Four-Group Design).

Threat 4: Instrumentation

• What it is: Changes in the measuring instrument or measurement procedures over time affect the outcome. This includes changes in scorers, observers, or the calibration of a device.
• Example:
  • Study: Researchers are observing children’s aggressive behavior (DV) on a playground before and after implementing an anti-bullying program (IV). Two different observers rate the behavior: one before the program and a different one after.
  • Threat: The second observer might be stricter or more lenient in their definition of “aggressive behavior” than the first observer.
  • Problem: If observed aggression changes, is it because of the program or because the way aggression was measured changed (instrumentation threat)?
• How to Mitigate: Use the exact same measurement tools and procedures throughout. Train observers thoroughly and check for consistency (inter-rater reliability). Calibrate instruments regularly.

Threat 5: Statistical Regression (Regression to the Mean)

• What it is: When participants are selected for a study because they have extreme scores (very high or very low), their scores tend to move closer to the average on a subsequent test, regardless of any treatment.
• Example:
  • Study: A school selects students with the lowest math scores (bottom 10%) for a special tutoring program (IV). Their math scores (DV) are measured again after the program.
  • Threat: Students who score extremely low on one test are statistically likely to score slightly higher on the next test just due to random variation and measurement error, even without tutoring. Extremely low scores often have an element of “bad luck” that isn’t likely to repeat.
  • Problem: If the low-scoring students’ scores improve, how much is due to the tutoring versus this natural tendency to regress toward the average?
• How to Mitigate: Use a control group selected from the same pool of low-scoring students who don’t receive the tutoring. Avoid selecting participants based only on extreme scores if possible.

Threat 6: Selection Bias

• What it is: Systematic differences between the groups being compared before the study begins. The groups are not equivalent at the start, which can influence the outcome. This is common when participants are not randomly assigned to groups.
• Example:
  • Study: A hospital compares recovery times (DV) for patients who chose Surgery A versus patients who chose Surgery B (IV).
  • Threat: Patients who choose Surgery A might be younger, healthier, or more motivated than those who choose Surgery B. These pre-existing differences could affect recovery time.
  • Problem: If Surgery A patients recover faster, is it because Surgery A is better, or because those patients were healthier to begin with (selection bias)?
• How to Mitigate: Random assignment is the gold standard. If not possible (quasi-experiment), try to match participants on key characteristics or statistically control for pre-existing differences.

Threat 7: Attrition (Experimental Mortality)

• What it is: Participants dropping out of the study before it’s completed. If the dropout rate is different between groups, or if specific types of participants drop out more from one group, it can bias the results.
• Example:
  • Study: A 12-month study compares a new, intensive weight loss program (Group A) with a standard advice program (Group B). The outcome (DV) is weight loss.
  • Threat: Many participants in Group A (the intensive program) find it too demanding and drop out. Perhaps mostly those who weren’t losing much weight leave. Group B has fewer dropouts.
  • Problem: At the end, Group A might look more effective because only the most successful and motivated participants remained. The comparison is skewed because the groups are no longer equivalent due to differential dropout (attrition threat).
• How to Mitigate: Encourage participation, keep demands reasonable, track dropouts carefully, and analyze why they left. Use statistical techniques (like intent-to-treat analysis) to account for dropouts.

Threat 8: Ambiguity of Temporal Precedence

• What it is: It’s unclear whether the supposed cause (IV) actually happened before the supposed effect (DV). This is common in correlational studies where variables are measured at the same time.
• Example:
  • Study: A researcher surveys employees and finds a positive correlation between job satisfaction (Variable 1) and job performance (Variable 2). They measure both at the same time.
  • Threat: Does high satisfaction cause high performance? Or does high performance lead to high satisfaction? Or does some third factor (like good management) cause both?
  • Problem: We can’t establish cause-and-effect because we don’t know which came first (ambiguity of temporal precedence). Correlation does not equal causation.
• How to Mitigate: Use experimental designs where the researcher manipulates the IV before measuring the DV. Use longitudinal designs that measure variables over time.

Threat 9: Diffusion or Imitation of Treatments

• What it is: Participants in the control group accidentally receive or learn about the treatment being given to the experimental group. This contaminates the control group, reducing the difference between the groups.
• Example:
  • Study: A teacher tries a new, engaging teaching technique (IV) with half the class (Group A) while the other half (Group B, control) uses the standard method. The outcome is student engagement (DV). The classroom setup allows students to easily see and hear what the other group is doing.
  • Threat: Students in the control group see the fun activities in Group A and start imitating them, or the teacher unconsciously uses some new techniques with the control group too.
  • Problem: The control group is no longer a true “no treatment” group. If there’s little difference in engagement between groups, it might be because the treatment “spread” (diffusion threat), not because the treatment was ineffective.
• How to Mitigate: Keep the groups physically separate if possible. Clearly instruct participants and researchers involved not to share details between groups. Use blinding if feasible.

Threat 10: Compensatory Rivalry (John Henry Effect)

• What it is: Participants in the control group learn they are not receiving the “special” treatment and work extra hard to outperform the experimental group. Named after the folk hero John Henry who raced against a steam drill.
• Example:
  • Study: A factory tests a new automated machine (IV) with one team of workers (Group A) to see if it increases productivity (DV) compared to a team using the old methods (Group B, control). Group B knows they are being compared to the “high-tech” group.
  • Threat: The control group workers feel challenged and determined to prove they are just as good or better than the machine. They work harder and more efficiently than usual.
  • Problem: If productivity differences are small, is it because the new machine isn’t much better, or because the control group artificially boosted their performance (compensatory rivalry)?
• How to Mitigate: Keep groups unaware they are being compared if possible (blinding). Provide equal attention and recognition to both groups. Measure performance discreetly.

Threat 11: Resentful Demoralization

• What it is: The opposite of compensatory rivalry. Participants in the control group learn they are not receiving the desirable treatment and become discouraged, resentful, or less motivated. Their performance might decrease.
• Example:
  • Study: A company offers a desirable flexible work schedule (IV) to one randomly selected department (Group A) but not to another similar department (Group B, control). The outcome is job satisfaction (DV). Group B knows Group A got the flexible schedule.
  • Threat: Employees in Group B feel unfairly treated and become unhappy and less satisfied with their jobs, below their usual level.
  • Problem: If Group A shows much higher job satisfaction, is it solely because the flexible schedule is great, or also because the control group’s satisfaction artificially dropped due to feeling left out (resentful demoralization)?
• How to Mitigate: Try to offer the control group something desirable (but unrelated to the IV) or promise them the treatment later if it proves effective. Maintain confidentiality about who is in which group if possible. Ensure fair treatment otherwise.

Threat 12: Experimenter Bias (Observer Bias)

• What it is: The researcher’s expectations or preferences unintentionally influence the study’s results. This can happen during participant interaction, data collection, or data analysis.
• Example:
  • Study: A researcher believes a new therapy technique (IV) is highly effective for reducing anxiety (DV). They personally conduct interviews to assess anxiety levels before and after therapy for both the treatment and control groups.
  • Threat: The researcher might unintentionally ask leading questions, interpret ambiguous responses more favorably for the treatment group, or be warmer and more encouraging to participants receiving the new therapy.
  • Problem: Any observed difference in anxiety reduction might be due to the researcher’s biased behavior or judgment, not the therapy itself.
• How to Mitigate: Use double-blinding, where neither the participants nor the researchers interacting with them know who is in the treatment or control group. Standardize procedures strictly. Have independent raters analyze the data.

Threat 13: Demand Characteristics

• What it is: Participants figure out (or think they’ve figured out) the study’s purpose and change their behavior to either support or undermine the researcher’s expected findings.
• Example:
  • Study: College students participate in a study supposedly about “visual perception” but which is actually testing the effect of perceived authority (IV: researcher wearing a lab coat vs. casual clothes) on obedience (DV: willingness to perform a boring task).
  • Threat: Students might guess the study is about obedience, especially if they see the lab coat. Some might try to be “good subjects” and obey more, while others might deliberately resist to assert independence.
  • Problem: Participants’ responses might reflect their reaction to the perceived hypothesis (demand characteristics) rather than a genuine effect of the independent variable.
• How to Mitigate: Use deception or cover stories (ethically approved). Use blinding. Make the dependent variable measure less obvious. Ask participants afterward what they thought the study was about.

Threat 14: Placebo Effect

• What it is: Participants experience a real change or improvement simply because they believe they are receiving an effective treatment, even if the treatment is inert (like a sugar pill).
• Example:
  • Study: A pharmaceutical company tests a new headache pill (IV). Group A gets the real pill, Group B gets an identical-looking sugar pill (placebo). Headache relief (DV) is measured.
  • Threat: Participants in Group B (placebo) might report significant headache relief simply because they expected the pill to work.
  • Problem: To demonstrate the new pill is effective, it must show significantly more relief than the placebo effect alone. The placebo effect provides an alternative explanation for improvement in the absence of an active ingredient.
• How to Mitigate: Always use a placebo control group in clinical trials. Use blinding (single or double) so participants (and ideally researchers) don’t know who got the real treatment versus the placebo.

Threat 15: Interaction Effects (e.g., Selection-History Interaction)

• What it is: A threat occurs when one of the above threats affects the groups differently. For example, an external event (History) might impact the treatment group more than the control group, especially if the groups weren’t equivalent to begin with (Selection Bias).
• Example:
  • Study: A study compares a new job training program (IV) offered at one inner-city location (Group A) with no program offered at a suburban location (Group B, control). The outcome is job placement rates (DV). The groups were not randomly assigned (potential Selection Bias).
  • Threat: During the study, a major new factory opens near the inner-city location (History event). This factory primarily hires people from the local community.
  • Interaction (Selection-History): The historical event (new factory) provides job opportunities primarily for Group A, but not Group B.
  • Problem: If Group A has higher job placement rates, is it due to the training program, or the location-specific job opportunity that disproportionately affected them (interaction of selection and history)? We can’t separate the effects.
• How to Mitigate: Random assignment helps make groups equivalent, reducing initial selection differences. Conduct the study in settings where groups are likely to experience similar historical events. Monitor for external events that might affect groups differently.

Strategies to Enhance Internal Validity

While it’s impossible to eliminate all threats perfectly, researchers use several strategies to maximize internal validity:

1. Randomization (Random Assignment): Assigning participants to treatment or control groups purely by chance. This is the single best way to ensure groups are equivalent at the start, minimizing selection bias and making it less likely other threats will affect groups differently.
2. Control Groups: Having a group that does not receive the treatment being tested. This group provides a baseline to compare against, helping to isolate the treatment’s effect from other factors (like history, maturation, testing, regression).
3. Blinding (Masking):
   • Single-Blind: Participants don’t know if they are in the treatment or control group (helps control placebo effects, demand characteristics).
   • Double-Blind: Neither participants nor the researchers interacting with them know group assignments (controls placebo effects, demand characteristics, and experimenter bias).
4. Standardized Procedures (Controlling Instrumentation): Using the exact same procedures, measurement tools, instructions, and environment for all participants in all groups. Training researchers and observers thoroughly.
5. Careful Participant Selection: While random assignment deals with group equivalence, the initial selection of participants should be clearly defined to understand who the results apply to. Avoid selecting only extreme scorers if possible.
6. Managing Attrition: Tracking why participants drop out and using statistical methods to account for missing data. Trying to keep dropout rates low and similar across groups.
7. Controlling the Environment and Timing: Conducting the study in a controlled setting (if appropriate) and keeping the duration as short as feasible to minimize exposure to historical events or maturation effects.

Conclusion: Trusting the Cause

Internal validity is all about whether we can confidently conclude that the treatment, and not something else, caused the observed outcome within a specific study. It’s the foundation for trusting research that claims to show cause and effect.

By understanding the common threats – from historical events and natural maturation to testing effects and selection bias – we can better evaluate the quality of research. The 15 examples illustrate how easily alternative explanations can arise if studies aren’t carefully designed.

Researchers strive to strengthen internal validity using techniques like randomization, control groups, and blinding. While no study is perfect, a high degree of internal validity gives us greater confidence that the findings are accurate and meaningful. The next time you read about a study showing that X causes Y, ask yourself: How well did they control for other explanations? How strong is the internal validity? This critical thinking is key to navigating a world full of information and making informed decisions based on reliable evidence.