Knowing that experiments can show cause is only the start. A badly run experiment can still mislead. Four principles turn an experiment into convincing evidence.
Key idea
Comparison: compare two or more treatments, often including a control group (no treatment or a placebo). Random assignment: use chance to assign subjects to treatments. Replication: use enough subjects in each group that chance differences average out. Blocking (optional): group similar subjects first and randomize within each block. Add blinding when expectations could affect results.
Why it works
Each principle removes a different way to be fooled. Without comparison, any change might be due to time passing. Without random assignment, the groups might differ from the start. Without replication, one unusual subject could decide the result. Blocking removes a known source of variation (like gender or field location) from the comparison, so a real treatment effect is easier to see. A placebo and blinding stop the act of being treated, or the researcher's hopes, from changing the response.
Worked example: Designing an experiment
Design an experiment to test whether a new fertilizer increases tomato yield, using 40 plants.
- Treatments: new fertilizer versus the usual one (comparison).
- Number the plants 1–40 and use a random number generator to pick 20 for the new fertilizer; the other 20 get the usual one (random assignment, replication).
- Grow all plants in the same conditions and compare the mean yield of the two groups.
- If half the plants are in sun and half in shade, randomize within the sunny plants and within the shady plants separately (blocking).
Answer: A randomized comparison of two groups of 20, blocked by location if needed.
Worked example: Counting treatments
An experiment tests 3 watering levels and 2 kinds of soil, with every combination used. Each treatment gets 5 plants. How many treatments and plants?
- Every watering level pairs with every soil: $3 \times 2 = 6$ treatments.
- Replication: $6 \times 5 = 30$ plants.
Answer: 6 treatments, 30 plants.
| Randomly assigned | Not randomly assigned | |
|---|---|---|
| Randomly sampled | Cause and effect, and generalize to the population | Generalize an association; no cause |
| Not randomly sampled | Cause and effect for these subjects only | Neither: describe these subjects only |
Common mistake
Confusing random sampling with random assignment. Random sampling (choosing who is in the study) lets you generalize to the population. Random assignment (choosing who gets which treatment) lets you conclude cause and effect. Volunteers randomly assigned to diets give causal evidence about volunteers like them, not a statement about all adults. Check which kind of randomness the study used.