PSLE Science Reality Lab Vol No.332 | “This Study Was Preregistered” — Does That Mean the Result Must Be Correct?

PSLE-SCI-REALITY-0332

Wait, What? A Study Can Be Preregistered and Still Be Wrong

A science-news graphic proudly says, “Preregistered study.” One student nods: “Good. That means the result has been proven.” Another student shakes her head: “No, preregistration means the study is bad because the researchers were not allowed to change anything.” Both have misunderstood what the label is for.

Preregistration is a transparency practice. A research team records important parts of its intended study—such as questions, hypotheses, methods or analyses—in a time-stamped record before seeing the relevant results. That record makes it easier for later readers to compare what was planned with what was eventually done and reported.

That can strengthen the evidence trail. It cannot magically make a weak measurement accurate, a biased sample representative, a coding error disappear, or an overconfident conclusion become true.

Quick Answer

Preregistration records a study plan before relevant results are known.
It helps readers distinguish planned tests from later exploratory choices.
It does not guarantee that the plan was good, the study was executed perfectly, or the conclusion is correct.
Changes from the plan are not automatically scientific misconduct; the important question is whether changes are disclosed and justified.
Evaluate preregistration as one piece of evidence about transparency, not as a truth stamp.

The Exact Learner Job This Page Owns

This page owns one real-world evidence-transfer job: evaluating a “preregistered” research label by separating transparency about the study plan from evidence that the measurements, execution and conclusions are valid.

It does not own experimental design, sampling, statistics, peer review, replication or research ethics as broad subjects. Existing PSLE Science and Reality Lab pages retain those jobs. This article teaches a learner how to read one communication object: the preregistration claim.

Original Reality Lab Case: The Locked Envelope

This is an original composite case. Imagine a research team planning to compare the growth of seedlings under two lighting conditions. Before collecting the main data, the team writes down:

which seedlings will be included;
the two lighting conditions;
how growth will be measured;
when the measurements will stop;
the main comparison they intend to make.

The plan is time-stamped in a registry. Later the researchers discover that one measuring device failed on two days. They replace those readings using a clearly described rule and explain the change in the final report.

Did preregistration fail because the study changed? No. The scientific value of the record is precisely that a reader can see the original plan, the deviation and the explanation. Transparency is not the same as pretending nothing unexpected happened.

Planned, Performed, Reported and Inferred

Layer	Question to ask
Planned	What did the researchers say they intended to do before seeing the relevant result?
Performed	What actually happened during data collection and analysis?
Reported	Did the final paper clearly describe important deviations and outcomes?
Inferred	How far do the data justify the final conclusion?

The Provenance Check: Can You Find the Time-Stamped Plan?

A vague sentence saying “we preregistered” is less informative than a traceable record. A strong communication object gives readers enough information to locate the registration, identify its date and compare it with the final study.

The Open Science Framework describes preregistration as a time-stamped, read-only or version-tracked study plan created before relevant data collection or analysis. The important evidence feature is temporal order: the plan existed before the result could influence those planned decisions.

The Plan-Quality Check: Early Does Not Mean Good

A plan can be recorded early and still contain a poor comparison, an unsuitable measuring method or an unrepresentative sample. Preregistration can show that a choice was made in advance. It cannot by itself show that the choice was scientifically strong.

This is a subtle but powerful distinction. Evidence about when a decision was made is not the same as evidence that the decision was correct.

The Execution Check: Did the Study Follow the Plan?

After finding a preregistration, compare it with the actual report. Were the intended variables measured? Was the planned sample obtained? Were important outcomes omitted? Did the analysis change? If it changed, is the change explained?

A difference between plan and execution is not automatically suspicious. Scientific work encounters broken equipment, unexpected data quality problems and new information. The key learner habit is to keep the difference visible so readers can judge what changed and why.

The Exploration Check: New Questions Are Allowed

Suppose a team planned to test whether light colour affects stem height. During the study they notice an unexpected leaf pattern and explore it. That new observation can be useful science. The problem would be to present the new exploratory finding as though it had been the original planned test all along.

Preregistration helps readers separate confirmation from exploration. Both can be valuable; they answer different evidential questions.

The Method Check: Transparency Does Not Repair Measurement

Imagine a preregistered experiment measuring water temperature with a faulty thermometer. The researchers may have followed the registered plan perfectly, yet the measurement problem remains. The preregistration tells us the faulty method was planned; it does not make the thermometer accurate.

The same principle applies to uncontrolled variables, non-independent samples, missing data and model assumptions. Transparent weakness is easier to inspect than hidden weakness, but it is still weakness.

The Outcome Check: Was Everything Reported?

One reason preregistration is useful is that it can expose selective reporting. If five outcomes were planned but only the most dramatic one appears in the final article, a careful reader can ask what happened to the other four.

That does not prove wrongdoing. Some outcomes can become unusable for legitimate reasons. The registry simply gives the reader a better question to ask.

Alternative Explanations for a Plan–Paper Difference

An instrument failed.
A planned variable could not be measured reliably.
Recruitment or sampling stopped early for a stated reason.
An assumption required a different analysis.
A genuine exploratory question emerged after the data were seen.
The reporting may be incomplete or selectively framed.

Healthy scepticism keeps several explanations alive. The next step is not accusation. It is evidence gathering.

What Evidence Strengthens a Preregistration Claim?

A persistent link or identifier leads to the registration.
The registration timestamp clearly precedes the relevant analysis or data access.
The main hypotheses, outcomes and analysis decisions are specific enough to compare with the final report.
Important deviations are disclosed and explained.
Exploratory analyses are labelled as exploratory rather than silently rewritten as planned.

What Weakens a Strong “Preregistered” Impression?

The registration cannot be located.
The plan is so vague that almost any result fits it.
The plan was registered after the relevant results were already visible.
Major outcomes or methods changed without explanation.
The communication uses the word preregistered as if it replaces peer review, replication or measurement checks.

Worked Case 1: A Perfectly Followed Weak Plan

A fictional team preregisters a comparison of two plant groups but places one group near a window and the other in a dim corner. The plan is followed exactly. Does preregistration make the comparison fair? No. The light difference is a confounding condition whether it was planned early or late.

Worked Case 2: A Transparent Necessary Change

A sensor breaks halfway through a preregistered investigation. The team switches to a backup sensor, reports the change, compares the two instruments and performs a sensitivity check. The deviation does not erase the study. It becomes part of the evidence the reader must evaluate.

Worked Case 3: An Unexpected Pattern

The registered question concerns germination time. Researchers later notice seed colour may also matter. They explore colour and clearly mark that analysis as post-hoc. Is the exploratory result useless? No. It may generate a new hypothesis, but its evidential status differs from the preplanned test.

Worked Case 4: The Badge Without the Record

A social-media graphic adds a shiny “PREREGISTERED” badge but gives no registry link, date or plan. A learner should not assume fraud, but should downgrade confidence in the badge until the underlying record can be inspected.

Tempting Reasoning That Fails

“Preregistered means proven.” It documents planning; it does not validate every scientific step.
“Any deviation means cheating.” Changes can be scientifically necessary if they are transparent and justified.
“Exploratory findings do not count as science.” Exploration can generate useful hypotheses; it simply has a different evidential role.
“Not preregistered means automatically false.” Many legitimate studies are not preregistered; evidence quality still depends on method and data.

How Far Can the Conclusion Travel?

A verified preregistration can support a statement such as: “These key questions and analysis decisions were documented before the relevant results were known.” It can strengthen transparency and make selective changes easier to detect.

It cannot alone support: “The result is correct,” “the sample represents everyone,” “the measurement was accurate,” or “another independent team has replicated the finding.” Those require different evidence.

PSLE-Style Transfer Case

A fictional study is preregistered before data collection. The final report follows the planned analysis but uses a measuring instrument later found to have a systematic offset.

Question: Does preregistration prove the numerical result is accurate?

Reasoned answer: No. Preregistration shows that the analysis plan was documented before the results were known. It does not remove measurement error. The instrument problem must be evaluated separately.

Explained Practice

Practice A: A study changes one analysis after discovering its original assumption is not met and explains the change. Is it automatically invalid? No. Evaluate the reason, transparency and effect of the change.

Practice B: A preregistration lists ten outcomes but the article reports one. What question should you ask? What happened to the other planned outcomes?

Practice C: A study is preregistered and peer reviewed. Does that equal independent replication? No. Planning transparency, editorial review and new independent evidence are different jobs.

Delayed Independent Return: P-L-A-N

P — Prior record: Was the plan recorded before the relevant result was known?
L — Locate it: Can the registration actually be inspected?
A — Align plan and action: What changed between plan and execution?
N — No truth stamp: Evaluate measurement, method and conclusion separately.

Parent and Tutor Teaching Guide

Use a sealed-envelope activity. Before revealing the outcome of a simple classroom demonstration, ask the learner to write a prediction and the rule they will use to judge the result. After the result, let them compare the original plan with what they now wish they had written.

The lesson is not “never change your mind.” The lesson is that a prior record helps separate prediction from hindsight. Then ask whether writing the prediction early made it correct. Of course not. It made the reasoning trail visible.

Authoritative Sources

The Quiet Return

Preregistration does not make science infallible.

It makes part of the scientific journey harder to rewrite after the destination is known.

Transparency strengthens judgement because it lets the reader see what came before the result.

Small Group Tutorials