PSLE Science Reality Lab Vol No.436 | “Systematic Review” — Does That Mean It Found Every Study Ever Done?

Series ID: PSLE-SCI-REALITY-0436

Wait, What? A Review Can Be Systematic Without Being a Search of the Entire Universe

A science-news card says: “Systematic review of 42 studies finds that salt reduces seed germination.” The phrase systematic review sounds almost like a promise that researchers found every experiment that has ever been done on salt and seeds.

That is not what the label means. A systematic review is valuable because it uses an explicit, planned way to search for, select, appraise and bring together relevant studies. Its strength comes from making those decisions visible and reproducible. But every review still has a question, a search date, information sources, search terms, eligibility rules and practical limits.

The Primary 5/6 learner’s job is not to become a research librarian. It is to learn a durable PSLE Science habit: when a communication object summarises many studies, reconstruct how those studies entered the evidence set before treating the summary as complete.

Quick Answer

“Systematic review” means the authors used a structured method to identify and assess evidence for a defined question. It does not mean every study ever done was found, every study was equally strong, every result agreed, or the review answers questions outside its stated scope.

Read the exact review question.
Check where and when the authors searched.
Check which studies were eligible.
Notice how many records were found, removed, excluded and finally included.
Ask why studies were excluded.
Separate “not included” from “does not exist”.
Check whether the included studies really match the public headline.
Keep the conclusion inside the review’s evidence boundary.

The Owned Learner Job

This article owns one evidence-transfer job: how to evaluate the communication label “systematic review” by tracing the search-and-selection pathway that produced the final set of studies.

It does not teach generic database searching, formal evidence hierarchies, advanced statistics or meta-analysis. It does not replace the existing eduKateSengkang owners for sampling, stronger evidence, data interpretation or model limits. Instead, it applies those ideas to one real-world scientific object: a review that claims to summarise a body of evidence.

Think of a Systematic Review as an Evidence Funnel

Imagine a funnel with five stages.

Question: What exact question is the review trying to answer?
Search: Which databases, registers, websites, reference lists or other sources were searched, using what terms, up to what date?
Screen: Which records looked potentially relevant?
Eligibility: Which studies met the pre-stated inclusion rules?
Synthesis: What can reasonably be concluded from the studies that survived the funnel?

A systematic method makes the funnel inspectable. That matters because the final number of included studies is not simply “everything scientists know”. It is the result of a defined process.

Observed, Claimed and Inferred

Suppose an original composite review reports:

1,240 records identified;
210 duplicates removed;
1,030 titles and abstracts screened;
86 full reports examined;
28 studies included in the final review.

Observed: the review process produced 28 included studies under its stated rules.

Claimed: those 28 studies are the evidence set used to answer the review’s question.

Tempting inference: “Only 28 studies exist in the world” or “the authors found every possible relevant study”. Neither conclusion follows merely from the inclusion count.

The learner should ask what happened between 1,240 and 28. A large reduction can be completely legitimate: duplicate records, unrelated topics, wrong populations, unsuitable methods, wrong outcomes or reports of the same study may all be removed. The scientific question is whether the rules were appropriate, explicit and applied consistently.

Why the Search Boundary Matters

Cochrane’s current handbook explains that systematic reviews aim for thorough, objective and reproducible searches across relevant sources, while recognising practical limits. It also explains why searching only one database can miss relevant studies and can make the retrieved evidence less representative.

Translate that into Primary Science language: where you look changes what you can find.

If a review searches only one journal, it may miss studies elsewhere. If it searches only up to 2022, it cannot include a study first published in 2025. If it includes only experiments on one plant species, it cannot automatically claim the same result for every plant species.

Search Date Is Evidence Too

A review is always made at a time. New research can appear later. That does not make the older review dishonest; it means its evidence set has a time boundary.

Cochrane even uses the idea of a “living systematic review” for reviews that are deliberately updated as new evidence appears. The existence of living reviews teaches an important lesson: ordinary reviews are snapshots of a searched evidence base, not permanent maps of all future knowledge.

The PRISMA Flow Diagram: A Map of Selection, Not a Quality Medal

PRISMA provides widely used reporting guidance for systematic reviews. Its flow diagram shows how records move through identification, screening and inclusion, including exclusions and reasons for them.

For this Reality Lab, the useful learner question is not “Does it have a PRISMA diagram?” as though the diagram were a magic badge. Ask instead: what does the diagram reveal about how the final evidence set was constructed?

A clear flow diagram improves transparency. It does not by itself prove that the search terms were perfect, that every database was searched, that every included study was strong or that the final interpretation is correct.

Original Composite Case 1: The Missing Year

A headline in 2026 says, “Systematic review proves blue light has no effect on plant growth.” The review itself states that searches were completed in December 2022. Three relevant experiments were published in 2024 and 2025.

Tempting reasoning: The review is systematic, so later studies do not matter.

Repair: The review may have been systematic for the evidence available up to its search date. The later studies sit outside that search window. They should not be pretended away, and they do not automatically overturn the review either. A current judgement should consider the older review together with relevant newer evidence.

The key job is recognising a time boundary, not attacking the word “systematic”.

Original Composite Case 2: The Wrong Species

A review asks: “How does salt concentration affect germination of rice seeds?” It includes 19 rice studies. A social-media graphic changes the conclusion to: “Systematic review shows salt prevents all seeds from germinating.”

The review label is not the problem. The problem is scope expansion. The question was about rice under specified conditions. “All seeds” travels to other species that were not part of the defined review question.

A learner should be able to point to the exact word that travels too far: all.

Original Composite Case 3: Eight Reports, Five Studies

A review search finds eight papers. A quick reader assumes that means eight independent experiments. But three papers describe different results from the same long-running study.

Cochrane explicitly distinguishes studies from reports of studies: one study can have several reports. This matters because counting reports as independent studies could make the evidence base look larger than it really is.

For Primary Science, keep the question simple: how many independent investigations are represented, not merely how many documents were found?

Original Composite Case 4: Excluded Does Not Mean Disproved

A flow diagram says 12 studies were excluded because they measured leaf colour rather than plant height. A student says, “Those 12 studies must have been wrong.”

No. They may be perfectly good studies for a different outcome. They were excluded because they did not answer the review’s chosen question.

This is a powerful transfer habit. Not relevant to this question is different from scientifically false.

Comparison and Baseline Check

When a review combines many studies, do not assume all of them made the same comparison. One experiment may compare 0% salt with 1% salt. Another may compare 0.5% with 2%. Another may use a different temperature or seed age.

A careful review explains how those differences are handled. A public infographic may erase them and show only one sentence. Your job is to restore the comparison conditions before repeating the broad claim.

Method Check: The Review Is Only as Useful as the Link Between Question and Evidence

Was the review question clear enough to decide what belongs?
Were the search sources suitable for that question?
Was the search date stated?
Were eligibility rules decided and reported?
Were exclusion reasons visible?
Were duplicate reports of the same study recognised?
Did the reviewers examine study quality or risk of bias?
Did the final conclusion stay within the included evidence?

You do not need every technical detail to ask these questions. They are simply a larger version of familiar PSLE Science reasoning: define the question, inspect the method, interpret the evidence, evaluate limits, communicate a conclusion that fits.

What Would Strengthen Confidence in the Review?

A clearly stated question and eligibility criteria.
Searches across appropriate sources rather than one convenient source.
A stated search date and reproducible search description.
A transparent flow from records found to studies included.
Clear reasons for exclusions.
Recognition of multiple reports from the same study.
Appraisal of limitations in included studies.
A conclusion that acknowledges disagreement, uncertainty and evidence gaps.
An update when important new evidence appears.

What Would Narrow Confidence?

The search sources are not stated.
The search is very old but the headline sounds current.
The inclusion rules appear after the results and seem chosen to favour one answer.
Records are counted as though each were an independent study.
Excluded studies disappear without explanation.
The public headline expands the review to populations, species, outcomes or conditions outside its question.
The phrase “systematic review” is used without enough method detail to see what was systematic.

Tempting but Invalid Reasoning

“Systematic means complete.” Better: systematic means the review follows an explicit method; completeness still depends on the search scope and available evidence.
“More included studies means a better review.” Better: relevance and quality matter; 100 weak or mismatched studies are not automatically stronger than a smaller well-matched evidence set.
“Excluded means wrong.” Better: exclusion may simply mean the study does not meet the review question or design rules.
“A review is newer than every included study, so it is fully current.” Better: check the last search date, not only the publication date.
“All studies in the review agree.” Better: a review can include disagreement and still produce a careful synthesis.

PSLE-Style Transfer Case

An original review card says: “Systematic review: coloured light does not affect bean growth.”

Method summary: Searches were run in two databases up to June 2023. Studies were included only if bean plants were grown for at least 14 days and final stem height was reported. Twelve studies were included. A 2025 experiment measured leaf number rather than stem height.

Question 1: Why was the 2025 experiment not part of the original review?

Answer: It was published after the review’s search date, and it measured a different outcome from the stated inclusion rule.

Question 2: Does its absence prove the review was badly conducted?

Answer: No. The review may have been systematic within its stated date and outcome rules. A current conclusion should simply recognise that newer or differently focused evidence exists.

Delayed Independent Return

Tomorrow, draw an evidence funnel from memory with five boxes: question → search → screen → include → conclude. Then take a made-up science headline and write one condition that could narrow each box.

If you can do that without repeating the wording from this article, you understand the job: the conclusion inherits the boundaries of the path that produced its evidence.

Useful eduKateSengkang Routes

Parent and Tutor Teaching Guide

Do not begin with technical vocabulary. Begin with a pile of 20 fictional study cards. Give the learner a review question such as “How does salt affect germination of rice seeds?” Mix in cards about other plants, other outcomes and duplicate reports of the same experiment.

Ask the learner to write inclusion rules before sorting. Then make them explain each exclusion. Change one rule and let them watch the final evidence set change. This makes the hidden architecture of a systematic review visible without turning the lesson into university research methods.

End with one case in which an excluded study is scientifically sound but irrelevant to the chosen question. That single contrast prevents a common mistake: confusing exclusion with falsity.

Authoritative Sources

The Quiet Return

A systematic review is powerful because it makes evidence collection less casual and more traceable. Respect that strength without turning the label into perfection. Ask: What question defined the search? Where did the authors look? What entered? What left? Why? And how far can the final conclusion travel?

Small Group Tutorials