PSLE Science Reality Lab Vol No.046 | “Both Groups Improved” — Did the New Product Cause the Improvement?

PSLE-SCI-REALITY-0046

Wait, What? The product group improved — but so did the group that never used the product.

A poster shows a dramatic result. Ten containers used a new insulating wrap. Their average water temperature after twenty minutes was 4°C higher than it had been in an earlier test. The headline is confident: “New wrap improves heat retention by 4°C.”

That sounds persuasive until you notice a second line in the report. Ten similar containers that did not use the new wrap also improved by 3°C between the earlier and later test days.

Now the scientific question changes. The important observation is no longer simply, “Did the product group improve?” It becomes, “Did the product group improve more than a suitable comparison group under comparable conditions?”

This is one of the most useful transfers of PSLE Science reasoning into the real world. Before-and-after evidence is often presented as though time itself were a fair test. But many things can change between “before” and “after”: room temperature, apparatus, operator skill, sample quality, starting conditions, measurement timing and ordinary variation. A comparison group helps us ask how much of the change may have happened anyway.

Quick Answer

If both groups improve, do not attribute the whole improvement in the product group to the product. First compare the amount of change in each group. Then check whether the groups started comparably and experienced the same background conditions. The tested factor may explain the extra change beyond what also happened in the comparison group, but even that conclusion depends on the fairness of the comparison.

Reality Lab habit: A change inside one group is an observation. A causal claim needs a comparison that helps rule out other reasons for that change.

The Exact Learner Job

This guide teaches one job: how to evaluate a real-world before-and-after improvement claim when both a tested group and a comparison group change. It applies PSLE Science ideas about fair comparisons, variables and evidence. It does not replace the existing eduKate guides that own those micro-skills. Here, the challenge is to use them on a persuasive communication object: a claim that points to improvement in one group and quietly forgets to ask what happened elsewhere.

Reality Lab Case: The Cooling Sleeve Test

Imagine a fictional company testing a reusable sleeve designed to keep a bottle cool. On Monday, it measures how much the water temperature rises over thirty minutes. On Friday, it repeats the test after adding its new sleeve to one set of bottles. A second set receives no sleeve.

Group	Monday average temperature rise	Friday average temperature rise	Change
New sleeve	8°C	4°C	4°C less warming
No sleeve	8°C	7°C	1°C less warming

If we looked only at the sleeve group, we might say, “The sleeve reduced warming by 4°C.” But the no-sleeve group also warmed 1°C less on Friday. Perhaps Friday’s room was cooler. Perhaps the bottles started at a slightly different temperature. Perhaps the lamp output changed. Whatever the reason, the comparison group tells us that some improvement occurred without the sleeve.

The more careful result is therefore not “4°C caused by the sleeve.” The observed difference in changes is:

4°C improvement in sleeve group − 1°C improvement in comparison group = 3°C extra improvement associated with the sleeve under these test conditions.

Even then, “associated with” is deliberately cautious. We still need to check whether the two groups were genuinely comparable.

Three Different Questions That Students Often Collapse Into One

Question 1: Did the tested group change?

This is a within-group question. Monday and Friday are compared for the sleeve group. It tells us what happened to that group over time.

Question 2: Did the comparison group also change?

This asks whether background conditions, practice, time or other shared factors may have changed the outcome even without the tested product.

Question 3: Did the tested group change more?

This is the comparison that begins to address the product claim. It does not automatically prove cause, but it is far more informative than pointing at a before-and-after change in only one group.

Observed, Claimed, Inferred

Layer	Cooling-sleeve example
Observed	The sleeve group warmed 4°C less on Friday than Monday. The no-sleeve group warmed 1°C less.
Claimed	“The new sleeve improves cooling performance.”
Inferred	The sleeve itself caused all or part of the extra improvement.

The inference may be reasonable if the comparison is well designed. It becomes weak if the groups differ in other important ways.

The Baseline Check: Did the Groups Start From Comparable Places?

Suppose the sleeve group began with bottles at 5°C while the comparison group began at 12°C. Or perhaps one group used smaller bottles. Or one group was placed nearer the air-conditioning vent. Then the groups do not differ only in the sleeve.

A comparison group is useful only when the comparison answers the intended question. Ask:

Were the objects or specimens comparable?
Were the starting values similar enough for the intended comparison?
Was the measured outcome defined the same way?
Were timing, apparatus and environmental conditions comparable?
Was the tested factor the main planned difference?

These are familiar PSLE Science questions. The real-world twist is that an advertisement may show the attractive result without showing whether these conditions were protected.

Why “Improved by 40%” Can Still Be the Wrong Comparison

Imagine a fictional plant tonic. Plants receiving the tonic grow from an average height of 10 cm to 14 cm over two weeks. The advertisement says, “40% growth improvement.”

But untreated plants grow from 10 cm to 13 cm during the same two weeks. Plants grow naturally. The scientifically relevant question is not whether tonic-treated plants grew. It is whether their growth differed from suitable untreated plants under comparable conditions.

The tonic group gained 4 cm. The comparison group gained 3 cm. The extra observed gain is 1 cm under these conditions. That is a very different communication object from “40% improvement”.

Common Background Changes

Both groups can change for many reasons that have nothing to do with the tested factor:

temperature or humidity changed between test days;
the operator became more skilled;
all specimens matured or aged;
all equipment was cleaned or recalibrated;
a measurement schedule changed;
the starting material came from a different batch;
ordinary variation produced a better or worse set of outcomes.

A comparison group does not magically remove these possibilities. It gives you evidence about how much change occurred without the tested factor.

What Would Strengthen the Product Claim?

Both groups start under comparable conditions.
The tested factor is applied consistently to only the intended group.
Other relevant conditions are kept similar.
The same measurement method is used for both groups.
The result repeats across enough independent trials or specimens to show it is not one lucky outcome.
The extra improvement in the tested group remains when the test is repeated under changed but relevant conditions.

What Would Weaken It?

Only the product group is measured before and after.
The comparison group starts from a different condition.
The environment changes differently for the two groups.
The product group receives extra handling, time or attention that the comparison group does not.
The headline reports the product group’s improvement but hides similar improvement in the comparison group.
The result disappears when the test is repeated fairly.

Worked Case 1: The Faster-Drying Cloth

A new cloth treatment is claimed to make wet cloth dry faster. On Day 1, treated cloth takes 50 minutes to dry. On Day 2, after a formula change, treated cloth takes 35 minutes. The advertisement says, “15 minutes faster.” Untreated cloth takes 48 minutes on Day 1 and 36 minutes on Day 2.

Reasoning: The treated cloth improved by 15 minutes, but the untreated cloth improved by 12 minutes. Something about Day 2 may have made drying faster generally. The extra observed improvement associated with the treatment is only 3 minutes in this simple comparison, and we still need to check humidity, airflow, cloth size, amount of water and measurement consistency.

Worked Case 2: The Cleaner Filter

A filter company reports that water passing through Filter X has 30 fewer visible particles after a cleaning upgrade. A reference set-up without Filter X shows 25 fewer particles on the same day because the incoming water itself was cleaner.

Reasoning: The headline “30 fewer particles because of the upgraded filter” overstates what the evidence shows. The comparison suggests that much of the change may have come from cleaner incoming water. The additional difference between groups is much smaller.

Worked Case 3: When the Comparison Group Is Not Fair

A cooling device is tested on aluminium containers. The comparison group uses glass containers. Both groups begin at the same temperature. Is that enough?

No. Container material can affect heat transfer. Starting temperature is only one condition. A fair comparison requires attention to all conditions that could materially affect the outcome.

PSLE-Style Transfer Case

Two groups of similar plants are measured at the start and after ten days. Group A receives Solution X. Group B receives only water. Group A grows from 12 cm to 18 cm. Group B grows from 12 cm to 16 cm.

A student writes: “Solution X caused the plants to grow 6 cm.”

A stronger answer is: Group A grew 6 cm while Group B grew 4 cm over the same period. The evidence therefore supports an extra 2 cm of average growth in Group A under the tested conditions, assuming the groups were otherwise comparable. It does not support attributing all 6 cm to Solution X because the comparison plants also grew.

Notice the structure: observe both groups, compare changes, then limit the claim.

Tempting Reasoning That Fails

“The product group improved, therefore the product worked.” Improvement can occur for other reasons.
“Both groups improved, therefore the product did nothing.” The product group may still have improved more.
“The groups started at the same average, therefore the comparison is fair.” Other relevant conditions may differ.
“The comparison group is called a control, therefore it is automatically valid.” Labels do not replace method checks.
“A bigger change means stronger evidence.” Evidence strength also depends on fairness, measurement quality and repeatability.

How Far Can the Conclusion Travel?

If the test used one bottle type, one room temperature, one duration and one batch of material, the evidence is strongest for those conditions. It may justify trying the product under other relevant conditions. It does not automatically prove the same effect for every container, climate, duration or user.

PSLE Science often rewards disciplined scope: say what the evidence supports, not what you hope it supports.

Practice: Compare the Changes

1. A treated surface lasts 18 days before visible wear, up from 12 days. An untreated surface lasts 16 days, up from 11 days. What is the extra observed improvement in the treated group?

Answer: Treated improvement = 6 days. Untreated improvement = 5 days. Extra observed improvement = 1 day. The evidence does not support saying that the treatment caused all 6 additional days.

2. Both groups improve by exactly the same amount. Does that prove the tested factor has no effect?

Answer: It weakens the claim that the tested factor produced extra improvement in this comparison. It does not prove that the factor can never have an effect under any condition. The conclusion stays inside the tested set-up.

3. The tested group improves more, but it also receives twice as much light. What must you say?

Answer: The evidence cannot isolate the tested product because light is another changed condition that could affect the outcome.

Delayed Independent Return

Tomorrow, find any “before versus after” scientific or product claim. Do not decide whether it is good or bad. Ask three questions only: What changed in the tested group? What changed without the tested factor? Are those groups genuinely comparable?

If the communication gives you no comparison at all, you have learned something important about the evidence boundary.

Route to the Canonical PSLE Science Skills

Teaching Guide for Parents and Tutors

When a child sees a dramatic before-and-after improvement, ask them not for a verdict but for a missing comparison: “What would we need to observe to know how much change might have happened anyway?”

Use simple paired stories. In one, the product group improves and the comparison group does not. In another, both improve equally. In a third, both improve but the product group improves more. Ask the learner to change the conclusion each time. This teaches that conclusions are not memorised phrases; they are responses to the evidence pattern.

The diagnostic weak link is often the jump from change to cause. Once the learner can describe both groups before explaining why they differ, causal reasoning becomes much more disciplined.

Authoritative Sources

The Quiet Return

Science does not ask only whether something changed. It asks what changed relative to what would otherwise have happened.

That extra comparison is often the difference between a persuasive story and a scientific claim.

Small Group Tutorials