The Tutor Handbook · Volume 0209 · Series ID THB-0209
A practice can be implemented well in one sense and badly in another
A tuition programme introduces a new feedback routine.
After six weeks, the dashboard looks encouraging.
Eighty-eight per cent of observed lessons use the routine.
Tutors can explain the steps.
Learners usually complete the reattempt.
The programme could declare success.
Then the details arrive.
The routine reaches nearly every Mathematics group but only half the English groups.
Tutors report that it is easy to run in ordinary ninety-minute sessions but difficult when school examinations create extra paper-review demands.
Learners understand the routine and generally accept it.
Two tutors are quietly adding ten minutes of unpaid preparation to make it work.
The instructional steps are being followed, but the final fresh verification item is often replaced with a near-identical question because the material bank is thin.
The practice is therefore doing well on some implementation outcomes and poorly on others.
That is normal.
The mistake is to compress those outcomes into one number.
The Implementation-Outcome Interpretation Gate asks how a tuition programme should read several implementation outcomes together when they disagree.
The gate protects a simple principle:
Fidelity, reach, feasibility, acceptability and sustainability are different questions. They may interact, but they should not be averaged into a single “implementation score” that hides the reason the practice is or is not working.
The programme needs a pattern, not a grade.
Direct answer
Read implementation outcomes in layers.
First, ask whether the intended people are actually receiving the practice: reach.
Second, ask whether the load-bearing parts of the practice are being delivered: fidelity.
Third, ask whether ordinary tutors can carry it inside real time, materials and workflow: feasibility.
Fourth, ask whether tutors, learners and families experience the practice as reasonable enough to use: acceptability.
Fifth, ask whether the practice can survive staff turnover, fading launch support, ordinary timetable pressure and material change: sustainability.
Do not combine those answers arithmetically.
Instead ask:
- Which outcome is currently limiting the educational job?
- Is the weak outcome upstream of the others?
- Is the weakness general or concentrated in one subject, tutor group, learner group or condition?
- Did a recent adaptation improve one outcome by damaging another?
- Which response would improve the limiting outcome without deleting the practice’s active ingredient?
- What should be monitored again after the response?
Then decide whether to continue, adapt, narrow, support, pause or stop.
Implementation outcomes are a decision map.
They are not school report grades for a programme.
The coverage boundary
The Tutor Handbook already has dedicated owners for individual implementation questions.
The Implementation Fidelity Check asks whether the intended route was actually run.
The Implementation-Feasibility Gate asks whether the practice can fit ordinary tutor time, materials and learner variation.
The Implementation-Reach Gate asks whether eligible learners and tutors actually receive it.
The Implementation-Sustainability Gate asks whether it can survive beyond launch conditions.
The Implementation-Acceptability Gate asks how participants experience and judge it without confusing popularity with effectiveness.
The Implementation-Readiness Gate asks whether the programme is ready to launch.
The Implementation-Barrier Response Gate asks how to match a response to the barrier that is actually present.
This article owns the synthesis problem:
What does the programme do when the outcomes disagree?
That is a different reader job.
A practice can have high fidelity and low reach.
High acceptability and low feasibility.
High reach and weak fidelity.
Strong launch feasibility and weak sustainability.
The programme needs to interpret the configuration before acting.
AERO now treats implementation outcomes as a deliberate monitoring job
The Australian Education Research Organisation published Staying on track: Monitoring implementation outcomes on 15 September 2026. The practice guide is explicitly designed to help school implementation teams monitor, reflect on and make decisions using implementation outcomes. It organises practical monitoring around feasibility and acceptability, fidelity and reach, and sustainability.
That is current research-informed implementation guidance for schools. It is not a trial of this private-tuition interpretation gate.
AERO’s March 2026 implementation-stage resource also advises schools to use data on acceptability, reach, fidelity and sustainability as implementation proceeds.
The Education Endowment Foundation’s third-edition A School’s Guide to Implementation, published 24 April 2024, similarly frames implementation as a structured but flexible process shaped by context, people, infrastructure, engagement and reflection.
These sources give a strong discovery signal: serious implementation work needs multiple outcomes, not one binary “implemented/not implemented” judgement.
The Tutor Handbook question is how a tuition programme reads those outcomes together without manufacturing false precision.
Why a single implementation score is attractive
Leaders like summaries.
One score fits a dashboard.
One traffic light fits a meeting.
One percentage appears objective.
Suppose the programme assigns:
- reach: 90;
- fidelity: 80;
- feasibility: 60;
- acceptability: 85;
- sustainability: 55.
Average: 74.
What does 74 mean?
Almost nothing unless the programme has a defensible measurement model.
The low sustainability score may represent a serious structural risk.
The feasibility score may come from one difficult month.
The reach score may hide an excluded learner subgroup.
The fidelity score may be based on the wrong active ingredient.
The acceptability score may be a short survey dominated by people who remained in the programme.
Arithmetic creates a smooth number by destroying the distinctions that should guide action.
Use the outcomes as separate dimensions.
Start with the active ingredient
Before interpreting fidelity, define what must be preserved.
A feedback routine may have several visible steps, but only some are load-bearing.
For example:
- learner attempts before answer-giving help;
- tutor identifies one decision-relevant issue;
- feedback points toward a next action;
- learner acts on the feedback;
- a fresh attempt checks whether the change travels.
If tutors shorten the wording but preserve those elements, fidelity may remain adequate.
If they complete the official form perfectly but skip the learner action, visible compliance can be high while mechanism fidelity is low.
The Active-Ingredient Adaptation Gate owns this distinction in more depth.
Implementation-outcome interpretation begins here because every later conclusion depends on what the programme thinks it is implementing.
Reach can look high while the important people are missing
“Eighty per cent of sessions used the practice.”
That sounds strong.
Which twenty per cent did not?
If the missing sessions are random, the interpretation differs from a pattern in which:
- younger learners rarely receive the practice;
- English groups are excluded because materials do not fit;
- learners needing accessibility adaptations are missed;
- new tutors avoid the routine;
- late sessions lose it;
- online sessions do not support it.
Reach should be read by eligibility and condition, not only by total count.
The reach question is:
Did the intended people receive a real opportunity to experience the practice?
High average reach can coexist with an important equity or design failure.
Fidelity can be high because the practice was narrowed too much
A programme may improve fidelity by reducing eligibility.
Only the easiest groups use the practice.
Among those groups, implementation looks excellent.
That is why reach and fidelity must be read together.
High fidelity + low reach can mean:
- the practice works only in a narrow condition;
- the programme has not trained everyone;
- difficult cases are being excluded;
- materials do not fit some subjects;
- the eligibility rule was intentionally narrowed and should be declared.
The correct response depends on which explanation fits.
Do not celebrate fidelity without asking who was left outside it.
Reach can be high while fidelity collapses
The opposite configuration is common.
Every tutor says they use the routine.
The routine appears in almost every lesson.
But tutors have simplified it until its educational job is gone.
A “fresh verification question” becomes another worked example.
A “private first response” becomes a whole-group discussion before anyone commits.
A “feedback reattempt” becomes the tutor correcting the original answer.
The practice has spread.
The mechanism has not.
High reach + low fidelity suggests that scaling happened faster than quality.
The response may be professional learning, material redesign, coaching or a narrower definition of the active ingredient.
It is not solved by increasing reach further.
Feasibility can explain later fidelity drift
A practice may launch with strong fidelity.
Tutors are motivated.
Materials are prepared.
Coaches are present.
After two months, small adaptations appear.
One step is skipped.
Then another.
A leader may interpret this as declining commitment.
But low feasibility can produce fidelity drift.
If the practice takes too long, duplicates documentation or depends on scarce materials, tutors will adapt to survive ordinary work.
The Implementation-Feasibility Gate already owns the feasibility differential.
The synthesis point is that fidelity data should be read beside feasibility evidence.
Repeated drift toward the same shortcut is often information about the design.
Acceptability is not a popularity contest
Tutors may dislike a practice because it is unfamiliar.
Learners may dislike it because it demands more effort.
Parents may prefer visible worksheets to a diagnostic conversation.
Those reactions do not prove the practice is educationally wrong.
But acceptability still matters.
A practice that participants experience as humiliating, confusing, disproportionate or misaligned with their values may never be implemented reliably even if the mechanism is sound.
High acceptability + weak learning evidence is also possible.
People can enjoy a practice that does little.
Implementation outcomes do not replace outcome evaluation.
They tell us about the embedding of the practice.
The programme still needs separate evidence about learner progress and educational value.
Sustainability is not “did we keep doing it?”
A practice can persist because one champion carries hidden extra effort.
That is not robust sustainability.
A tutor can spend Sunday night preparing materials no one else knows how to create.
A programme lead can personally check every implementation record.
A single coach can repair every deviation.
The practice appears stable until one person leaves.
Sustainability asks whether ordinary programme structures can maintain the active ingredient.
It is therefore partly a future-facing outcome.
The strongest launch month cannot answer it.
Composite case: the practice everyone likes but nobody can sustain
This case is fictional and constructed for teaching.
A programme introduces a “two-minute learner reflection” at the end of selected lessons.
Tutors like it.
Learners like it.
Parents appreciate the clarity.
Reach is 92 per cent.
Fidelity looks strong: learners state what changed and what they need to do next.
After eight weeks, tutors report that the reflection creates a five-minute overrun because the programme also requires written handover notes.
Some tutors begin asking the reflection question while packing materials.
Others write the learner’s reflection for them.
The practice is still present, but the active ingredient is weakening.
Interpretation:
- acceptability: high;
- reach: high;
- early fidelity: high;
- feasibility: falling under ordinary end-of-session workflow;
- sustainability: threatened.
A weak response would be:
“Remind tutors to follow the protocol.”
A stronger response investigates whether duplicate handover work can be reduced or integrated.
The practice does not need more popularity.
It needs a sustainable workflow.
Interpret configurations, not isolated numbers
Several recurring patterns are useful.
High reach + high fidelity + low feasibility
The practice is being carried through effort that may not last.
Investigate hidden time, preparation, cognitive load, technology or administrative burden before sustainability deteriorates.
High feasibility + high acceptability + low fidelity
People can do it and are willing to do it, but the practice is being executed incorrectly.
Training, examples, coaching or a clearer active ingredient may be needed.
High fidelity + low acceptability
The practice is being complied with despite negative experience.
Investigate the source of the objection. Do not assume resistance is irrational, and do not assume dislike proves the practice should stop.
High acceptability + low reach
People who receive the practice like it, but many eligible people never receive it.
Investigate allocation, access, scheduling, materials or eligibility decisions.
High reach + low fidelity + high acceptability
The practice may have spread as a pleasant surface routine while losing its educational mechanism.
Strong launch outcomes + weak sustainability
Implementation may be dependent on temporary support.
The programme should test ordinary operation after launch resources taper.
These patterns are not validated diagnostic signatures.
They are structured prompts for interpretation.
Look for the upstream outcome
Outcomes can constrain one another.
Low feasibility can cause fidelity drift.
Low acceptability can reduce reach when tutors quietly avoid the practice.
Poor leadership clarity can create variable fidelity.
Weak materials can reduce both feasibility and reach.
The programme should ask which outcome is upstream enough that improving it could unlock the others.
Do not respond to every red flag independently.
Otherwise one problem generates five implementation initiatives.
The Implementation-Barrier Response Gate owns the next step once the likely barrier is identified.
Do not let one strong outcome compensate for a weak load-bearing one
This is the implementation version of the aggregate-score problem.
A practice cannot become “good enough overall” because high acceptability compensates numerically for near-zero reach.
If the active ingredient is absent, high feasibility cannot rescue fidelity.
If a key learner group is excluded, strong average reach elsewhere does not make the exclusion vanish.
Some outcomes are gating conditions for particular decisions.
Before scaling, the programme may require minimum evidence of feasibility and fidelity.
Before declaring broad access, it may require reach across intended groups.
Before withdrawing launch support, it may require sustainability evidence.
The thresholds should be tied to decisions, not converted into one composite percentage.
Segment before you generalise
Programme-wide averages can hide context.
Read outcomes by:
- subject;
- age or level;
- tutor experience;
- delivery mode;
- group composition;
- timetable condition;
- material version;
- learner access condition;
- launch phase.
Do not over-segment tiny samples into noise.
Use segmentation when there is a plausible implementation reason and enough evidence to guide action.
A practice may be sustainable in Mathematics but not English because the material preparation differs.
That calls for differentiated implementation support, not a programme-wide verdict.
Outcome measurement itself has quality conditions
Reach counts require a clear denominator.
Fidelity requires an active-ingredient definition.
Feasibility requires evidence from ordinary conditions, not only launch week.
Acceptability should include the people whose experience matters, not only the most enthusiastic implementers.
Sustainability cannot be measured convincingly on day three.
Every implementation outcome has a measurement problem.
AERO’s 15 September 2026 guide is useful precisely because it treats monitoring as a deliberate design task.
A dashboard does not become valid because it is labelled “implementation”.
Distinguish missing evidence from poor outcome
“No data on sustainability” does not mean sustainability is low.
“No family feedback” does not mean acceptability is high.
“Only two observations” does not mean fidelity is 50 per cent in a stable sense.
Programmes should keep an unknown state.
Unknown is uncomfortable.
It is also more honest than filling gaps with assumptions.
The Missing-Response Differential teaches the same discipline at learner level: absence of a response is not automatically one kind of failure.
Implementation evidence deserves the same restraint.
Separate implementation success from learner outcome success
A practice can be implemented exactly as intended and still fail to improve the target learning outcome.
That tells us something important.
The implementation explanation becomes less plausible.
A practice can also appear ineffective when implementation was weak.
Then the outcome evaluation is harder to interpret.
This is why implementation outcomes and learner outcomes should be read together but not merged.
Implementation asks:
“Did the programme deliver the intended practice under workable conditions?”
Learner outcome evaluation asks:
“Did the target educational state improve, and what can we responsibly infer about why?”
The Causal-Attribution Gap remains the owner for claims about causation.
Composite case: perfect compliance, wrong practice
This case is fictional.
A tuition centre adopts a “three-question retrieval opener”.
Tutors deliver it in 98 per cent of sessions.
The questions appear at the right time.
Records are complete.
Learners participate.
Fidelity to the visible procedure is excellent.
A review finds that many questions are copied from the immediately previous worksheet. Learners recognise them easily, but the opener is not testing durable retrieval or useful prerequisite access.
The programme has implemented the form faithfully and the job poorly.
This exposes a deeper issue.
Fidelity should not be defined only by visible steps.
It must refer to the active ingredient.
Otherwise high fidelity can become bureaucratic correctness.
Use implementation outcomes to choose the next question, not to produce a verdict
After a monitoring cycle, a useful meeting should end with a small number of decisions.
Example:
“Reach and acceptability are strong. Fidelity is uneven in English because the current examples do not fit extended-response work. Feasibility is acceptable elsewhere. We will redesign the English examples, coach two representative tutors, then recheck English fidelity and feasibility in four weeks.”
That is actionable.
Compare:
“Implementation score: 82%. Amber.”
The second statement looks concise.
It tells the programme almost nothing about what to do.
Monitoring frequency should match the outcome
Reach can change quickly.
Fidelity may need periodic observation.
Feasibility can shift during examination periods.
Acceptability may change after people gain experience.
Sustainability requires time.
Do not measure every outcome every week.
A monitoring system can become an implementation barrier itself.
The Evidence-Capture Burden Gate applies here.
Collect evidence when it can change a decision.
Programmes should expect trade-offs
An adaptation that improves feasibility may reduce fidelity.
A change that improves acceptability may narrow the practice too far.
Increasing reach quickly may stretch coaching capacity and weaken fidelity.
Tighter fidelity monitoring may reduce tutor autonomy and acceptability.
No implementation system eliminates trade-offs.
The programme should make them visible.
Ask:
“What did we gain?”
“What might we have weakened?”
“What evidence will tell us?”
This is more mature than treating every change as a pure improvement.
Do not punish the messenger outcome
Suppose acceptability data reveal that tutors find a practice unworkable.
The programme responds by emphasising the importance of positive attitudes.
Now the next survey improves because tutors stop reporting problems.
The implementation did not improve.
The measurement environment changed.
The same can happen with fidelity observations if tutors stage the observed lesson.
Outcome monitoring needs enough trust that bad news remains reportable.
The Tutor-Observation Sample Gate is relevant here.
A practical interpretation table
For each outcome, record five fields:
Observed state
What happened?
Evidence quality
How representative and comparable is the evidence?
Condition
Where, when and for whom was it observed?
Decision implication
What decision could this change?
Next discriminating check
What evidence would reduce the remaining uncertainty?
This is enough for most small tuition programmes.
No composite score is required.
A practical programme review
At a review meeting:
- Re-state the practice and active ingredient.
- Review reach.
- Review fidelity.
- Review feasibility.
- Review acceptability.
- Review sustainability at a time horizon that makes sense.
- Inspect important subgroup or condition differences.
- Note unknowns.
- Identify the current limiting outcome.
- Choose one or two responses.
- Pre-state what improvement should look like.
- Recheck the affected outcomes after the response.
- Keep learner outcomes separate but visible.
The meeting should produce a smaller action set than the evidence set.
Failure modes
The implementation average. Unlike outcomes are averaged into one number.
The compensation trap. A strong outcome hides a weak load-bearing outcome.
The fidelity-only view. The programme checks whether people follow the procedure but ignores reach, feasibility, acceptability or sustainability.
The popularity view. High acceptability is treated as proof of educational effectiveness.
The launch-week illusion. Early feasibility under extra support is treated as sustainability.
The missing-equals-good failure. Lack of complaints or missing data is treated as positive evidence.
The reach denominator error. Reach is calculated against everyone rather than the intended eligible population, or the eligible population is never defined.
The compliance fidelity error. Form completion is mistaken for preservation of the active ingredient.
The monitoring burden failure. Data collection becomes heavy enough to make implementation less feasible.
The every-red-flag initiative. Each weak outcome produces a separate intervention instead of looking for an upstream barrier.
The learner-outcome merge. Implementation measures and learner progress are blended into one success claim.
Evidence boundaries
AERO’s Staying on track: Monitoring implementation outcomes, published and updated 15 September 2026, is the most direct current authority used here. It treats feasibility, acceptability, fidelity, reach and sustainability as distinct implementation outcomes to monitor within a structured implementation process.
EEF’s A School’s Guide to Implementation, third edition published 24 April 2024, supports attention to behaviours, contextual factors and a structured but flexible Explore–Prepare–Deliver–Sustain process.
The implementation-outcome configurations in this article are not validated scoring rules.
They are interpretation aids.
No universal threshold is proposed for “good fidelity”, “enough reach” or “acceptable sustainability”. The appropriate threshold depends on the practice, the educational stakes, the evidence quality, the cost of error and the decision being made.
The central claim is modest:
Distinct implementation outcomes should remain distinct long enough to tell the programme what is happening and what decision follows.
The end state
A mature tuition programme can look at a mixed implementation picture without rushing to one verdict.
It can say:
“The practice is reaching the right learners, but the active ingredient is drifting.”
“Tutors are implementing it accurately, but the preparation burden is not sustainable.”
“The practice is feasible and liked, but one learner group is not receiving it.”
“Implementation is strong; learner outcomes still have not improved enough, so we need to examine the practice itself rather than blaming fidelity.”
Those are useful conclusions because they preserve the reason behind the judgement.
The Implementation-Outcome Interpretation Gate is therefore not a dashboard trick.
It is a discipline of keeping different questions separate until their pattern becomes actionable.
Measure the dimensions.
Read the configuration.
Find the limiting outcome.
Then change the smallest thing that could improve the educational system without hiding the evidence inside an average.
Add confidence to the interpretation, not decimals to the score
Implementation teams often respond to uncertainty by reporting more precise percentages.
“Fidelity = 83.7%.”
The decimal can create confidence the evidence does not deserve.
A more useful review attaches an evidence-quality statement.
For example:
“Fidelity appears high in Mathematics based on six ordinary observations across three tutors; English evidence is still thin.”
“Reach is well measured because session records cover the full eligible population.”
“Acceptability evidence is provisional because only tutors, not learners or families, have been sampled.”
“Sustainability cannot yet be judged because launch coaching is still unusually intensive.”
This language makes uncertainty operational.
It tells the programme where another measurement could be valuable.
It also prevents a clean-looking dashboard from giving weak evidence more authority than it has earned.
The Learning Claim is written at learner level, but the same epistemic discipline applies to programme claims: say only what the evidence can support.
Compare outcomes before and after a response under comparable conditions
Suppose feasibility is weak during examination season.
The programme simplifies documentation and then checks feasibility in the quiet December period.
Tutors report that the practice is now easy to run.
The programme may have improved the workflow.
It may also be comparing two different operating conditions.
Implementation monitoring needs condition awareness.
When possible, compare like with like:
- ordinary term against ordinary term;
- similar session length;
- similar learner load;
- the same material version;
- the same delivery mode;
- comparable tutor experience.
Perfect comparability is rarely possible.
The solution is not to wait forever.
It is to keep condition changes visible so the programme does not attribute every improvement to the latest intervention.
This is especially important for sustainability, where temporary launch support, novelty and unusually high attention can make the early months look better than ordinary operation.
A weak outcome can be acceptable when it is consciously bounded
Not every implementation outcome must be maximised.
A programme may deliberately accept lower reach because the practice is intended only for a narrow eligible group.
A demanding diagnostic routine may have moderate acceptability because it is used rarely for consequential cases where the information value justifies the effort.
A practice may have lower fidelity in one surface feature because tutors adapt examples while preserving the active ingredient.
The programme should distinguish uncontrolled weakness from declared scope.
If reach is 40 per cent because only 40 per cent of learners are eligible, that may be correct implementation.
If reach is 40 per cent because half the eligible tutors cannot access the material, that is a barrier.
The denominator and eligibility rule are part of the interpretation.
Use an outcome narrative when the pattern is complex
For a small programme, a paragraph can be more useful than a scorecard.
Example:
“The fresh-verification routine now reaches almost all eligible Mathematics and Science groups. Observed fidelity is strong when tutors use the prepared question bank, but English tutors often substitute near-identical questions, so transfer verification is weaker there. Tutors report the routine is feasible in ordinary weeks but difficult during intensive school-paper review. Learner acceptability is generally positive. Sustainability is not yet established because the programme lead still prepares many fresh items centrally. The next action is therefore an English material redesign and a sustainability test, not another whole-programme training session.”
That paragraph preserves relationships among outcomes.
It can be wrong.
But it is falsifiable.
The next monitoring cycle can test whether English fidelity improves and whether central preparation dependence falls.
A single “82% implemented” score cannot do that work.
Stop monitoring an outcome when it no longer changes the decision
Implementation monitoring should taper.
Once reach is stable and complete under ordinary conditions, the programme may sample it lightly rather than count every eligible session.
Once feasibility problems are resolved, frequent surveys may add little.
Sustainability may need periodic checks after staff or material changes rather than constant measurement.
This is the programme-level equivalent of fading support.
Monitoring is valuable because it informs decisions.
When a measure no longer changes a plausible decision, its collection burden should be questioned.
That protects tutor time and keeps the implementation system focused on information value rather than data accumulation.
Do not let targets become ceilings
A programme may set a monitoring target such as “at least 80 per cent of eligible sessions use the routine”.
That can be useful for planning.
It can also become a ceiling on interpretation.
If reach rises to 82 per cent, the programme may stop asking who remains in the missing 18 per cent.
If fidelity passes a checklist threshold, the programme may stop asking whether the active ingredient is actually present.
Targets should trigger questions, not end them.
Use them as precommitted decision aids:
“Below this level, we investigate immediately.”
“Above this level, we still inspect important exclusions, drift and subgroup patterns.”
The same applies to acceptability and feasibility thresholds.
A number can reduce arbitrary judgement.
It should not replace professional interpretation.
This matters especially in small programmes where one or two groups can materially change a percentage. Report the count and context alongside the rate when the denominator is small.
A programme that understands its implementation outcomes should be able to explain the pattern in words even if the dashboard disappeared.
