The Tutor Handbook · Volume 0177 · Series ID THB-0177
Series route: The Tutor Handbook — Complete Series Index.
A tuition programme begins the term with one hundred learners. At the final review, seventy-eight have complete end-of-term results. Their average improvement looks strong.
What happened to the other twenty-two?
Some moved school. Some changed schedule. Some stopped tuition because the original problem improved. Some stopped because the fit was poor. Some missed the final assessment. Some families became too busy to return the work. Some learners with the greatest difficulties may also have been the hardest to retain.
If the programme reports only the seventy-eight complete cases, the arithmetic may be accurate and the interpretation may still be incomplete.
The Attrition Evidence Gap is the uncertainty created when learners who started a tutoring programme are absent from later outcome data, especially when the reasons for being missing may also be related to learning, attendance, fit, difficulty, support or programme experience.
This is not merely a research-method problem. It is a practical programme-governance problem. Tuition centres make decisions about materials, tutors, grouping, dosage and parent communication from the data they manage to collect. If the missing learners are systematically different from the learners who remain, the apparent result can become more flattering—or sometimes less flattering—than the full reality.
Quick Answer
Do not treat final outcomes from retained learners as though they automatically describe everyone who enrolled. Always report the denominator: how many learners started, how many had usable follow-up data, and what is known about the missing group. Separate ordinary missingness from possible selection. Investigate patterns in discontinuation without inventing motives. Use attendance and exit information as context, not as substitute achievement data. When attrition is substantial or plausibly related to learner difficulty, make the programme claim smaller.
At the individual tutoring level, missing evidence means uncertainty. At programme level, patterned missing evidence can distort the story.
1. What This Volume Owns
This volume owns the evidence problem created by incomplete follow-up. It does not replace The Attendance Differential, which asks why one learner misses sessions. It does not replace the Enrolment–Engagement Boundary, which asks how to work with a learner who did not personally choose tuition. And it does not replace formal statistical treatment of missing data.
The distinct question is: when the programme’s final evidence does not include everyone who began, what can the programme honestly conclude?
2. Why the Denominator Is Part of the Result
“Seventy-eight learners improved by an average of X” means something different if seventy-eight of seventy-eight learners were measured than if seventy-eight of one hundred and fifty were measured.
The outcome number is not enough. Readers need to know who had the opportunity to appear in it. This is the denominator problem. When programmes publish percentages, gains or completion rates without the starting count and follow-up count, the reader cannot see how much evidence disappeared between entry and outcome.
A tutor does not need to become a statistician to practise this discipline. Write three numbers before interpreting a cohort: number enrolled or eligible for the analysis, number with baseline evidence, and number with relevant follow-up evidence. Then ask what happened in between.
3. Missing at Random Is a Dangerous Assumption to Make Casually
Sometimes data are missing for reasons unrelated to learning: a family moves abroad, a school timetable changes, an assessment file is corrupted, or a learner is absent on the only collection day for a reason that has nothing to do with tuition.
But some missingness can be related to the same conditions that affect outcomes. Learners who are struggling may become less willing to attend. Families dissatisfied with progress may leave. Learners with difficult schedules may receive lower dosage and also be harder to assess at the end. A programme that requires high family responsiveness for final testing may disproportionately lose evidence from households under greater time pressure.
The tutor does not need to prove the missingness mechanism before becoming cautious. Plausibility is enough to reduce confidence in a clean retained-sample story.
4. Research Calls This Attrition for a Reason
The What Works Clearinghouse uses the term attrition for loss of participants from the analysed sample after they were initially included. Its standards care about attrition because missing outcomes can undermine comparability and bias estimates, particularly when loss differs between groups or is related to outcomes.
A private tuition programme is not automatically conducting a WWC-style impact study. It should not borrow formal ratings it has not earned. But the underlying reasoning travels well: when the people missing from the final analysis may differ meaningfully from those retained, the complete-case result deserves a smaller claim.
The research standard is a useful warning against a very human habit—looking only at the data we still have because the missing data are inconvenient.
5. Attrition Can Make a Weak Programme Look Stronger
Imagine that learners who struggle most are also more likely to discontinue. The remaining cohort gradually contains a higher proportion of learners for whom the programme is working well, whose families can sustain the schedule, or whose school circumstances make participation easier.
If the centre reports final gains only for completers, those gains may overstate what a new enrollee should expect from the programme as offered to everyone who begins. The programme has not falsified any individual score. The selection happened through disappearance.
This is why a high completion rate and a strong outcome rate answer different questions. A programme should want to know both.
6. Attrition Can Also Make a Programme Look Weaker
The bias is not always flattering. Some learners leave because the problem that brought them to tuition has been resolved, because they no longer need the service, or because they move into a different support arrangement after improvement. If those learners lack a final outcome measure, the retained group may contain a higher concentration of learners with persistent needs.
Therefore, “attrition biases results upward” is too simple. Attrition creates uncertainty about the direction and size of the distortion unless the programme understands who is missing and why.
The honest response is not to assume the worst. It is to stop pretending the missing cases are invisible.
7. Constructed Case: Alicia Leaves After Improvement
This is a constructed case. Alicia joins for a narrow Mathematics repair. After eight weeks, the target error has fallen sharply on tuition and school work. The family decides routine tuition is no longer necessary. Alicia therefore has no end-of-term centre assessment three months later.
If the centre classifies her as a “dropout” and excludes her, it loses evidence of a successful bounded intervention. If it simply carries forward her earlier good score as though it were the final outcome, it creates a different distortion.
The correct record is more modest: exited after target improvement; no end-of-term programme outcome available. Her instructional story remains useful without pretending the missing measure exists.
8. Constructed Case: Beatrice Leaves Because the Route Is Not Working
Beatrice attends for six weeks. Her family reports frustration, the tutor struggles to establish a workable reading route, and attendance becomes irregular. The family stops tuition before the scheduled progress assessment.
Removing Beatrice from programme reporting would make the retained results look cleaner precisely because a difficult case disappeared. The programme should not invent a final score. But it should preserve the exit category and the evidence that existed before exit.
For quality improvement, this case may be more important than a routine completer. It points toward fit, implementation, communication or route-design questions that average gains cannot reveal.
9. Constructed Case: Ciara Has Missing Data, Not Attrition From Tuition
Ciara completes the programme, attends well and remains enrolled. She simply misses the final assessment because of an unrelated school commitment, and the centre never reschedules it.
Her outcome is missing, but she did not leave the programme. This distinction matters. “Attrition” can refer technically to absence from the final analytic sample, while operationally a centre may also track programme withdrawal. These are not identical events.
A clean data system separates participation status from outcome-data status. Otherwise, the centre may confuse retention success with measurement completeness.
10. Constructed Case: Denise Changes Tutor and Becomes Hard to Classify
Denise begins with one tutor, changes group after a timetable conflict and finishes with another. The second tutor collects a final assessment, but the route, materials and dosage changed substantially.
Denise is not missing. Yet her data raise a related programme question: what exactly does the outcome represent? The programme may include her in overall results while flagging the route change, or analyse continuity separately if that serves a legitimate quality question.
This is a reminder that complete data can still have interpretation problems. Attrition is one evidence gap, not the only one.
11. Exit Reasons Are Context, Not Verdicts
Centres should record exit reasons carefully without turning them into causal truth. “Schedule conflict”, “financial reason”, “moved school”, “family decision”, “goal met”, “fit concern”, “unknown” and similar categories can support operational learning.
But the reason written in a form may be incomplete. Families may give the easiest socially acceptable explanation. A schedule conflict can coexist with low perceived value. A family may say “goal met” when several motives are involved. Tutors may infer “motivation” where the real issue is burden or access.
Treat exit reasons as reported context. Do not let them become retrospective diagnoses.
12. Attendance Data Do Not Replace Outcome Data
High attendance is valuable. Recent high-impact tutoring research has also examined attendance as an outcome in its own right. But a learner who attends thirty sessions does not therefore demonstrate the target academic capability.
Similarly, low attendance can explain why a learner received little opportunity to benefit, but it does not prove what the learner would have achieved with full attendance. Received dose, engagement and academic performance are related but distinct evidence streams.
When final academic outcomes are missing, attendance can describe implementation exposure. It cannot be promoted into a substitute learning score.
13. Minimum Programme Reporting Should Keep the Missing Visible
A useful internal report can remain simple. State how many learners were eligible or enrolled for the relevant cohort. State how many had usable baseline data. State how many had usable follow-up data. State how many remained enrolled. Describe major known reasons for missing outcomes using broad categories. Then report the outcome among the measured group with language that keeps the denominator visible.
For example: “Among 78 learners with both baseline and end-of-term assessments, the average score increased by X. One hundred learners began the term; 22 lacked a comparable final assessment for mixed reasons including withdrawal, schedule changes and missed testing.”
That sentence is less marketable than “our students improved by X”. It is much more informative.
14. Do Not Solve Missing Data With Casual Imputation
A programme may be tempted to fill gaps: carry the last score forward, estimate from homework, substitute a school mark or predict a final result from earlier performance. These methods can make a spreadsheet complete while making the evidence less honest.
Formal research has developed methods for handling missing data, and those methods come with assumptions. The What Works Clearinghouse is deliberately cautious about imputed outcomes in designs where missingness could bias interpretation. A tuition centre should not borrow advanced statistical techniques without the expertise and design needed to defend them.
For most operational tutoring decisions, a visible null is better than a fabricated precision. Unknown is a legitimate data state.
15. The Attrition Evidence Card
- Starting cohort: Who was included at the beginning?
- Baseline completeness: Who had usable starting evidence?
- Follow-up completeness: Who had comparable later evidence?
- Retention: Who remained in the programme?
- Missingness: Which learners lack the outcome needed for this claim?
- Known reasons: What broad, directly reported reasons are available?
- Pattern: Is missingness concentrated by attendance, starting need, tutor, schedule, site or another relevant condition?
- Claim size: Does the final statement describe completers, measured learners or everyone who started?
- Quality response: What operational change might reduce preventable missingness next cycle?
16. Patterns Matter More Than Anecdotes
One learner leaving tells the programme little about attrition as a system property. Ten learners leaving for apparently unrelated reasons may also tell little. But repeated patterns deserve investigation.
If final outcomes are consistently missing among learners scheduled late in the evening, ask whether timing affects attendance and assessment completion. If one site has much lower follow-up completion, inspect local processes. If learners with the lowest baseline scores are disproportionately absent from final data, programme claims about improvement should become more cautious and the retention problem more urgent.
This is not permission to mine data until a story appears. Choose plausible questions connected to programme design, then examine whether the pattern is stable enough to act on.
17. Prevention Is Better Than Statistical Rescue
The cleanest way to reduce attrition uncertainty is to design programme operations that make follow-up easier and more consistent. Schedule progress checks as part of ordinary tuition rather than optional extra appointments. Collect small comparable measures periodically rather than relying on one distant final test. Make attendance support practical. Maintain communication with families before problems become exits. Keep assessment burden proportionate.
NSSA’s tutoring quality resources treat enrolment and retention, data use, dosage, student relationships and communication as connected implementation dimensions. That systems view is useful. Missing final data are often produced by programme design long before the spreadsheet is analysed.
Better evidence begins with better opportunities to remain in the evidence stream.
18. Attrition and Equity
Attrition can have an equity dimension when the conditions that make participation difficult are unevenly distributed. Transport, caregiving responsibilities, device access, school schedules, health appointments, family work patterns and financial pressure can affect who receives the intended tutoring dose and who appears in final data.
A tuition programme should not infer demographic causes from incomplete information or stereotype families. It should, however, notice when operational barriers repeatedly exclude some learners from full participation or measurement.
The right response is to investigate access conditions and programme design, not to label learners as less committed.
19. Parent Communication When a Cohort Result Has Missing Outcomes
If a programme shares cohort outcomes publicly or with parents, the missing-data boundary should survive the communication.
These results are for learners who had comparable starting and end-of-term assessments. Not every learner who enrolled had both measures, so the figures should not be read as the outcome for every family who began the programme. We track retention and missing follow-up separately because those cases also matter to programme quality.
This language is not a weakness. It signals that the programme understands the difference between evidence and presentation.
20. Research Foundation and Boundary
The What Works Clearinghouse Standards Briefs define attrition as loss of initially included participants from the final analysis and explain why attrition can threaten confidence in study findings. WWC guidance on quasi-experimental designs and missing data also emphasises that missing outcomes can create bias when data are not missing at random.
Those are research-design standards, not a turnkey formula for a private tutoring centre. This volume borrows the reasoning discipline: do not let unavailable outcomes disappear from the interpretation simply because they disappeared from the dataset.
NSSA’s current Tutoring Quality Standards and programme toolkit treat data use, dosage, enrolment and retention as implementation concerns. That supports the operational recommendation to connect outcome completeness with programme delivery rather than treating missing data as an analyst’s problem at the end.
21. Common Failure Modes
Completer-only triumph: reporting strong outcomes without stating how many starters are missing. Dropout equals failure: assuming every exit reflects poor progress. Exit equals success: assuming families who stop did so because the problem was solved. Attendance substitution: using sessions attended as a learning outcome. Spreadsheet repair: filling missing outcomes with estimates that look more certain than the evidence. Unknown erased: forcing every learner into a neat exit category. Equity blindness: ignoring repeated access patterns behind missingness. Attrition fatalism: treating missing data as unavoidable rather than improving collection and retention processes.
Each failure is a form of narrative completion: the programme fills a gap in the story because gaps feel uncomfortable. Evidence discipline allows the gap to remain visible while still improving the system around it.
22. The Independence Direction
Learners and parents also benefit from understanding that absence from a dataset is not absence from reality. A learner who misses a final test still has a learning state. A family who leaves a programme still has reasons and outcomes, even if those outcomes are not measured by the centre.
This matters because education increasingly produces dashboards, cohort summaries and programme statistics. A mature reader asks: who is included, who is missing, and could the missingness change the interpretation?
That habit travels far beyond tuition. It is one of the foundations of responsible evidence reading.
Final Compression
When learners disappear from final outcome data, do not make them disappear from the reasoning.
Keep the denominator visible. Separate programme withdrawal from missing measurement. Record known exit context without inventing motives. Inspect patterns. Reduce preventable missingness through better programme design. Make claims about the measured group when that is the group the evidence actually describes.
The cleanest-looking dataset is not always the most truthful one. Sometimes the blank spaces are part of the result.
That is the Attrition Evidence Gap.
That is Tutor Handbook Volume 0177.
