A student wants to prove that a new revision app improves examination results. They invite friends who already enjoy the app, give them an easy practice quiz afterwards and compare those scores with marks from a harder test completed weeks earlier. The numbers may look impressive. Yet the study was set up in a way that cannot separate the app’s effect from recruitment, different questions and other learning.
Another class measures whether changing the wing length of a paper helicopter alters how long it falls. One version uses thicker paper, one is dropped from a higher point and several inconvenient trials are removed. The pupils may have collected real data, but those data cannot cleanly answer which wing length caused the difference. The design problem began before the final graph.
Study design is the skill of planning a fair, meaningful and feasible investigation before its results are known. It connects a clear research question to appropriate participants or objects, measures, comparison conditions, procedures, records and evidence standards. A successful design does not guarantee a preferred outcome. It makes honest answers possible, including answers that contradict the original hypothesis.
Design also depends on purpose. Describing experiences through interviews is not the same job as estimating a population proportion, comparing mathematics methods or establishing an intervention’s causal effect. A small, carefully bounded classroom study can be suitable for learning without being presented as a national research trial.
This article is a practical planning companion to Research Question Refinement Skills, Sampling Frame Skills, Counterfactual Comparison Skills, Intervention Evaluation Skills and Evidence Sufficiency Skills. Those guides own individual technical decisions. The job here is the *prospective architecture that makes those decisions fit together*.
Before the Top 10: Choose the Evidence Before Seeing the Answer
Begin with the final question rather than a method you happen to like. “How many pupils used this service?” calls for a clearly defined count. “How did respondents experience the service?” may call for appropriately collected feedback. “Did the service cause better independent comprehension?” calls for a more demanding design that addresses the unobserved alternative outcome.
An elegant study plan must also be executable. If it needs private records, unsafe classroom experiments, unauthorised pupil interviews or an instrument the school does not have, it is not ready. Scope, permissions, accessibility and feasibility are part of research quality, not administrative decorations.
Before collecting observations, document the proposed participants, operational definitions, outcome measures, timing, comparator and analysis. Record material changes later instead of pretending the original design never changed. A written protocol helps distinguish planned analysis from explanations invented after seeing attractive results.
The ten headings are a Wintour House / eduKate editorial synthesis of practical study-design choices, not an official MOE/SEAB research rubric or a validated ten-factor psychological scale. Professional guidelines from the EEF, the US What Works Clearinghouse and CDC illustrate why research question, design, implementation and limitations must remain aligned.
QUESTION → POPULATION → VALID MEASURE → COMPARISON → PROCEDURE → DATA → WARRANTED RESULT
1. Write One Answerable Research Question and Decide Its Type
Study design begins by determining whether the intended answer is descriptive, comparative, causal, explanatory or evaluative. A question can combine these interests, but each requires different information. If the question keeps changing, the researcher may select results after seeing them instead of testing a defined uncertainty.
A fictional school might ask whether recorded entries into a new reading space changed during its first month. That can be studied with consistently defined entry logs. The stronger question of whether the room caused better reading comprehension needs an assessment and an appropriate comparison or other credible causal strategy. A survey of enjoyment answers a third question.
Write the target claim, population, period, main outcome and type of inference. Ask what evidence would justify the proposed verb: describe, compare, explain or caused. If the available design cannot answer the question, narrow it transparently before recruitment rather than saving the problem for the conclusion.
Worth learning because a clear research question tells every later design choice what it is supposed to accomplish.
2. Specify the Eligible Population and Recruitment Route
A study must say who or what could be included. The target population, sampling frame, selected units and final respondents may be different groups. Recruiting only the easiest participants can be suitable for a limited pilot, but not automatically for broad population claims.
Suppose an optional English workshop is intended for all Secondary 2 pupils, while the invitation is shared only through a reading club. Pupils outside the club cannot enter the study through that route. Their likely experiences cannot be recovered by collecting more responses from the same club.
Set inclusion criteria and a safe, permitted selection method. Keep a simple record of eligible, invited and participating units. If a convenience group is unavoidable, plan to report results about that group rather than casually generalising to every pupil. Protect student privacy and voluntary participation.
Worth learning because who can enter the investigation shapes the limits of every result it might produce.
3. Choose a Measure That Represents the Intended Outcome
A measurable variable is not necessarily a valid measure of the skill or change of interest. Attendance, satisfaction, completion of worksheets and independent competence describe different outcomes. A design that selects an easy proxy because it is available may fail its own learning question.
If a clinic aims to strengthen explanation of inference questions, the final measure should ask learners to select textual clues and justify an inference in a suitably unfamiliar passage. Counting the number of passages read or the number of pupils who attended cannot establish that the reasoning skill improved.
Define the main outcome before collecting it. Specify its scoring rules, time of observation, permitted help and relationship to the learning goal. Separate secondary measures, such as perceived usefulness, from the primary capability the study set out to evaluate.
Worth learning because a trustworthy method cannot rescue an outcome measure that answers the wrong question.
4. Establish a Fair Comparison Before the Intervention Begins
When an investigation asks whether a change made a difference, the design needs information about what would plausibly have happened without it. Simply comparing a pupil’s score before and after tutoring does not establish a treatment effect, because ordinary learning, practice and different assessments may contribute.
Two classes could be assessed on appropriately comparable comprehension tasks while one receives a clearly defined additional teaching strategy and the other continues ordinary instruction. Their starting performance and other learning opportunities still matter. Ethical random allocation may be possible in some studies, while other questions require a carefully defended observational comparison.
State the intended comparison condition, how groups are formed and which features could make them different. Do not present a nonrandom group as equivalent merely because it has the same school year. Record baseline information before any consequential part of the intervention occurs.
Worth learning because causal claims depend on credible alternatives rather than the attractiveness of a before-and-after graph.
5. Control or Document Other Factors That Could Explain Results
A fair design identifies influential variables beyond the planned exposure. In simple experiments, keep relevant conditions comparable while changing the intended variable. In human studies, prior ability, motivation, teaching time or recruitment may shape both participation and outcomes.
A paper helicopter experiment comparing wing lengths should avoid simultaneously changing paper thickness, drop height and folding method without documenting the differences. An optional revision-app study should consider whether pupils who use the app also differ in prior attainment or time available for study.
Create a short list of the most plausible common causes and procedural variations. Decide which can be held constant, measured or addressed through design. Do not assume that every listed factor is a formal confounder; identify its proposed causal role and timing.
Worth learning because a fair comparison must reveal which changes were intended and which alternative influences could remain.
6. Plan Timing, Repetition and Assessment Conditions
A measured change depends on when the baseline and follow-up occur, how often observations are taken and whether conditions are comparable. An immediate quiz following a model answer can reflect short-term accessibility rather than durable independent learning. One physical trial may reflect random variation.
A reading clinic could use an initial unfamiliar passage, four documented weeks of practice, a new comparable passage at the end and a later independent retrieval task where feasible. A Science class testing falling objects can record repeated safe drops using the same planned release and timing procedure.
Specify observation dates, relevant intervals, repetition and allowed help in the protocol. Choose a schedule that matches the question. If delayed retention is the desired outcome, a same-minute practice quiz alone is inadequate. If the task is merely descriptive, do not impose needless extra testing.
Worth learning because timing and testing conditions determine what a measured improvement can reasonably mean.
7. Protect Ethical Participation, Privacy and Learner Access
Even a technically interesting study can be inappropriate if it requires collecting sensitive details without permission, excludes learners unnecessarily or creates unsafe demands. Ethical design begins with what information is genuinely needed for the research question and whether the benefit justifies the burden.
A class may compare two teacher-approved revision formats using anonymised outcome summaries, where school rules permit. This does not authorise public publication of identifiable test histories, medical information or private family circumstances. An optional survey should make participation conditions clear and should not pressure pupils to reveal personal explanations for nonresponse.
Check required school approvals, informed participation, safety procedures and access needs before data collection. Store only necessary information under appropriate safeguards. If a question depends on private records that cannot be lawfully used, revise the question or seek an appropriate authorised data source.
Worth learning because ethically obtainable evidence is part of sound research design, not a hurdle to bypass after a plan is written.
8. Predefine Data Handling, Missingness and Analysis
A study can be distorted by decisions made after seeing results: dropping awkward trials, redefining success or treating blanks as zeros. Planning these rules in advance makes the work more transparent and reduces opportunities for selective interpretation.
If twenty pupils are intended to complete a post-test and only sixteen scores are recorded, the design should already distinguish a valid zero score from a missing result. If a paper helicopter collides with a desk, a pre-specified exclusion rule may be defensible, but selectively discarding slow or fast drops because they weaken the preferred conclusion is not.
Write down data definitions, eligible records, error checks, handling of missing information and the main planned comparison. Keep raw observations and document necessary protocol changes with reasons. More advanced statistical analysis requires appropriate expertise rather than arbitrary spreadsheet formulas.
Worth learning because the treatment of data should follow the research question instead of changing to favour whichever outcome appears.
9. Pilot the Procedure Before Claiming It Is Ready
A small pilot can reveal whether instructions are understandable, instruments usable, recruitment feasible and outcome measures appropriate. A pilot is not automatic proof that the intervention works; its principal purpose may be to improve the feasibility and quality of a subsequent investigation.
A teacher trials the comprehension scoring rubric on two sample answers and discovers that “effective explanation” is interpreted differently by different markers. Clarifying the criteria before the main data collection improves comparability. A physical experiment might reveal that the proposed fall time is too short for consistent hand timing.
Use a limited, suitable rehearsal to test procedure rather than a preferred hypothesis. Record problems, make justified revisions and specify which pilot data—if any—can be used in the eventual study. Do not report a feasibility pilot as a definitive causal evaluation.
Worth learning because a small test of the method can prevent a large study from collecting unusable evidence.
10. Match the Final Conclusion and Next Decision to the Actual Design
An honest design supports an honest conclusion. A descriptive survey may report what respondents said; an observational comparison may report an association; a well-conducted causal study may support a bounded effect estimate under its assumptions. None should be described as stronger merely because it is presented in a polished report.
If a fictional reading club reports fifteen of twenty respondents valuing its sessions, the observed figure is 75% of those respondents. It does not establish that 75% of all pupils prefer the club or that attendance caused higher examination marks. A next study could appropriately investigate independent reading outcomes or wider access.
Prepare a concise result plan in advance: intended question, observed evidence, relevant limitations and next decision. At the end compare the actual findings against the plan. If the evidence answers only a narrower question, say so. Knowing that limitation is often a genuine educational success.
Worth learning because a good study design leads to conclusions that follow from how information was collected, not how confidently the report was written.
The Top 10 Study Design Skills as One System
The sequence is RESEARCH QUESTION → TARGET POPULATION → VALID OUTCOME → COMPARISON → IMPORTANT OTHER FACTORS → TIMING → ETHICS → DATA RULES → PILOT → WARRANTED CONCLUSION. These are choices, not ten separate forms pupils must complete before a simple classroom observation.
A useful final test is whether another careful teacher could understand what the study intends to answer, reproduce its major procedures and identify which claims the design cannot support. If not, the plan needs refinement.
Study Design Is Not the Same as Evaluation After a Programme
Intervention Evaluation Skills covers the broader judgement of a programme’s need, reach, delivery, outcome, equity, costs and next action. Study design owns the earlier research architecture: who will be assessed, how the comparison will be made, what data will be collected and what will count as an informative result.
Descriptive projects, qualitative inquiries and experiments can all have designs. Not every valid study is an intervention evaluation, and not every classroom reflection needs a formal trial.
Worked Primary Science Design: Paper Helicopters
A Primary class asks whether wing length changes the fall time of a teacher-approved paper helicopter. Choose a defined pair of designs, use equal paper material and mass where practical, release from an agreed height and make repeat observations with the same timing method.
Write the prediction before the test, record every valid trial and note collisions or other exceptions consistently. The resulting observations may describe the two designs under the classroom conditions; the pupil should not claim a universal aerodynamic law from a few drops.
The important design skill is seeing why changing paper thickness and release height at the same time would make the comparison harder to interpret.
Worked Secondary Mathematics Design: Comparing Two Practice Methods
A fictional teacher wants to compare worked-example review with unaided problem-solving practice for a specific algebra skill. The study should define the participating group, baseline proficiency, amount of instructional time, the two activities and a common appropriately aligned outcome measure.
Suppose one group practices for thirty minutes using worked examples and another for thirty minutes using suitably matched problems. If pupils choose their preferred method, prior motivation or knowledge may affect both the choice and result. A fair design must acknowledge this selection or use a defensible allocation strategy where appropriate.
The result should concern the tested skill, duration and learner group. A single classroom demonstration does not automatically establish which method is universally best.
Worked English Design: An Unseen Inference Task
A teacher notices that pupils often quote a phrase correctly but fail to explain its effect. The proposed support teaches claim–evidence–reasoning links. A suitable measurement could use a fresh unseen passage and a consistent rubric for explaining inference, rather than merely asking whether pupils enjoyed the lesson.
An assessment administered immediately after an identical model may overstate independent transfer. The study could include a differently worded task under the same relevant skill demand and a later check if feasible. Marking criteria should be known and applied consistently.
Students can see the difference between evaluating a narrow taught capability and claiming that the whole of English ability has improved.
Worked JC Research Design: From Correlation to a Causal Question
A JC team observes that pupils attending optional workshops also report higher grades. To ask whether the workshops cause improvement, they must account for how pupils entered the programme, prior attainment, concurrent learning and assessment comparability.
A randomised offer could help answer a particular causal question in an ethical, well-run evaluation, while observational designs might use appropriate comparisons and explicit assumptions. Neither approach makes every pupil identical or eliminates attrition. The study must state whether it estimates the effect of being offered a programme or of actually participating.
This more demanding task highlights how the design follows the claim type rather than the analyst’s preferred statistical technique.
How to Separate Exploratory and Confirmatory Work
Exploratory analysis asks what patterns might exist and can generate new questions. Confirmatory analysis tests an appropriately specified proposition under planned procedures. Both can be useful, but they should be labelled honestly.
If a pupil notices an unexpected pattern after examining ten graphs, they may develop a sensible hypothesis. Testing that hypothesis on independent appropriate evidence is stronger than reporting the same discovered pattern as though it had been predicted beforehand.
At higher levels, pre-specifying the primary outcome and main analysis improves transparency. A classroom investigation should not misrepresent a post-hoc explanation as proof of a prospectively tested theory.
When the Test Itself Is the Weakest Part of the Design
A programme intended to improve scientific explanation may use an exit quiz that only asks pupils to name definitions. High quiz marks then show vocabulary recognition rather than the independent ability to justify a mechanism. A design that measures the wrong construct can look successful while missing the learning goal.
Test outcome validity by asking whether a student could score highly without possessing the capability the intervention is supposed to develop. If yes, revise the assessment before the main study, not after disappointing results appear.
For Primary Students: Fair Comparisons Come First
Primary learners can plan a simple investigation by asking what will change, what will be observed and what important conditions should stay the same. Teachers can model a prediction, a safe observation table and a conclusion that refers only to the tested cases.
Formal statistical tests and causal inference terminology are unnecessary at this stage. The essential habit is to give the observation a fair chance to answer the question.
For Secondary Students: Design Before Calculation
Secondary pupils should connect subject questions to defined measures, fair comparison, graph labels, missing-value rules and appropriate outcome statements. In Science, this can include controlled variables and repeatability. In Mathematics, an observational case study can develop understanding of survey denominators and percentage comparisons.
Ask pupils to criticise a fictional study before seeing its results. This reveals whether they can identify design problems independently rather than explain away any unfavourable pattern after the fact.
For JC Students: Identify the Causal Estimand and Assumptions
JC learners can distinguish a descriptive quantity from a causal effect, a treatment offer from attendance, and a group average from a promise for every pupil. Baseline measures, attrition and statistical uncertainty may be relevant depending on the method.
An advanced study plan should state which comparison supports the intended inference and which assumptions must hold. More complicated analysis is not a substitute for a suitable original design.
A Fifteen-Minute Study Design Clinic
Minutes 1–3: write the question and target population. Minutes 4–6: define the main measure. Minutes 7–9: choose a fair comparison or appropriate descriptive approach. Minutes 10–12: identify important confounders, ethical concerns and missing-data rules. Minutes 13–15: write the study’s likely limitations and one pilot check.
This is an illustrative teaching exercise, not a validated optimal duration. Actual studies involving identifiable pupils require appropriate school permission, data protection and sometimes professional review.
Study Design in the Age of AI
AI can produce a polished research protocol instantly, but it may invent inaccessible data, propose unethical recruitment, select an outcome that does not measure the target skill or treat a convenience comparison as a randomised experiment.
Ask a model to identify its assumptions and possible failure points, then independently check feasibility, instruments and sample design. Do not upload private pupil records or protected assessments without appropriate permission. The learner’s goal is to defend a meaningful plan without relying on the tool’s fluent wording.
The Wintour House Test: Does the Method Actually Answer the Question?
With instant dashboards and AI-generated research summaries, the scarce skill is choosing evidence that could genuinely answer an important question. No level of statistical polish can turn the wrong measurement or inappropriate comparison into the right study.
The Gold Standard is a prospective design that specifies the question, participants, measures and analysis; treats uncertainty honestly; and remains open to an outcome contrary to the designer’s preference.
Research Anchors and Limits
The Education Endowment Foundation’s Evaluation Design resources include guidance on impact evaluation, implementation and process evaluation, outcome measure selection, statistical uncertainty and cost evaluation. These are professional resources, not an official Singapore school checklist.
The EEF’s Process and People overview distinguishes impact evaluation, often using robust randomised designs, from implementation and process evaluation examining how the programme operated.
The US What Works Clearinghouse Procedures and Standards Handbook, Version 5.0, provides standards for examining research quality, baseline equivalence, attrition and intervention comparisons. Its technical procedures exceed ordinary classroom exercises.
The CDC Program Evaluation Framework (2024) emphasises context, programme description, focused questions, credible evidence and justified conclusions. This guidance originates in public-health programme evaluation and is adapted here as a reasoning analogy.
The ten skills, school examples and teaching routines are original Wintour House editorial synthesis, not official MOE/SEAB research rules, a formal research protocol or a promise of causal evidence from a small school activity.
Final study-design test: Can the learner describe what will be studied, who or what will be observed, which measures and comparisons answer the question, how data will be handled and exactly what conclusion the method could support?
