Deliver and observe

Deliver and observe runs the program with a cohort and collects evidence about how the program performs, not only about how learners perform. Problems that are felt but never recorded cannot be fixed.

What happens

• Deliver the release named at stage 8 to its cohort, and record which version they received.
• Collect the delivery signals below, and the context they came from: who taught, in what mode, with what changes.
• Log complaints, appeals and assessor disputes against the part of the program they concern.
• Hold changes to assessment until the next release, unless they fix a safety, accuracy or compliance defect.

Signals worth collecting

Each signal is tied to the outcome descriptors it concerns:

• Pass, resubmission and appeal rates per assessment item.
• Points in the sequence where learners stall or withdraw.
• Actual time on task compared with the planned session time.
• Trainer and assessor notes on what they skipped, reworked or supplemented.
• Learner feedback tied to specific modules, not only to the course as a whole.

Complaints, appeals and assessor disputes are treated as reports of possible defects in the program, each linked to the descriptor, task or content it concerns.

Guardrails for this stage

• Delivery data are signals to investigate, never automatic verdicts on content or people.
• Collect for program improvement only, with clear purpose limits, minimisation and transparency to learners.
• Record the delivery context, because the teacher is part of how a program performs.
• No changes to assessment mid-cohort, unless they correct a safety, accuracy or compliance defect.

At the gate, ask

• Is the cohort complete, and do we know which version every learner received?
• Was every problem that staff or learners raised written down, against the part of the program it concerns?
• Can we tell content issues from delivery issues in what we collected?

See all six guardrails and the practice of observability.