Co-design Evaluation Criteria with Young People

Proposed evaluation criteria for outcomes, experiences and fairness

Owner: Recorder

Supplement indicators decided by adults alone.

Cases where judgements differ and points for revising criteria

Owner: Recorder

Find flaws that overvalue only what can be measured.

Table of agreement and certainty of evidence for each criterion

Owner: Recorder

Record agreement with each criterion in this session separately from certainty of its supporting evidence, and distinguish provisional adoption from additional testing. Link criteria with low certainty of supporting evidence to additional data, a person to check, and a review date.

Everyday tools

Broad-tip pens

Quantity: 1 per person

Format: Dark broad-tip pens or whiteboard markers

Check that the ink flows and place the pens where participants can reach them. Avoid pale-coloured pens.

Pens

Quantity: 1 per person

Format: Dark broad-tip pens or whiteboard markers

Check that the ink flows and place the pens where participants can reach them. Avoid pale-coloured pens.

Sticky notes for proposed evaluation criteria

Quantity: About 8 per person

Format: Source material specification: Approximately 8 challenge sticky notes per person, 75 × 75 mm

Source-bound format display correction; the other fixed helper-derived guidance is unchanged.

Do not buy specialised notes. Write Proposed evaluation criterion at the top of each paper slip. Record one criterion and its number per slip, then place it on the adaptation sheet for agreement with criteria and certainty of supporting evidence. Do not write personal names or scores of individual ability.

Sticky notes for current handling

Quantity: About 6 per group, for every group

Format: Sticky notes around 75 × 75 mm, or A4 copy paper cut into four pieces.

Write Current handling at the top. If using the same colour as the proposed-criterion notes, draw a double border. Do not buy another colour; distinguish notes with wording and borders. Write the corresponding criterion numbers and handling, such as provisional adoption or additional testing. Multiple criterion numbers may share a note if their handling is the same. For criteria with low certainty of supporting evidence, retain additional data, a person to check, and a review date beside the relevant criterion on the adaptation sheet.

Thick pens

Quantity: 1 per person

Format: Dark broad-tip pens or whiteboard markers

Check that the ink flows and place the pens where participants can reach them. Avoid pale-coloured pens.

Sheets to prepare by hand

Table for checking evaluation criteria

Quantity: 1 sheet per group

Format: Two A4 sheets placed side by side. Join them with masking tape if a larger surface is needed.

Divide the working surface into 3 fields: “Evaluation criterion”, “Evidence used for judgement” and “Matters excluded from evaluation”. Write each heading in words rather than relying on colour alone.

Table for safely checking a criterion

Quantity: 1 sheet per criterion

Format: A4 copy paper

Divide the working surface into 2 fields: “What can be observed or checked” and “Personal data not to collect”. Write each heading in words rather than relying on colour alone.

Contact card for checking the day’s safety response

Quantity: 1 per person

Format: A4 copy paper

Divide the working surface into 4 fields: “Person responsible for safety”, “Backup if the person responsible for safety is absent”, “Contact for the day” and “Emergency source of advice”. Write each heading in words rather than relying on colour alone.

Table for checking the scope of publication of records

Quantity: 1 for the whole group

Format: A4 copy paper

Divide the working surface into 4 fields: “Publication scope”, “Secondary use”, “Storage location and deadline” and “Deletion method”. Write each heading in words rather than relying on colour alone.

Cards for choosing how to respond

Quantity: 1 set per person

Format: Sticky notes around 75 × 75 mm, or A4 copy paper cut into four pieces.

Write “Speak”, “Respond with a drawing”, “Write in large letters” and “Do not answer this time” on separate sheets, preparing the required number of each type. Distinguish types by their written headings even if the paper is the same colour.

Case cards

Quantity: 3 per person

Format: Sticky notes around 75 × 75 mm, or A4 copy paper cut into four pieces.

Before starting, write just one example close to the content used in this method. Show it as an example of how to write, not a correct answer for participants to copy.

Table separating applicable, uncertain and inapplicable cases

Quantity: 1 sheet per group plus 1 for whole-group consolidation

Format: Source material specification: One A3 grid per group with ‘Applies’, ‘Uncertain’, and ‘Does not apply’ columns, plus one for whole-group integration

Source-bound format display correction; the other fixed helper-derived guidance is unchanged.

Create 3 columns from left to right: “Applicable”, “Uncertain” and “Not applicable”. For uncertain cases, record what needs to be checked and who is responsible.

Principle-statement cards

Quantity: 1 per group

Format: Sticky notes around 75 × 75 mm, or A4 copy paper cut into four pieces.

Do not buy specialised cards. Write “Principle statement” at the top of each paper slip. If participants write on them, use one item per slip and handwrite the necessary headings before the session starts.

Programme-specific adaptation sheet comparing agreement with criteria and certainty of supporting evidence

Quantity: 1 sheet per group of 3–6

Format: Source material specification: One A3 four-quadrant sheet showing agreement and certainty per group of 3–6

Source-bound format display correction; the other fixed helper-derived guidance is unchanged.

Label this as a homemade adaptation sheet made for this programme, not as the same specification as the referenced method's Agreement on the solution × Certainty of the outcome diagram. Label the horizontal axis Agreement with the criterion among participants in this session and the vertical axis Certainty of the supporting evidence, with high and low at their ends. Number each criterion and place it in the four quadrants. Retain differences of view about the same criterion. Record its current handling beside it; where certainty of supporting evidence is low, add fields for Additional data, Person to check, and Review date. Do not score individuals, and write headings in words.

Information to gather before the session

Guide helping individuals decide whether to participate

Quantity: 1 copy per person

Format: A4 copy paper or digital viewing data with the same content

Use words understandable to a first-time reader to describe “How the individual gives consent”, “Handling records after withdrawal”, “The choice not to participate” and “Conditions for honoraria”. Add the preparation date, person who checked it and contact for questions. If distributing it, provide the same content in large print or a viewable digital version.

Available at the venue

Wall surface or large paper for whole-group display

Quantity: 1 surface

Format: Large-format paper or a whiteboard

Write Agreement and certainty of evidence by criterion at the top. Arrange each group's criterion numbers, agreement with the criterion, certainty of supporting evidence, and current handling. Link criteria with low certainty of supporting evidence to their additional data, person to check, and review date. Do not display records that identify individuals or holders of minority views. Use lettering that can be read from a distance.

Youth Evaluator Criteria Workshop

1. Display the safety lead, backup, same-day contact arrangements, local emergency procedures, consultation contacts, and the adult responsible for responding to results. Cancel if any one is missing.
2. Explain plainly personal consent appropriate to age and circumstances, required guardian and organisational procedures, withdrawal, no disadvantage from non-participation, payment conditions, and choices to return, delete, or anonymise existing records and stop their use after withdrawal.
3. Provide dictation, drawing, large text, breaks, and passing. Collect no personal data or only the minimum, and decide publication, secondary use, storage, and deletion in advance. Separate criteria and evidence designed with young people from adult responsibility for safety and final judgement. Specify the implementation lead, response date, whether suggestions are adopted, and reasons implementation is not possible. Do not ask anyone to speak for all young people or rubber-stamp adult-created materials.

Boundary Case Grid

1. One display surface at seated eye level where everyone can read the principle throughout
2. One work surface per group where only uncertain cases can be brought into the centre of the table
3. One wall at least 1 m wide for keeping the original and revised principles side by side

Agreement and Certainty Matrix

1. One island table per group of 3–6; distribute people evenly
2. One wall for the quadrant sheets, visible to everyone