Effective assessment in science and maths Daisy Christodoulou,

Effective assessment in science and maths Daisy Christodoulou, Director of Education, No More Marking

Quality model and difficulty model

Quality model of assessment The most obvious [method of assessment] is to ask the students to do something and see how well they do it. By referring to a scaled set of criteria or descriptors, the examiner judges which number on the scale best indicates how well each student has performed. Competitive ice dancing uses a pure quality model. The skater is expected to go out and perform in a way that impresses the judges as much as possible. Ahmed, Ayesha, and Alastair Pollitt. "The support model for interactive assessment." Assessment in Education: Principles, Policy & Practice 17.2 (2010): 133-167, p.134 3

Difficulty model of assessment “The usual approach consists of setting a series of tasks that vary in difficulty, often starting with easy ones and moving on to harder ones. What we ‘see’ is which tasks each student can do successfully. We call this the difficulty model of measurement because central to it is the mapping of students’ ability to the difficulties of a graded set of tasks (it has also been called the Counting strategy)” A high jump competition is a clear example of the difficulty model, consisting of a series of tasks of ever increasing difficulty which continues until everybody has failed. 4

Quality model and difficulty model – equal technical challenges, but in different ways “The technical concern in the quality model is the reliability of the judges, whereas in the difficulty model it is the internal consistency of the questions.” Quality model –a lot of work to do after the assessment! Difficulty model –a lot of work to do before! 5

What’s appropriate for maths and science? Traditionally, we use the difficulty model for maths and science. This is totally appropriate and can make assessment very easy for the teacher / marker. Advantages: as long as you use externally-set tests, you can quickly and easily get nationally standardised data on your pupil’s performance. Disadvantage 1: tests you’ve set yourselves might look just like a GCSE paper – but they aren’t! You can’t use these to get nationally standardised results. Disadvantage 2: you can’t use more open questions. 6

Why can’t I set my own questions and get nationally standardised results? ‘Can compare two fractions to identify which is larger’ Which is bigger: 3/7 or 5/7? 90% get this right Which is bigger: 3/4 or 4/5? 75% get this right Which is bigger: 5/7 or 5/9? 15% get this right Qtd in Wiliam, Principled Assessment Design, SSAT 2014 7

What if I want to use quality model styles of question in maths and science? People often want to use quality model questions because they think they are more valid. The problem is that – traditionally - quality model questions are hard to mark reliably. If an assessment is not reliable, it cannot be valid either!! Reliability is a necessary condition of validity. The risk is that you introduce an exciting and original real-world question designed to shed new light on a pupil’s thinking – but because it is so hard to mark it consistently, it doesn’t actually shed any new light! You don’t know if the results from it are down to a pupil’s original thinking – or marking error! 8

Other problems with the way quality model questions are traditionally used in maths and science People attempt to address the unreliability problem by producing very specific mark schemes – the problem with this is that it then ends up stereotyping the response from the pupil (this was a real problem with the old maths & science coursework) Further risk with maths and science is that you end up rewarding literacy, not maths and science, as open questions tend to involve extended writing 9

Maths and science GCSEs include more ‘quality model’ style questions – and it’s not always a good thing 10

Can Comparative Judgement provide a solution? 11

Normally, we ask: does this essay meet the criteria?

Instead, we should ask, is this essay better than this essay?

Maths Good questions for Comparative Judgement Not so good Add ¾ and ½ What rules do you know in maths that are A block of cheese costs 2.66 per kg. How much always true? does 4.5kg cost? How do you add and subtract fractions? How do you solve equations? 14

Science Good questions for Comparative Judgement Where does the human body get energy from? What is a chemical bond? Why are some elements more reactive than others? Not so good Write the formula for a molecule of chlorine gas. Give one purpose of chlorine in industry. 17

Quality model and difficulty model – equal technical challenges, but in different ways “The technical concern in the quality model is the reliability of the judges, whereas in the difficulty model it is the internal consistency of the questions.” Quality model –a lot of work to do after the assessment! Comparative judgement solves this problem by getting teachers / markers to judge the pupil work Difficulty model –a lot of work to do before! Comparative judgement solves this problem by getting teachers / markers / experts to judge the difficulty of the questions in advance of them being taken by pupils 18

No More Marking Twitter: @daisychristo Email: [email protected] [email protected] Blog: https://blog.nomoremarking.com/ 19