Distinguishing attainment and learning

Assessment has become such a cornerstone of our educational system that asking questions of it is almost inevitably traumatic. The sheer volume of statistics with which teachers, parents and other interested parties are now faced produce an air of objectivity which is simply not matched by the authenticity of that data.

Here I intend to outline a series of problems with assessment that I have encountered. They are not tied specifically to one area, but are a range of issues that I have identified and thought would be worth reflecting on.

This is the first topic I have chosen to consider. More will (hopefully) follow when I have time to write them.

1) Choice in a specification can improve attainment but not learning

For this I am going to jump to an example from my own past. When I was doing GCSEs, I chose short-course Geography. My school allowed you to do either full course History OR Geography, or short-course both (those misty pre-EBacc days!). Obviously, by taking the short-course option, it meant that I only studied 50% of the content that you study on a full-course GCSE. Fair enough. However, it was actually much better than that. The kind of kid that I was (depressingly nerdy) I tracked down the specifications for all my GCSEs so I could be sure that all the content had been covered properly by the teachers (I uncovered a range of errors). One of the things that I noticed about the Geography specification that we were doing was this: that although the course content amounted to 50% of the full-course GCSE, the exam was structured such that of the four topics we had been taught, all of them would come up on the exam in strictly ‘contained’ questions. This meant that the topic on rivers would always be question 1, the topic on immigration would always be question 2 etc*. There would be no crossover. Even better, as short-course candidates, we were able to choose which two questions from the paper we were to answer.

My mind quickly worked out a very simple strategy – all I had to do was choose the two topics that I could do best, learn them backwards and then ace them in the exam. I could actually ignore 50% of my own course content and still get an A*! I told this to my teacher, who said that it would be better not to do that because ‘they might change it’. This surprised me at the time – the whole point of a specification was that it specified how the assessment would happen and how the qualification worked. In any case, I ignored my teacher, utilised my strategy (whilst admittedly doing some very half-hearted revision of one more topic just in case) and secured the A*. The question, though, is this: did I learn more by utilising this approach? Clearly my attainment couldn’t have been better, but by ignoring half of my own course content, surely I actually compromised the spirit of the qualification?

To those of you that think this might be an approach buried in the past, it is not. At one of my training schools, and other places I am aware of, a similar approach is taken with regards to the A Level in my subject. Edexcel has designed their exam so that certain topics will always come up every year. This means that crafty schools have chosen to ignore part of the course content, safe in the knowledge that another topic they instead devote more time to will always come up. This led to a very amusing encounter when the teacher shows students a past paper. At the bottom was a question on Sexual Ethics. The students asked if they were going to study it. ‘No,’ came the reply. ‘You don’t need to know about it because you’ll have done War and Peace.’ The students were surprised – ‘But we want to study it, miss. It sounds really interesting.’ ‘But you don’t need to! You only need to know about one of them for the exam.’ This conversation went on for a short while – the teacher couldn’t understand why students might want to learn something that they simply did not need to know for the exam, whilst the students couldn’t understand why the teacher was refusing to teach them part of the course content.

If you were to ask me which one is conducive to better attainment, the answer is clear – limiting course content in line with assessment design will probably lead to better results. However, if you were to ask me which one is more conducive to better learning, the answer is less clear. You could argue that surely by focusing on a more limited range of topics, the students will have learnt those topics better and thus have learnt more. I, however, am not convinced by this argument. Let’s remember that these qualifications are accredited by Ofqual on the basis of all the course content. To study an AS Level in a subject is in their minds to have covered all of that content. It seems to do a disservice to the students to simply block them from studying parts of their own course.

And here lies the fundamental issue. As a school, your incentive is to maximise results; thus schools are determining strategies to do so (and they are doing this very well). However, much as the statistical measures may suggest an improvement, it is quite a leap to say that ‘learning has improved’. At the very least, it seems unfair to compare one school’s teaching of most of the course content, and one school’s teaching of all of it and argue that the former school is doing a better job.

A counter argument could, from a pragmatic point of view, be that schools utilising such strategies are more effective. You could even argue that my own research on my Geography spec or those schools’ research into past exam papers constitutes evidence of precisely the qualities that make me/them deserving of improved results. I was dedicated enough to track down and read the spec? Well done, I deserve the A*. A school worked out that they could ignore part of the course? Well done, they deserve the good results. Yet I am uneasy with this argument. The question I think we should ask is: Has the learning in the subject improved? And to me the answer is ‘No.’

The solution to this problem is fairly simple – design assessments where no topic is guaranteed to come up, so that schools are forced to cover all the content. As a teacher who is nervous about my own results, I can’t deny feeling tense about this recommendation. Nor can I deny that the strategies outlined above have had some pull on me. But at the same time, I don’t think it’s easy to argue that being able to ignore part of a course really means better learning has taken place. Hence, the best approach is to eliminate the issue altogether through assessment design.

 

*These aren’t necessarily the actual topics on the spec, they’re just for illustration.

Please note: I am not suggesting that Edexcel have intentionally designed their qualification this way.