The problem of inter-subject comparability

Students receiving qualifications. But are those qualifications graded equitably? Credit: Shutterstock/michaeljung
Students receiving qualifications. But are those qualifications graded equitably? Credit: Shutterstock/michaeljung

This is the first of three blogs on the thorny issues of inter-subject comparability, grading severity, student choice and subject difficulty. In this one, I am following up a presentation I made at an Ofqual conference of Thursday a fortnight ago.

Ofqual are running a consultation on the question of inter-subject comparability. On 4 February they arranged a conference to discuss whether anything should be done to address this question and what that might be. The question arises because, for a given student, there are predictable differences between the likely grade outcomes in different subjects at A-level. That is, they might be predicted an A in business studies but a C in physics.

As became apparent, this is a problem that has been around for more than 50 years. And the affected subjects have been consistent over that time.

The question about addressing it has been asked (and passed over) for a couple of decades. One way of phrasing the questions is: is it possible to compare the A-level (or GCSE) grades of one person who took maths, physics and chemistry and another who took English, psychology and business studies?

However, this phrasing of the question is a diversion. The answer is clearly no. Most people agree with Glenys Stacey’s introductory point that it is almost meaningless to talk about comparability, in an absolute sense, at a subject level. Neither is it very meaningful to base discussions about the system on comparisons of two people. However, it is meaningful to talk about the chance (among a large population) of getting a given grade in two different subjects.

Therefore, it would be more helpful to ask: should there be a similar likelihood of students getting a given grade in two different subjects? And the answer to this is a resounding yes (with some provisos for similar aptitude, prior attainment, and so on).

There are three main reasons for my answer. They are:

  • to provide and encourage open and beneficial choices for students at age 16.
  • to ensure equity and transparency in selections based on grades
  • to prevent this issue feeding into the unhelpful language and notion of difficulty.

I will briefly discuss each of these reasons in this blog. Furthermore, I will argue that they have different effects on different groups of students – thereby contributing, in different ways, to the lack of social diversity and the lack of gender-balance in some subjects.

Removing barriers to choice

The first – and possibly most important point given the current, dominant problems – is that our education system needs to have a grading system that does not dissuade or debar anyone from taking the subjects that they want to take. Their choice should be based on their interests, abilities, ambitions, future desires and so on.

Currently, we do not have such a system. Some subjects (physics, maths, chemistry, biology) are graded more severely than others (psychology, English, business studies and law). This is shown in figure 1 of the CEM report from 2008.

When choosing their A-levels, students (with input from their schools) are having to play half-informed strategic games and choose A-levels based on likely grade outcomes rather than those that match their desires. Lack of information often means that their choices aren’t necessarily in their long-term interest.

Consequently, the differences in grading severity are leading to barriers to students’ choice. I will expand on these barriers in my second blog. Suffice to say that those barriers have existed for a couple of decades, they are becoming more noticeable and extreme over time and they play differently to different groups of students.

If subjects were graded more equitably, then these barriers would come down.

Parity for selection

The second point is about the comparability of grades for those who use them for selection (employers and admissions tutors). If the same grades in different subjects are seen as equivalent, then that is clearly inequitable. That is, if an A grade in psychology is seen as equivalent to an A grade in biology, then the psychology student is given an unfair advantage. On average, they will have achieved a better grade simply by choosing psychologybecause it is graded less severely than biology.

In reality, it is probably not the case that the two A grades are seen as equivalent. The reality is that admissions tutors and employers will have views about the different values of an A grade in different subjects. But these views are not known, so there is a lack of transparency. They were partly expressed in the Russell Group’s Informed Choices report of 2011, which introduced the term ‘facilitating subjects’. However, that is only one group of selectors. There remains a vast, variable and opaque system of decision-making based on the perception of subjects and their difficulty.

This means that, not only are students playing the strategic game, they don’t know the rules. No-one does. They might choose a subject that is generously graded, only to find that it has less value than the severely graded subject that they wanted to take all along.

With equitable grading – based on equivalent likelihood of getting grades – the system would be transparent. It would become clear that it was the grade – and not the reputation of the subject – that mattered. It would be clear that an A in media studies should be as valued by employers as an A in mathematics.

Unhelpful language of difficulty

Thirdly, there is the effect of how both of the points above play into and feed the unhelpful language of difficulty and how this influences and reduces free choices. I will write about difficulty in the coming weeks. However, the point to make here is that it is better to refer to grading severity as its own issue rather than conflating it with difficulty. The term ‘difficulty’ only muddies the waters.

Diversity

Finally, the barriers above contribute to a lack of diversity in severely graded subjects.

In particular, the first two (relating to free choice and selection) affect social diversity because the system favours those who have better access to information – either from their parents or what is on offer in their schools. So the effects of variable grading severity disadvantage those who do not have the privilege of access to good information.

Furthermore, it is likely that the way in which grading severity plays into the notion of difficulty contributes to girls turning away from subjects with a reputation for being hard.

So, let’s return to the original question from Ofqual:

Does something need to be done about inter-subject comparability?

I repeat my resounding yes. In the Ofqual online poll, one of the options is to do nothing. This is the worst possible option. Let me summarise why.

  • This is not about trying to achieve parity, but about enabling unfettered choice of A-level subjects
  • While it is not addressed, the system will either be inequitable or it will be opaque. Currently, it is probably both. The only way to be both equitable and transparent is to have a justifiable system of grading.
  • This isn’t just about spurious notions of which subject is more difficult. It is a question of social equality and access for everyone to the same information and courses that are available to the more privileged.

Finally, it is worth considering what we would do if we invented a grading system from scratch. Would we try to have parity (based on likelihood) between subjects? Or would we set up the current system in which some subjects allow students, on average, to get a grade and a half higher than in other subjects?

My answer would be the former (a goal of parity). Unfortunately, we aren’t necessarily starting from scratch. We have a system which has evolved to include this unplanned oddity. And it is being retained simply because it seems too painful to address it. Not because it is philosophically or ethically justifiable. So what can we do?

How should it be addressed?

There are various methods for analysing the differences in grading severity – the simplest to understand are statistical and are based on expected progression from GCSE grades. Therefore, this is the method I have referred to in this blog. However, there are other, more subtle methods. Any changes should bring to bear everything we know in order to make the grading equitable. But they are likely to involve a statistical element.

Option 1: Subtly shift the outliers so that grade likelihoods become more similar in different subjects.

Option 2: Calculate and rebalance the grades every year using a statistical means to ensure that likelihoods are equal between subjects.

Option 3: Overhaul the grading system and bring all subjects into line (using everything we know). This would be a one-off change. Given the changes to the grading system at GCSE, there would be an opportunity (in the near future) to introduce new grades (possibly following the 1 to 9 model at GCSE) that can be shown to have equal likelihoods across subjects.

Option 4: Weighting the grades in school performance measures. This is to prevent schools recommending subjects to students because it will help the school’s position in the league tables.

Option 5: Introduce diplomas or baccalaureates to prevent the need to strategically choose combinations of A-levels. In effect there would be a small number of pre-determined combinations.

Naturally, all of these options need further discussion and consideration. However, the worst possible option is to do nothing.

Related posts
Charles Tracy

Charles Tracy

Charles is the IOP’s head of education. He came to the Institute in 2006 after 12 years as a physics teacher, including two as head of physics and three as head of science.
Charles Tracy
FacebooktwitterFacebooktwitter

Comment via Facebook

Comment via Disqus

iop-blog

Comment via Google+