Students receiving qualifications. But are those qualifications graded equitably? Credit: Shutterstock/michaeljung

The problem of inter-subject comparability

Students receiving qualifications. But are those qualifications graded equitably? Credit: Shutterstock/michaeljung
Students receiving qualifications. But are those qualifications graded equitably? Credit: Shutterstock/michaeljung

This is the first of three blogs on the thorny issues of inter-subject comparability, grading severity, student choice and subject difficulty. In this one, I am following up a presentation I made at an Ofqual conference of Thursday a fortnight ago.

Ofqual are running a consultation on the question of inter-subject comparability. On 4 February they arranged a conference to discuss whether anything should be done to address this question and what that might be. The question arises because, for a given student, there are predictable differences between the likely grade outcomes in different subjects at A-level. That is, they might be predicted an A in business studies but a C in physics.

As became apparent, this is a problem that has been around for more than 50 years. And the affected subjects have been consistent over that time.

The question about addressing it has been asked (and passed over) for a couple of decades. One way of phrasing the questions is: is it possible to compare the A-level (or GCSE) grades of one person who took maths, physics and chemistry and another who took English, psychology and business studies?

However, this phrasing of the question is a diversion. The answer is clearly no. Most people agree with Glenys Stacey’s introductory point that it is almost meaningless to talk about comparability, in an absolute sense, at a subject level. Neither is it very meaningful to base discussions about the system on comparisons of two people. However, it is meaningful to talk about the chance (among a large population) of getting a given grade in two different subjects.

Therefore, it would be more helpful to ask: should there be a similar likelihood of students getting a given grade in two different subjects? And the answer to this is a resounding yes (with some provisos for similar aptitude, prior attainment, and so on).

There are three main reasons for my answer. They are:

  • to provide and encourage open and beneficial choices for students at age 16.
  • to ensure equity and transparency in selections based on grades
  • to prevent this issue feeding into the unhelpful language and notion of difficulty.

I will briefly discuss each of these reasons in this blog. Furthermore, I will argue that they have different effects on different groups of students – thereby contributing, in different ways, to the lack of social diversity and the lack of gender-balance in some subjects.

Removing barriers to choice

The first – and possibly most important point given the current, dominant problems – is that our education system needs to have a grading system that does not dissuade or debar anyone from taking the subjects that they want to take. Their choice should be based on their interests, abilities, ambitions, future desires and so on.

Currently, we do not have such a system. Some subjects (physics, maths, chemistry, biology) are graded more severely than others (psychology, English, business studies and law). This is shown in figure 1 of the CEM report from 2008.

When choosing their A-levels, students (with input from their schools) are having to play half-informed strategic games and choose A-levels based on likely grade outcomes rather than those that match their desires. Lack of information often means that their choices aren’t necessarily in their long-term interest.

Consequently, the differences in grading severity are leading to barriers to students’ choice. I will expand on these barriers in my second blog. Suffice to say that those barriers have existed for a couple of decades, they are becoming more noticeable and extreme over time and they play differently to different groups of students.

If subjects were graded more equitably, then these barriers would come down.

Parity for selection

The second point is about the comparability of grades for those who use them for selection (employers and admissions tutors). If the same grades in different subjects are seen as equivalent, then that is clearly inequitable. That is, if an A grade in psychology is seen as equivalent to an A grade in biology, then the psychology student is given an unfair advantage. On average, they will have achieved a better grade simply by choosing psychologybecause it is graded less severely than biology.

In reality, it is probably not the case that the two A grades are seen as equivalent. The reality is that admissions tutors and employers will have views about the different values of an A grade in different subjects. But these views are not known, so there is a lack of transparency. They were partly expressed in the Russell Group’s Informed Choices report of 2011, which introduced the term ‘facilitating subjects’. However, that is only one group of selectors. There remains a vast, variable and opaque system of decision-making based on the perception of subjects and their difficulty.

This means that, not only are students playing the strategic game, they don’t know the rules. No-one does. They might choose a subject that is generously graded, only to find that it has less value than the severely graded subject that they wanted to take all along.

With equitable grading – based on equivalent likelihood of getting grades – the system would be transparent. It would become clear that it was the grade – and not the reputation of the subject – that mattered. It would be clear that an A in media studies should be as valued by employers as an A in mathematics.

Unhelpful language of difficulty

Thirdly, there is the effect of how both of the points above play into and feed the unhelpful language of difficulty and how this influences and reduces free choices. I will write about difficulty in the coming weeks. However, the point to make here is that it is better to refer to grading severity as its own issue rather than conflating it with difficulty. The term ‘difficulty’ only muddies the waters.


Finally, the barriers above contribute to a lack of diversity in severely graded subjects.

In particular, the first two (relating to free choice and selection) affect social diversity because the system favours those who have better access to information – either from their parents or what is on offer in their schools. So the effects of variable grading severity disadvantage those who do not have the privilege of access to good information.

Furthermore, it is likely that the way in which grading severity plays into the notion of difficulty contributes to girls turning away from subjects with a reputation for being hard.

So, let’s return to the original question from Ofqual:

Does something need to be done about inter-subject comparability?

I repeat my resounding yes. In the Ofqual online poll, one of the options is to do nothing. This is the worst possible option. Let me summarise why.

  • This is not about trying to achieve parity, but about enabling unfettered choice of A-level subjects
  • While it is not addressed, the system will either be inequitable or it will be opaque. Currently, it is probably both. The only way to be both equitable and transparent is to have a justifiable system of grading.
  • This isn’t just about spurious notions of which subject is more difficult. It is a question of social equality and access for everyone to the same information and courses that are available to the more privileged.

Finally, it is worth considering what we would do if we invented a grading system from scratch. Would we try to have parity (based on likelihood) between subjects? Or would we set up the current system in which some subjects allow students, on average, to get a grade and a half higher than in other subjects?

My answer would be the former (a goal of parity). Unfortunately, we aren’t necessarily starting from scratch. We have a system which has evolved to include this unplanned oddity. And it is being retained simply because it seems too painful to address it. Not because it is philosophically or ethically justifiable. So what can we do?

How should it be addressed?

There are various methods for analysing the differences in grading severity – the simplest to understand are statistical and are based on expected progression from GCSE grades. Therefore, this is the method I have referred to in this blog. However, there are other, more subtle methods. Any changes should bring to bear everything we know in order to make the grading equitable. But they are likely to involve a statistical element.

Option 1: Subtly shift the outliers so that grade likelihoods become more similar in different subjects.

Option 2: Calculate and rebalance the grades every year using a statistical means to ensure that likelihoods are equal between subjects.

Option 3: Overhaul the grading system and bring all subjects into line (using everything we know). This would be a one-off change. Given the changes to the grading system at GCSE, there would be an opportunity (in the near future) to introduce new grades (possibly following the 1 to 9 model at GCSE) that can be shown to have equal likelihoods across subjects.

Option 4: Weighting the grades in school performance measures. This is to prevent schools recommending subjects to students because it will help the school’s position in the league tables.

Option 5: Introduce diplomas or baccalaureates to prevent the need to strategically choose combinations of A-levels. In effect there would be a small number of pre-determined combinations.

Naturally, all of these options need further discussion and consideration. However, the worst possible option is to do nothing.

Charles Tracy

Charles Tracy

IOP head of education
Charles Tracy

7 thoughts on “The problem of inter-subject comparability

  1. As a physics teacher I couldn’t agree more with this blog. I have written about similar issues in my blog including on issues of gender balance – see link below.

  2. If I look through our sixth form’s ucas offers the science & eng. courses are all at the top end, including three A*s to do physics. Doesn’t my unscientific survey imply that the Uni’s regard science A levels as over marked?

  3. Thanks Dan. That’s an interesting and useful blog. It covers a lot of the statistical evidence that I referred to but did not dwell on in detail. I like the bell curve graph – we generated something similar when we tried that analysis. I think the divergence is less at higher grades because – to use an amplifier analogy – we simply hit the rails. There is no headroom. Which is an indirect result of grading severity (that I will revisit): the intake has become distorted (with severely graded subjects being skewed towards those with higher prior attainment). Consequently, there is not enough discrimination at the top end. I shall develop this a bit in my response to Steve.

  4. Thanks Steve. I’m not sure we can infer that universities regard science as over-marked. They are simply making offers that:
    a) they know will not reduce their intake (i.e. enough students will get those offers
    b) enable them to select the ones with the best academic record
    c) enable them to discriminate at the top end
    d) probably encourage the more academic students to apply to them (it is a phenomenon that if you put your offers up, you will get people with better results applying – i.e it encourages students rather than discourages them).

    I have to say the highest offer I have seen is A*A*A – for example at Durham ( There are physics courses from BBC up to Durham’s offer.
    These high offers are a result of the point I have made above. Severely graded subjects have a skewed intake (towards those with higher prior attainment). Consequently, in physics the modal grade is A*/A (with 30% of the entry achieving that grade). That’s about 8,500 students in the UK. And nearly 3000 get an A*. So, you can see that universities (particularly those who are selectors) can afford to ask for the top grades.

    I don’t think they are basing their offers on the severity of the grading – simply on the numbers getting the grades.

    Also, for physics (and some engineering courses), physics is a requirement anyway (and so is maths), so there isn’t any question of them taking another subject (whether or not it is graded less severely).

  5. Thanks Charles for raising the profile of this issue. The problem seems to becoming even more severe for the less high-flying students – the Ofqual report you are using suggests that, of the current intake about 15% will be awarded E or U, and they think it should be only 2%. Grade compression has been becoming very noticeable in the last few years: Students who score over 50% raw marks in A2 exams, and are good physicists (not outstanding, but those are not the only students we want to teach) are being awarded U. (The same exams award A* for about 75% raw marks, and all the other grades are squeezed between).

    Physics has the reputation as a hard A level. It is, not because it is a technical and exacting subject to study, but because the grade boundaries have been set too high. We are in great danger of discouraging reasonable students who could become competent technical professionals in the future. Many of these may be female.

  6. Hi Charles,
    I agree with everything in your reply, except maybe the implication that Eng courses aren’t as severe in their requirements. That being so I wonder what benefit you think bringing Phys into line with other A levels will have? Won’t the Unis, who can fill their courses now, just move their grades accordingly & the impression of difficulty persist, albeit at one remove?

  7. Why is the “notion” of difficulty “unhelpful”? What if it is true that certain fields of human endeavour actually are more difficult than others? In order to achieve some spurious “compatibility” do we make school-level physics and maths even easier than it now is, and so further removed from the level of the actual subject as practised in the world outside, or do we make the other subjects harder and so deny people the opportunity to make progress in something?

    40 years ago O-level physics/maths was a good preparation for A-level physics. A-level (with maths) was a good preparation for university. My belief is that this is less true today; pupils report a big jump from GCSE to A-level and universities see quite unprepared entrants despite their A-level grades. We should be moving things in the opposite direction to what you seem to be proposing. However, it should be clearly understood that this has happened so an A in physics (or French) is worth more than an A in media studies or psychology.

    Let’s not shy away from “difficulty”. My PhD in electronic engineering was intellectually less demanding than my final year as a physics undergraduate. An EE lecturer on hearing that I had in the past done some work on quantum gravity said “Wow – that’s difficult!” Physics is difficult. That is why it attracts some of the brightest people on the planet. Make it easier and they may go elsewhere.

Leave a Comment