The TEF: why it isn’t good enough

 

In December, the Higher Education Funding Council for England (HEFCE) released a document aimed at students to explain what the Teaching Excellence Framework (TEF) is. In it we see statements claiming that the TEF will “give students clear information about what teaching quality is” and to “encourage a stronger focus on the quality of teaching in higher education”. Interestingly, these statements seem to be misleading and considering this is a document for students that’s slightly worrying. Dr. Andrew Gunn, a researcher in higher education policy, has said that the TEF “is assessing everything that goes on before admission – from outreach, choosing candidates and interviews – to the actual teaching environment and then everything that students do after graduation.” So not simply teaching excellence but the “entire teaching function of a university”.

Are these aims possible using the metrics that the TEF has set out? These metrics are: the National Student Survey (NSS), retention rates, graduate employability, and a 15 page narrative statement on the institutional context of the university. The 15-page narrative statement will be used as the deciding factor of whether a university can be upgraded or not after a provisional rating is given according to the other three metrics. All of this is being measured so that they can charge future students more money depending on whether they achieve bronze, silver, or gold.

The NSS

Research into the 2014 results of the NSS as to whether students at universities that rated research highly or where they had a high proportion of professionally qualified faculty found that “students are happiest on degree programmes where high proportions of staff have the following characteristics: white, full professors, holding doctorates, and on fixed-term contracts”. The researchers then questioned whether students, while completing the NSS, failed to see the impact that research may have on their studies and the potential that going to a more ‘elite’ university may instil expectations which are not found at non-elite universities thus increasing the dissatisfaction scores of the NSS. These findings and musings call the validity of the NSS into question as a metric for teaching excellence let alone the teaching function of a university. Even the Chair of the TEF’s assessment panel has admitted to the inadequacy of the NSS stating that, “I do not think student satisfaction is an accurate proxy for teaching quality” and yet in the same breath claims that this doesn’t mean we mustn’t use the NSS. He argues that we must discover how the data it provides could be used ‘effectively’. Surely this is contradictory? Claiming that the NSS can’t be used as a useful metric but finding some way that it can?

Retention rates

Research into retention rates at universities found that those from disadvantaged backgrounds were more likely to leave before completion of their studies and those who went to universities with lower UCAS requirements were more likely to withdraw from their courses. There were universities effectively challenging these trends, with those achieving low UCAS entry points and coming from disadvantaged backgrounds remaining at university through various interventions- but this isn’t the fault of universities.

Other research indicates that the problem is not with universities but with secondary schools. If students were better prepared for the pressures and expectations of university they would either: a) not attend because it was not something they wanted to do, or b) already be developing the skills required to engage with undergraduate studies.

From speaking with students on various courses, I have found that a lot of lecturers seem to ‘spoon-feed’ certain students if they wish them to stay on. The very nature of ‘paying’ for an education (which is, in fact, borrowing money from taxpayers) has had a negative impact on poorer students’ choice of university which could be also affecting retention rates as it isn’t their first choice.  Also, other research has shown that some courses are not worth the cost when compared to graduate earnings. Inevitably, students may not feel their ‘education’ is worth the amount of money they are borrowing to pay for and stop attending these courses. This is not solely the fault of universities but of the policymakers who are pushing an ideological vision of what education must be in the UK.

Graduate employability

Research into whether there are too many graduates in the UK found that graduates are filling jobs that were originally filled by school leavers; whether their degree is relevant to the position or not. This inevitably creates a false hierarchy of ‘educated’ vs the uneducated amongst employers, considering both will need to be trained in a graduate-irrelevant job. I am a firm believer that education – knowledge – must not be commodified or turned into a means to an end (i.e. a job). Knowledge and education must be made available to all. The argument that there are too many graduates for graduate related jobs is true, hence the mismatch of skills required and degree obtained – but this doesn’t mean the answer is to limit the people going to university or to funnel more people into university. It means we need to address the role of the university. It must be a centre of knowledge and learning, not an attempt to mark a step on career progression. There is a pressure on the education sector that need not be there: a notion that university will better the lives of those who attend and that we must fit as many students as possible into university for them to ‘better themselves’ (i.e. graduate relevant jobs). It isn’t working.

15-page contextual narrative

This doesn’t need much attention. One simple point: giving the universities the ability to colour their function in a way that will inevitably see them in a positive light, even if that isn’t the reality, is like giving someone a test and then asking them why they believe they should pass.

What’s your point?

Most of the issues with satisfaction scores, retention rates, and graduate employability come down to the idea that education is a commodity to be bought and sold to further careers rather than having a right to exist on its own merit.

These metrics of ‘teaching excellence’ are heavily influenced by factors that the metrics don’t account for. If a university doesn’t meet a student’s lived experience of education, for example, (which in the UK isn’t really education but a banking of information) then they may be dissatisfied and/or drop-out.

Therefore the metrics for the TEF make no sense in regards to teaching excellence no matter how you spin it (even by changing the definition of ‘teaching excellence’). We shouldn’t be making the best of the metrics, as the Chair of the TEF assessment panel said re the NSS, but trying to figure out what higher education needs to be and how to measure teaching excellence.

In conclusion

The NSS is subjective and insufficient, drop-out rates are affected by background and scores in secondary school, and employability can be factored down to whether an employer believes a degree is a good thing (regardless of job relevance).

And this information is being used to charge more money to ‘learn’?

Think about that.

(Disclaimer: I am a post-graduate research student at the University of Northampton. My views do not represent the University of Northampton or the University of Northampton Students Union).