New research reveals the variability of policies, practices and student experience in the age of AI

QAA's new report on the impacts of artificial intelligence on assessment has exposed variations not only between the policies of different institutions but also inconsistencies in the application of those policies in assessment practices within institutions.

The research finds that this variability affects the practices of different academic departments, programmes, modules and individual practitioners, resulting in confusion among staff and students, and material inconsistencies in the learner experience.

The report, The Perfect Storm: AI, assessment and a sector under pressure, draws new insight from a series of roundtables held with student and sector staff stakeholders, offers fresh analysis of the issues and trends identified by QAA’s own reviews, and collates the findings of a range of recent studies of GenAI use in higher education. The report identifies five key areas of risk:

• The validity of assessment design.
• The parity of student experience.
• The breakdown in trust between students and staff.
• The acquisition of foundational skills and the meaning of a degree.
• Constraints upon the sector, during a period of tightening budgets and rapidly changing AI tools.

Student participants in this research have commented:

• "I didn't know it existed for most of the first year and then by the end all you could see in lectures was AI."
• "I think a lot of people that I know got into their degree not wanting to use AI and then just as the degree progresses and as it becomes a lot more time-consuming, people start relying on it more and more, even if a lot of people initially didn't want to."
• "I think that it's very disheartening when you put a lot of time and effort into something and then you message your friends saying, well, how long did it take you to do that? And they say, oh, four minutes."
• " AI makes assessment feel unfair, particularly if some students use it to enhance or generate work while others rely entirely on their own effort."
• "If AI produces a really good piece of work and that work gets marked 90%, I feel subconsciously lecturers, or whoever it is, will mark everything against that new standard that's been created and work that humans create will not be considered perfect."

The report concludes by making a series of key recommendations to the higher education sector, highlighting the importance of:

• Investment in substantive staff and student training.
• Building student voice into the development of AI policy and guidance from the outset.
• Addressing the consistency of student experience as a priority.
• Engaging in sector initiatives to agree shared standards, including the refresh of the Academic Integrity Charter during the forthcoming academic year.

Report co-author Rebecca Robinson, Data Analyst at QAA, said:

"Our research demonstrates that students and colleagues across the sector very much want to discuss these issues – and have lots of useful and interesting things to say. But it's also shown that it's not always easy to see what others are doing in this space, the directions they've taken and how much progress they've made. What we really need to see is a greater level of transparency and more opportunities to discuss these challenges and to share our solutions, because everyone's tackling the same issues. This is why we at QAA have committed to establish an AI in Assessment Community of Practice to support this important work."

Report co-author Ciaran Donaghy, Lead Policy Officer (Devolved Nations) at QAA, said:

"Some variability is good – and necessary. Adapting to subject content, assessment methods and student characteristics can serve learners well, and a sector in rapid transition in response to the challenges posed by GenAI is always going to be one in which practice is uneven. But where variability is driven not by pedagogy but by mismatched policies, lack of clarity for students, gaps in staff confidence and understanding, or differences in student access to and skills with AI tools, it becomes a fundamental risk to the conditions under which academic standards can be assured."