Open Book Assessments
Why would we return to exclusive reliance on timed exams in the post-pandemic world?
Professor Philip Young Director of Undergraduate Studies, School of Life Sciences, University of Warwick
The pandemic caused a scramble throughout HE to move assessments on-line. Universities took several approaches to this, most notably 24-hr windows in which students sat timed exams. However, this approach posed issues with assessment integrity and opened the ethical minefield of exam proctoring.
It is timely for us to review how we assess. For several years, following open discussion with industrial partners, the Royal Society of Biology has been pushing for the introduction of more authentic assessments. However, the sector needs to truly review how “authentic” timed exams are and how they benefit students in the post-graduate world. Although problematic, the pandemic has allowed the traditional resistance to altering formats to be hurdled, meaning institutions are now in a better starting point to truly review how we should be assessing students.
Here at the School of Life Sciences (SLS, University of Warwick) we have always held final year exams using traditional formats (timed 3 hr exams, with students having to complete 2 essays). This format is robust, is defendable and ranks students effectively.
However, in March 2020 we took the left field decision to convert all our Y3 exams to open book assessments. The format was similar – releasing four essay-based questions per module and asking students to complete two essays. We gave students four weeks to complete their essays with an additional requirement for students to consolidate their taught material with independent research (evidenced by the inclusion of data from peer reviewed articles not covered in delivered sessions). The results were outstanding.
The standard of submitted work is much higher:
The assessments were converted to true capstone experiences, with students showcasing their knowledge and research skills, coupled with improved critical appraisal of primary data. The background reading element ensures students were using the assessments as a learning exercise, through which they were reviewing primary sources they would otherwise not have read, coupled with knowledge consolidation. Importantly, the changes meant we were asking students to produce mini-review articles, which is a transferable skill and a truly authentic assessment. Feedback from our academics was that the resulting essays were diverse, interesting, and substantially better those submitted under traditional timed exam settings.
Students love the approach:
Feedback from our 2020 and 2021 graduating cohorts is overwhelmingly positive around the assessment changes, stating the approach reduces stress and anxiety, removes the need to memories 15-20 lectures worth of material (which is not an authentic skill and discriminates against some of our neurodiverse students) and allows students to take ownership of their assessment structure, timetable and approach. Most importantly though, students reported it allowed them to truly showcase the skills they had acquired during their degree.
The approach maintains academic integrity:
Importantly, the open book approach removes the problems that plague on-line timed exams; there were no obvious issues with collusion and no need for proctoring software. All essays were submitted and assessed through Turnitin and screened by our Academic Integrity Committee. Of the >3000 submitted essays, only 25 needed to be reviewed for signs of plagiarism.
The approach did not produce grade inflation but did close the attainment gap:
The requirement for data analysis and background reading meant the assessments were robust. Importantly, there was no obvious grade inflation (one of the main arguments usually used to prevent introduction of open book assessments of this type). The changes also produced a closing of the attainment gap, although this needs a more nuanced review, because similar results have been reported throughout the sector and raises a red flag (if we are all offering an inclusive and welcoming environment, why are our BAME students able to work to a higher standard when completing assessments remotely?).
Engagement with pre-recorded lectures:
One major concern with the approach was that the new format meant students could get away with not engaging with the taught content, wait for essay titles to be released and then simply researching their two chosen topics during the assessment period. However, using learning analytics we have monitored how each student accessed taught material throughout the year, correlated this to assessment performance and shown that engagement throughout the year is significantly associated with better attainment and that the level of engagement also results in improved scores. This means students who tried to “game” the system have, in general, performed in the lower quartile.
As a result, SLS has decided to maintain the open book format for all Y3 final assessment, because the benefits are too visible to be ignored.