Glossary:
Closed-book examination: a time-constrained assessment task, usually with unseen questions, where students cannot use any additional resources.
Open-book examination: a time-constrained assessment task, usually with unseen questions, where students can bring in books or folders
Stakes: a term commonly used to indicate the importance of an assessment task. A high stakes task represents a higher proportion of the final award, and a low stakes task has less value in relation to the final award. The Credit value of the module will indicate its importance.
What does the literature say?
There has been an interesting review study published recently on closed and open book examinations (Permzadian & Cho, 2023), and another on high stakes (ie counting for a lot of a student’s grade) examination (French et al., 2023). I thought I would have a dive into these and a few others which didn’t make it into these studies, to see if there are any general recommendations for teachers and examiners. Both studies are well worth a read – they both look at purpose and outcomes, and provide a really balanced view of the impacts of different types of examinations.
In addition to these, I had a look at some papers on open book examination in different disciplines which reflect on open book examinations which were implemented to replace closed book examinations during the Covid pandemic, and which are intended to help teachers decide whether to continue with such examinations (Driessen et al., 2022; Roberts & Berry, 2023).
There have also been some recent papers comparing takehome examinations with on-site examinations, which I will try to have a look at later. For now, I want to focus on those examinations which are constrained by place as well as time, although I suspect there may be some overlap with some of the papers used in these review articles.
Closed book versus open book
Permzadian and Cho point out that these different types of examination use different cognitive processes: retrieval from memory for the closed book exam, and open book better for assessing knowledge transfer and application. As so often, we come back to the need to decide on purpose before designing the task, something discussed at length in my book and in these posts on grades and enjoyment of assessment the blog.
They conclude that closed book exams are good for cumulative exams, that is, exams where students need to remember key information from one examination to the next, because they it takes longer for students to forget the knowledge than it does for open book exams. However, students overwhelmingly prefer open book exams, and there is clear evidence they are better for student wellbeing and anxiety. They suggest that allowing a single page of notes prepared by the student – is a good compromise between closed and open book exams. Which is a good excuse to share Juuso Nieminen’s epic version again (Nieminen, undated).
The other papers, which are generally case studies mainly based on student perceptions of different types of exams, back up the findings from this review. Roberts and Berry found that students believed open book exams improved their development of higher order skills, and that students prepared differently for them. This was also found by Permzadian and Cho, who found that students tended to use less effective study strategies for open book exams. Both sets of authors recommend that preparation is embedded in the course teaching, to improve the efficacy of student study.
Focusing on Biology, Driessen et al echoed the findings of Permzadian and Cho: check purpose, and evaluate the design AND implementation of exams, explaining the difference between open and closed book exams and their differing purposes, to help improve student understanding and self-regulation.
High Stakes examinations
French et al have done a very good review of high-stakes examinations and they debunk many of the traditional arguments for keeping them. The discussion of validity in the assessment context is particularly valuable, and should make plenty of people think about some long-held beliefs. If people want to keep controlled closed book examinations, then they suggest that there should be several low stakes examinations, rather than one single chance at the end of the course. This is mainly because of the potential unfairness of these high stakes single shot exams, which “have exclusionary effects and disadvantage marginalised student groups”, and the increase in stress and anxiety they cause. To summarise, they say “testing may assist with knowledge retention, high-stakes summative examinations are ultimately ill-suited to deliver benefits of test-enhanced learning because they do not involve repetition, occur towards the end of the learning process, involve large amounts of content that is difficult to retain, and encourage rote learning.”
Recommendations
I am aware that this might be confirmation bias, but I am even less convinced that high stakes controlled examinations are useful than I was before, and that’s saying something. I would take away from this that a series of short tests is the way to examine knowledge. If you need to test application, then there are many other ways to do it, and you can still control the conditions, but you can have seen questions where the students can prepare, allow open book/resources, or use practical or clinical situations of different types. The right hand column in this figure has a selection of ways to do this, all described in detail in part 3 of my book.
From the book, sorry about the typo in the table.
Whatever you choose, it is always important is to design your assessment task carefully. Start with the purpose(s), align the task with the intended learning outcomes and provide structured support to students as they prepare. Don’t just assume they know how to do it or what it is expected of them.
References
Driessen, E. P., Beatty, A. E., & Ballen, C. J. (2022). Evaluating open-note exams: Student perceptions and preparation methods in an undergraduate biology class. PLOS ONE, 17(8), e0273185. https://doi.org/10.1371/journal.pone.0273185
French, S., Dickerson, A., & Mulder, R. A. (2023). A review of the benefits and drawbacks of high-stakes final examinations in higher education. Higher Education. https://doi.org/10.1007/s10734-023-01148-z
Permzadian, V., & Cho, K. W. (2023). Does the format of an assessment (closed book or open book) affect learning? A systematic review of the literature. Teaching in Higher Education, 1-18. https://doi.org/10.1080/13562517.2023.2289128
Roberts, L., & Berry, J. (2023). Should open-book, open-web exams replace traditional closed-book exams in STEM? An evaluation of their effectiveness in different disciplines. Journal of Learning Development in Higher Education(28). https://doi.org/10.47408/jldhe.vi28.1030
Assessment in Higher Education 
One thought on “Controlled Assessments (examinations): what should we think about?”