Computer-based Testing Facilities for Scalable, Excellent, and Secure STEM Assessment
Exams are a widely used method for summative assessment in college education, especially in introductory courses. However, at many universities, introductory courses are large (e.g., 200+ students). Running traditional pencil-and-paper exams (especially at this scale) presents management challenges that in practice prevent exams from reaching their potential as a means to improve learning.
In this xTalk, Dr Craig Zilles will present an effective, secure, and efficient alternative to traditional pencil-and-paper exams. At the Grainger College of Engineering at the University of Illinois, we've been running a Computer-Based Testing Facility (CBTF) for more than five years now and have been running at scale (30+ courses, 50,000+ exams/semester) for the past couple years. The CBTF is a proctored, "locked-down" computer lab that is operated as a service to courses.
Dr Zilles will discuss the basic operation of our CBTF and the four principles that make it work:
- Complex, authentic (e.g., numeric, programming, graphical, design) auto-gradable questions
- Writing question generators so that each student can be generated a unique exam
- Allowing students to schedule their exams at a time convenient to them
- Centrally managing proctoring so exam delivery is effectively free to courses
Dr Zilles will present studies demonstrating increased learning gains and reduced failure rates in courses using the CBTF and will discuss a range of ways that the CBTF has changed the way we teach, including more frequent, but smaller exams and the introduction of second-chance testing.
Additional details can be found in the follow research papers:
- Overview of the philosophy underlying testing facilities: http://zilles.cs.illinois.edu/papers/zilles_csedu_cbtf_2019.pdf.
- Detailed description of the operation of Zilles' facility: http://zilles.cs.illinois.edu/papers/zilles_cbtf_fie_2018.pdf.
Bio:
Craig Zilles (BS/MS MIT `95) is an Associate Professor of Computer Science at the University of Illinois. His research has spanned from haptic interfaces to computer architecture to his current focus on technology for education and learning analytics. He is an award winning instructor, having won a 2018 campus-level Award for Excellence in Undergraduate Instruction, an Illinois Student Senate Teaching Excellence Award, the IEEE Education Society's 2010 Mac Van Valkenburg Early Career Teaching Award. He is a recipient of an NSF CAREER award and best paper awards from ASPLOS in 2010 and 2013. He holds five patents.
Read MIT student Kevin Shao's personal reflections on this xTalk.