-
The end of multiple choice tests: using AI to enhance assessment
Authors:
Michael Klymkowsky,
Melanie M. Cooper
Abstract:
Effective teaching relies on knowing what students know-or think they know. Revealing student thinking is challenging. Often used because of their ease of grading, even the best multiple choice (MC) tests, those using research based distractors (wrong answers) are intrinsically limited in the insights they provide due to two factors. When distractors do not reflect student beliefs they can be igno…
▽ More
Effective teaching relies on knowing what students know-or think they know. Revealing student thinking is challenging. Often used because of their ease of grading, even the best multiple choice (MC) tests, those using research based distractors (wrong answers) are intrinsically limited in the insights they provide due to two factors. When distractors do not reflect student beliefs they can be ignored, increasing the likelihood that the correct answer will be chosen by chance. Moreover, making the correct choice does not guarantee that the student understands why it is correct. To address these limitations, we recommend asking students to explain why they chose their answer, and why "wrong" choices are wrong. Using a discipline-trained artificial intelligence-based bot it is possible to analyze their explanations, identifying the concepts and scientific principles that maybe missing or misapplied. The bot also makes suggestions for how instructors can use these data to better guide student thinking. In a small "proof of concept" study, we tested this approach using questions from the Biology Concepts Instrument (BCI). The result was rapid, informative, and provided actionable feedback on student thinking. It appears that the use of AI addresses the weaknesses of conventional MC test. It seems likely that incorporating AI-analyzed formative assessments will lead to improved overall learning outcomes.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
-
Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol
Authors:
James T. Laverty,
Sonia M. Underwood,
Rebecca L. Matz,
Lynmarie A. Posey,
Justin H. Carmel,
Marcos D. Caballero,
Cori L. Fata-Hartley,
Diane Ebert-May,
Sarah E. Jardeleza,
Melanie M. Cooper
Abstract:
Many calls to improve science education in college and university settings have focused on improving instructor pedagogy. Meanwhile, science education at the K-12 level is undergoing significant changes as a result of the emphasis on scientific and engineering practices, crosscutting concepts, and disciplinary core ideas. This framework of "three-dimensional learning" is based on the literature ab…
▽ More
Many calls to improve science education in college and university settings have focused on improving instructor pedagogy. Meanwhile, science education at the K-12 level is undergoing significant changes as a result of the emphasis on scientific and engineering practices, crosscutting concepts, and disciplinary core ideas. This framework of "three-dimensional learning" is based on the literature about how people learn science and how we can help students put their knowledge to use. Recently, similar changes are underway in higher education by incorporating three-dimensional learning into college science courses. As these transformations move forward, it will become important to assess three-dimensional learning both to align assessments with the learning environment, and to assess the extent of the transformations. In this paper we introduce the Three-Dimensional Learning Assessment Protocol (3D-LAP), which is designed to characterize and support the development of assessment tasks in biology, chemistry, and physics that align with transformation efforts. We describe the development process used by our interdisciplinary team, discuss the validity and reliability of the protocol, and provide evidence that the protocol can distinguish between assessments that have the potential to elicit evidence of three-dimensional learning and those that do not.
△ Less
Submitted 6 October, 2016; v1 submitted 26 April, 2016;
originally announced April 2016.
-
Develo** the Next Generation of Physics Assessments
Authors:
James T. Laverty,
Melanie M. Cooper,
Marcos D. Caballero
Abstract:
Science education at all levels is currently undergoing dramatic changes to its curricula and develo** assessments for these new curricula is paramount. We have used the basis of many of these new changes (scientific practices, crosscutting concepts, and core ideas) to develop sets of criteria that can be used to guide assessment development for this new curriculum. We present a case study that…
▽ More
Science education at all levels is currently undergoing dramatic changes to its curricula and develo** assessments for these new curricula is paramount. We have used the basis of many of these new changes (scientific practices, crosscutting concepts, and core ideas) to develop sets of criteria that can be used to guide assessment development for this new curriculum. We present a case study that uses the criteria we have developed to revise a traditional physics assessment item into an assessment item that is much more aligned with the goals of current transformation efforts. Assessment items developed using this criteria can be used to assess student learning of both the concepts and process of science.
△ Less
Submitted 23 October, 2015; v1 submitted 2 July, 2015;
originally announced July 2015.
-
Engaging Physics Faculty in Course Transformation
Authors:
James T. Laverty,
Stuart H. Tessmer,
Melanie M. Cooper,
Marcos D. Caballero
Abstract:
Research has shown that it is challenging to change departmental culture to support the adoption of transformed teaching practices. Michigan State University is working to transform its introductory physics courses by engaging faculty in discussions about the scientific practices and core ideas of the discipline, and to use those discussions to drive change to both assessments and classroom instru…
▽ More
Research has shown that it is challenging to change departmental culture to support the adoption of transformed teaching practices. Michigan State University is working to transform its introductory physics courses by engaging faculty in discussions about the scientific practices and core ideas of the discipline, and to use those discussions to drive change to both assessments and classroom instruction. Here we discuss our model of change, its implementation in the Physics and Astronomy department, the challenges encountered along the way and how we've mitigated those challenges, and tools to measure the impact of this change.
△ Less
Submitted 30 September, 2014; v1 submitted 11 July, 2014;
originally announced July 2014.
