What is mental model assessment?
How do we really know what students are getting from our courses? Do they truly understand the material, or are they merely memorizing content? Elicitation of students’ mental models is a powerful approach that goes beyond traditional assessment methods. Mental models represent how well an individual organizes content in meaningful ways. Model analysis reveals inaccuracies and omissions that are crucial for deep understanding and application of course material, thus informing improvements in course design.
Learning theory research has revealed that individual students process and organize information in unique ways affected by their own particular experiences, cognitive abilities, beliefs, etc. Twenty students in a classroom will leave with twenty varying notions of what transpired during the course. Traditional testing methods do not effectively capture these differences, which are crucial to knowing whether students are “getting it” or not—the ultimate goal being to enable students to think and perform similar to experts in that field. In particular, we are most interested not merely in knowing whether they have memorized the content, but more crucially whether they are acquiring the higher-level skills required for advanced performance, problem-solving, and transfer of learning in a given domain. This requires meaningful organization of that content, making connections in ways that facilitate these skills.
Cognitive psychologists have developed several methods for attempting to discover how individuals “see” a particular subject. Mental models are internal conceptual and operational representations of a subject and are comprised of content knowledge (information), structural knowledge (the meaningful connections among that information), and procedural knowledge (doing useful work utilizing these connections). One effective technique is to elicit student models and compare with that of an expert. The expert model presumably demonstrates an accurate, highly developed representation of what the subject is about. Student models are inevitably inaccurate, incomplete, and unrefined, but by discovering what they look like, instruction can be revised to address common areas of confusion and error. There are several commonly employed techniques, but the card sort has proven highly reliable, easy to complete for both administrator and participant, and provides data that can be analyzed at various levels of detail.
Most of the information provided on this site will focus on card sorts as the primary tool for mental model assessment as this method is easy for most instructors to implement. Keep in mind, however, it’s not the only way, and works best when triangulated in combination with other methods. My intention here is to pass along what I’ve learned over the years of digging into the issues and techniques and applying them at the college where I teach. Feel free to comment and send me ideas or experiences you’ve had along these lines.
Barry R. Hill
Lebanon Valley College