Applications
General applications of mental model research
Mental models are perhaps most often thought to be relevant to the study of computer systems and design/use of technical devices. De Kleer and Brown (in Gentner & Stevens, 1983) are often cited regarding their examinations into how people build an understanding of mechanistic devices. Borgman (1986) investigated how users conceived the virtual structure of an electronic catalog system. Gray (1990) similarly examined mental model construction while users navigated through a hypertext information system. Norman (1983, 2002) has provided a useful terminology for comparing the models between the designer of a device, the user of a device, and the device itself. A designer has a model in mind when developing a device, though sometimes the implementation of the device itself may end up presenting a different conceptualization (system image) to the user. The user then develops their own conceptual model of how it works. The objective for designers (hardware and software) is to map the original model as closely as possible to the actual device and ensure that this model is conducive to fostering an accurate user model. “The mental model of a device is formed largely by interpreting its perceived actions and its visible structure” (Norman, 2002, p.17).
Though technology-related disciplines represent a significant share of mental model research, mental models are generally agreed to be applicable to most any content domain (Jonassen, 1995). Coll and Treagust (2002) compared high school, undergraduate, and graduate students’ mental models of ionic bonding. Borges and Gilbert (1999) examined mental models of electricity, one of many studies on people’s conceptions of physics concepts. Buzydlowski (2002) applied mental model elicitation techniques to determine how experts in the field of humanities literature compared and related well-known authors. Evans et al. (2004) compared models of team members to determine how to improve interaction and foster like-minded decision-making processes for military purposes. All of these studies revealed differences in structural knowledge where more advanced students or skilled performers possessed a more highly organized, interrelated awareness of concepts in the content domain.
Mental model research in education and training
Mental model techniques have been applied to various aspects of education and training. Measuring outcomes of instruction, either for comparison with experts or for official assessment, has been employed extensively. One of the most popular techniques for assessing students’ conceptions of content matter is through concept mapping, a method for representing knowledge structures. Williams (1995) examined varying instructional approaches to the teaching of calculus by comparing concept maps of instructors and students, finding that instructor and student maps varied considerably in core concept organization and awareness. Freeman and Urbaczewski (2002) used concept mapping as an enhancement to traditional assessment by examining the mental models of undergraduate students in a telecommunications course. By having undergraduate students draw concept maps at three points during the semester, a positive progression of development was easily seen in student mental models of the subject content, especially when compared with a referent expert model. Using different techniques, Scielzo et al. (2002) also dealt with assessing learners’ developing mental models in a training environment to determine how accurately they represented the knowledge structures being acquired. Quantitative methodology concluded that accuracy of one’s mental model directly correlates to performance ability, knowledge acquisition, and instructional efficiency.
Tools used for representing mental models can also be effective throughout the instructional process by helping learners gradually build appropriate knowledge structures of the subject matter. Richard Mayer has advocated that instruction can be designed specifically to help learners construct meaning and therefore develop mental models of the subject, even without “hands-on” opportunities (1989, 1999). His SOI model of learning (selecting, organizing, and integrating) specifically directs learners’ attention to the significant concepts and shows how they relate, the goal being to help them form a mental picture of this organization that is encoded into long-term memory. This is provided through specific diagrams and other materials carefully designed to make such connections (such as cause and effect) explicit. Enger (1998) qualitatively examined concept maps drawn by 7th graders before and after an instructional unit and documented increased conceptual understanding after instruction. Similarly, Kinchin and Hay (2000) describe a qualitative approach to evaluating student concept maps, having students begin diagramming maps early in instruction. These are used as an ongoing tool to help them self-monitor and develop their knowledge base as well as provide the instructor with feedback on how students perceive content. Clement and Steinberg (2002) detail a learning aloud case study featuring a student who was presented with an electric circuit problem to solve. Interactive discussion between the instructor and student helped the individual’s mental model evolve into a more accurate and appropriate knowledge structure in the content domain. Macklin (2003), in a discussion paper directed toward librarians, discusses how to apply constructivist learning concepts when developing instruction for academic library usage. His ideas are based on David Jonassen’s constructivist learning environments (CLEs) where various experiences and problem solving processes work together to build each learner’s particular mental model. For complex problems, the issue is helping students become aware of their own thinking (metacognition) to determine how accurate their model is for solving problems (finding information in the library, for example), thus enabling them to successfully confront future questions as they arise.
Another approach, though not well-grounded in the literature, is to have students simply draw their mental models as they attempt to diagram what they see in their mind about the subject (Glynn, 1997). These were then used more as a diagnostic tool rather than official assessment, thereby providing feedback for instructional decisions and strategies. Part of a more formal study conducted by Gray (1990) involved having participants draw diagrams showing their personal conceptions of how a hypertext information system was structured. This was an attempt to allow participants to articulate what was in their heads, which can be difficult to accomplish verbally (Borgman, 1986). In this example, most early drawings resembled linear document (book) models, but as users spent more time navigating through the electronic information their models gradually morphed, though in various directions. The virtual nature of a hypertext information system makes it difficult for new users to comprehend how it is structured, therefore the drawings were quite revealing in how different individuals perceived how such a system operated. Methodologically, the diagrams were a complementary approach to a procedural think-aloud protocol method, which enabled a clearer overall picture.
Fiore, Cuevas, and Oser (2003) manipulated the inclusion of diagrams during training to determine resulting effects on task performance and mental model development. The question of whether performance improvement would result by providing a design model of a system during instruction has also been studied, for example by Kieras and Bovair (1984) who compared providing system diagrams against rote procedural training during instruction to see if they helped users learn how to operate a simple control panel. They found that the model group “learned procedures faster, retained them more accurately, executed them faster (and) simplified inefficient procedures…” (p. 255). Studies from Cuevas, Fiore, & Oser (2002), Gyselinck and Tardieu (1999), and Mayer (1989) support this notion.
Though much research has shown that providing learners with appropriate system or conceptual models during instruction can be beneficial, the presence of a mental model may not be helpful for simple tasks and may possibly even impede performance (Borgman, 1986; Halasz & Moran, 1983). The power of a mental model is to predict the performance or outcome of a system or action. This requires “running the model” using data imbedded within the individual’s knowledge structure. The model then extrapolates this information to form conclusions. For rote procedural actions that are straightforward, this process can get in the way (Parush, 2004).
Kieras and Bovair (1984) discovered that though providing system models to learners can be helpful, there are a few issues to keep in mind:
- The model must support inferences about how the system works in detail. Thus general metaphors or analogies do not support the user to be able to infer performance or operation of the system.
- Any relevant information on how the system works (as opposed to how to work it) does not need to be very detailed or in-depth.
- If operation of a system does not require any inferring from basic information, then a model is not necessary. Using an ordinary telephone is an example where people do not need to really understand the system model in order to operate the device.
So, providing a system or conceptual model during instruction has been shown to improve performance, but there are issues to keep in mind. Mental models are useful in that they allow an individual to infer (predict) causes and results of a system. Therefore, instruction must provide enough information to allow inference of the system operation, but not necessarily all the detail (and yet not lacking in enough information to foster the ability to “run” their model).
Expert-novice mental model comparisons
Expertise is developed over time through extensive experiences with novel situations; this accumulation and assimilation of new knowledge and connections in turn facilitates a highly advanced network of structural knowledge that novices simply do not possess. Johnson (1988) cites several studies that indicate that it is this knowledge structure, or organization, that determines expert versus novice performance (Egan & Schwartz, 1979; Chase & Simon, 1973; Anderson, Spiro & Anderson, 1978) and that mental models are fundamental to expert development (Kuipers & Kassirer, 1984; Kieras & Bovair, 1984; Bouwman, 1983; Lajoie, 1986; Logan & Eastman, 1986; White & Frederiksen, 1987). “Expert knowledge consists of increased connectedness among critical concepts…” (Fiore et al., 2003, p. 193). It is important to note that the emphasis is on increased connections, not just new knowledge. The network that experts develop, implicitly as well as explicitly, is the key to their ability in making decisions and solving problems efficiently. Experts approach a novel task or problem by identifying relevant components and developing solutions while bypassing irrelevant information. Novices, while they may “know” some amount of information (declarative knowledge) about the subject, and may have even learned some procedural tasks, do not yet possess meaningful connections that facilitate such a process; therefore they must consider and process one bit of data at a time regardless of its actual importance, relevance, or sequence (Johnson, 1988; Landa, 1999). “Beginner’s knowledge is spotty, consisting of isolated definitions and superficial understandings of central terms and concepts. With experience, these items of information become structured, are integrated with past organizations of knowledge” (Glaser, 1989, p. 272). Villachica et al. (2001) cites other studies that have identified differences in knowledge structure sophistication between experts and novices. Some of these involve the game of chess, (De Groot, 1978; Chase & Simon, 1973; Reingold, Charness, Pomplun, & Stampe, 2001) others in the field of physics (Chi, Feltovich, & Glaser, 1981). In one of their own studies, Villachica et al. (2001) sought to elicit expert knowledge structures in the field of human performance technology. The objective was to see “how HPT experts organize their knowledge of the discipline” as well as to compare how HPT knowledge was organized differently between experts and novices (p. 437).
The most obvious application for expert-novice research is for informing instructional practice. Since the intent of most educational programs is to help learners begin to think and operate in a more mature way, i.e. like professionals in a particular field, it is first necessary to understand what experts know and are able to do. This proves more complex than might be first imagined. Typically when designing instruction educators rely on experts’ own ideas, whether from an author’s textbook, journals in the discipline, or the knowledge of the instructor himself, to determine the content and structure of a particular subject (Villachica et al., 2001; Fiore et al., 2003). However, understanding exactly how experts conceive of a particular subject and apply their expertise in solving problems and completing tasks is not easily nor completely accurately accomplished by relying on expert self-description. Often the individual does not consciously realize why or how they understand or do certain things. Some knowledge is implicitly learned, making it impossible for an individual to articulate. Other knowledge has become automated to the extent they no longer consider it during performance. Mental model elicitation and representation have shown to be effective at getting the “meat” of the mental knowledge structure an expert possesses (Villachica et al., 2001).
Therefore, it is logical to then take this and do two things: 1) compare to the mental models of novices, perhaps students or beginners in the field, to determine gaps and inconsistencies, and 2) to develop instructional materials that attempt to transfer expert knowledge structures to learners. Expert-novice mental model comparisons have been performed extensively in a variety of educational and training situations.
Faculty at the University of Illinois Urbana-Champaign needed to determine how in-service and student teachers conceptualized computer networks as part of an effort for improving technology literacy (Levin, Stuve, & Jacobson, 1996). They specifically wanted to know how teachers used computer networks in their everyday activities and what their mental models were for networks and related tasks. As described earlier, subjects actually did apply their own particular models of networks to task performance; these models were more structured for the experts than for novices. The outcomes were used to examine predominant models that might emerge which could then be incorporated into the educational program. Williams (1995) compared concept maps of instructors and students in two aspects: instructor vs student comparison as well as two different methods for calculus instruction. Results indicated that students’ mental models of the material were much less structured and sophisticated and lacked core concepts that instructors’ models showed.
Troubleshooting, operating complex systems, and solving novel problems require individuals to possess rich, accurate mental models of the task and content domain (Jonassen & Henning, 1999; Gott, Benett, & Gillet, 1986). Jonassen and Henning (1999) believe that mental models cannot be solely represented by content analysis of structural knowledge, as many elicitation techniques are based, but also by considering actual performance as an individual applies their knowledge in a meaningful way. They therefore have investigated mental models of experts and novices through a series of techniques targeted at content (presence of concepts and relationships) as well as procedure (how content is applied in situations). Although not unique in combining content and procedural methodology for analysis, this study in particular contrasts with many mental model investigations where typically researchers have utilized one or two methods for extracting participants’ models. Jonassen and Henning employed three techniques aimed at triangulating a rich model outcome. The results provided significant comparisons that showed that the participants who performed the slowest on the procedural tasks also generated Pathfinder nets featuring “fewer links, fewer levels…(that were) less integrated than that of the fastest performer” (p. 39). Similarly the slower performers could not provide well-structured descriptions of how the system worked (system images). This provides a much more complete picture that reinforces the results from each of the individual exercises.
Scott Johnson at the University of Illinois Urbana-Champaign has also investigated expert-novice troubleshooting performance (1988). He compared cognitive and behavioral (performance) knowledge of experienced and novice service technicians in the diagnosis of faulty complex equipment. The result, typical of expert-novice comparisons, is that the novices did not possess the rich structural knowledge that facilitates efficient and accurate analysis of a problem situation.
Fiore et al. (2003) studied the effects of taking expert models, developing corresponding diagrams, and incorporating them into instruction to “encourage the acquisition of knowledge structures more similar to an expert model” (Fiore et al., 2003, p. 188). They found that this was effective in terms of helping participants to “accurately draw connections”, and that the model diagrams “facilitated performance on measures of integrative knowledge” (p. 185). However, results did not seem to indicate performance improvements of declarative knowledge.
Another example of using experts to derive structural knowledge models that can be used to inform instructional design is from Diekhoff and Wigginton (1982). Working with college faculty in psychology, history and systems, and statistics, they employed multidimensional scaling to produce cognitive maps that were then used in class instruction. Test scores for students in these classes were superior to those in classes using traditional instruction.
The issue of team members and their ability to anticipate each other’s actions during a task was the focus of a potential expert/novice study designed to enhance military training efforts (Evans et al., 2004). Their hypothesis was that team members who possess similar mental models should perform at greater levels as a unit. This article merely describes how such a study would be conducted and does not include actual data and findings.
