Representing models
Representing the data from the elicitation procedures described earlier is usually performed through quantitative analysis that produces network diagrams. A variety of procedures have been shown effective, the most popular being multidimensional scaling (cognitive mapping) and cluster analysis. Resulting diagrams visually indicate presence of concepts and relatedness among these (see diagrams below). Multidimensional scaling, a technique for cognitive mapping, produces a geometric spatial representation of similarity ratings or card sort data. Physical proximity on the graph equates to how the individual connects those concepts in their mental model, so items that are perceived as similar or related in some way are more closely grouped together on the graph. Primarily used in psychology, MDS is designed to help the researcher discover psychological perceptions underneath the data. As explained by Borg and Groenen (1997), “…when used in an exploratory manner, MDS thus typically carried with it, as an implicit purpose, the search for “underlying dimensions” that would explain observed similarities or dissimilarities…in the kind of exploratory MDS that is typical for psychologists the researcher is interested in discovering psychological dimensions that would meaningfully explain the data” (p. 9). MDS is only one statistical method for analyzing and producing cognitive maps, but it is the most common. (See the card sort analysis page on this site for information on how to process sort data in MDS). The assumptions behind cognitive mapping is found from Fenker (1975) (cited in Jonassen et al., 1993):
- Information about a topic area is organized and interpreted on the basis of a set of dimensions which represent organizational features of the topic area.
- These dimensions can be represented in n-dimension geometric space.
- There are many relationships that can exist among concepts. (p. 62)
Cognitive mapping is effective for comparing knowledge structures of experts and novices, getting a “picture” of how students perceive content domain material, and for other similar applications that can inform instructors and students of class progress. They have even shown to be practical alternatives for classroom assessment instead of basic recall and essay exams (Jonassen et al., 1993). Whereas essay exams are subject to grader bias and subjective interpretation, cognitive maps “provide a relatively objective measure of an individual’s cognitive structure in a given content area” (Jonassen et al., 1993, p. 70). Some additional advantages for employing cognitive mapping include (Jonassen et al., 1993):
- Effective for assessment of students’ understanding of higher-level knowledge.
- Can measure knowledge structures for a group of people, such as students in a class for instructional assessment and feedback.
- Similarity rating exams are easier to create than multiple choice exams. (p. 70)
MDS cognitive map (shaded areas added by investigator)
Cluster analysis is another popular method for representing sort data for analysis, though many find it somewhat difficult to detect patterns in more complex cluster trees. Cluster analysis output can be generated in statistics software; xSort card sort software features a built-in cluster analysis feature. Cluster connections are indicated from sort terms that were placed in the same piles, and hierarchies represent the complexities of multiple sorts where cards were placed in differing piles. Many psychologists are familiar with cluster analysis and can aid in interpreting diagrams in the event one is not well versed in how they represent data.
Pathfinder nets are another method for model representation, but this technique is not currently used much as far as can be determined. Pathfinder nets represent structural knowledge by connecting concepts (nodes) with links. The stronger the relationship between two concepts, as obtained from similarity ratings data, the shorter the link on the resulting graph. The algorithm defines a minimum level of relatedness among concepts, so not all relationships will be graphed; this is different from the dimensional graphing of cognitive maps that provides a more overall picture of connections. Pathfinder nets focus on lower-level comparisons between specific concepts, providing links that are more meaningful. It should be noted that the diagram does not indicate any hierarchy of concepts, which might be first assumed by looking at a Pathfinder net result. Pathfinder nets (PFNets) are based on a specific software algorithm found in a program called PC-KNOT (Knowledge Network and Orientation Tool). They are ideal for comparison of structural knowledge and have been used extensively for expert/novice comparisons. They have also been used to compare how people from different backgrounds or opinions view a certain subject (Jonassen et al., 1993). (Author’s note: I have not determined whether PC-KNOT software has been updated, having tried using an MS-DOS version a number of years ago.) A few advantages for employing PFNet analysis include:
- Identifying meaningful links between concepts.
- Comparing experts and novices.
- Combining knowledge structures from multiple individuals, such as obtaining an overall mental model from a group of experts.
- Can be easily produced through fully-automated software.
Figure 2.2. Pathfinder net of sports concepts (from Jonassen et al., 1993, p. 74).
