Analysis
“Analyzing card sort data is part science, part magic. Analysis can be done in two ways: by looking for broad patterns in the data or by using cluster analysis software” (Maurer & Warfel, 2004, Analyzing the results).
Manually reviewing the resulting piles for each sort is the quickest means for getting feedback from the participants, making it quite feasible for instructors to employ during a busy academic year. Just look for emerging patterns in how students organized the concepts. Some will be simply inaccurate, others will reveal a perspective on how the concepts are related that differs from what would be considered “normal”. Don’t discredit these–they are an indicator that 1) instruction was not clear enough for all students, 2) some students did not invest enough effort to understand the material, 3) most importantly, different people filter new concepts and information through their own unique mental model of experiences. During one study a student presented a very different organization of the terms–but it wasn’t wrong, it was just very unique and represented a more philosophical approach as compared to the more technical sorts other students completed. This would never have been revealed during a standard exam, and was quite helpful in understanding the student as well as how the material was being processed by different individuals in the degree program.
Sort space screenshot for manual analysis
Sort results can also be analyzed through quantitative processing methods using Excel or statistics software. For example, cluster analysis and multidimensional scaling techniques produce cognitive maps that can reveal patterns among the data. This is described further on the Representing Models page. Also included here is a document excerpted from a study I did that employed MDS for analyzing sort data. It steps through the entire process (in mind-numbing detail) for those who would like to see how each component is actually accomplished. A few details in that study would be much easier now, however, such as getting the actual raw data into the stats software. The new xSort software application automatically outputs the data file ready for MDS processing–an immense time-saver. It also has a built-in cluster analysis output, requiring no use of stats software. Otherwise you get the idea of what’s involved. See the Resources page for more information and background on MDS and cognitive mapping, and below I’ve included a screenshot of both a cluster analysis and an MDS map.
Multidimensional scaling outputs this cognitive map – the points are spaced according to perceived similarity or dissimilarity (distance) from each other.
Cluster analysis provided by xSort.
Mike Lehr said,
September 29, 2008 at 12:38 pm
Barry-
Do we have access to the Xsort software for MDS applications?
Thanks
Mike
B Hill said,
September 29, 2008 at 1:57 pm
Yes, at LVC we have a site license for Xsort. Very easy to use. Mac-based, so this determines which lab to run it in. I’ll give you a call for specifics. Thanks Mike,
Barry