Preparation & administration
Selection of the terms is the primary task for developing a sort exercise. Look for terms/concepts that are relevant to some aspect you are investigating. This could be from a current unit in class, the entire course, or even for a degree program. Avoid a list of terms that are widely different–they should represent some cohesive aspect of the subject that can be organized in a clear way. There are also levels of abstraction to consider. At one level, sorts can be based on terms that are very concrete in their relationship. For example, if you were asking students to organize music composers by time period, there’s only one correct answer. This can be helpful, but it’s basic. You can create a more abstract, subjective sort based on higher-level conceptual perceptions. Examples include solving a business problem or opinions on current events. These will result in a wide range of possibilities, providing tremendous insight into how people are viewing the issue.
During the selection process sort the terms yourself to see how they organize. You may have a term that is better suited as a category header, and perhaps you want the participants to come up with that organizing concept themselves based on the terms they are piling together (you can ask participants to label each pile). You may also have a term that just doesn’t fit the operational or categorization level of the others. Make sure the exercise works for you, the expert, before administering to students.
Some options for administering sorts include whether to let participants define their own piles (open sort) or pre-determine pile headers for them to place terms into (closed sort). Most often the open-sort is preferred because it allows more freedom for students to organize as they see things–not as you want them to see it. This, of course, is the whole purpose for mental model analysis. So, just give them the list of terms and have them sort anyway they see fit. No rules, no guidelines–just ask them to sort terms that seem most similar into piles. You can allow them to create a “I don’t know” pile if you wish. This avoids having terms they don’t understand being placed into random piles, thereby skewing the model results. Have them label each pile with a descriptive name–this can be quite revealing to see what they actually call it.
By the way, make sure you let them know there is no grading involved and that their performance will not be judged in any way. They should not see this as yet another test or exam they have to survive.
Actual administration of the sort can be done with cards or software. Software applications provide tremendous benefits in terms of scoring and analysis. Manual card sorts take forever to create the cards and also require data compilation to score which cards were in which pile–for each participant. This data must then be entered into a computer for processing either in Excel or stats software. Even if you’re only manually reviewing the resulting sorts, software provides a permanent record of each sort by capturing the sort screen and printing it out. There are a few programs available today that are very well designed and easy to use. See the Resources page for a short list and links to their websites.
A common question asked when I present this method to faculty is how to handle large classes. Even with software, if 50 students are completing the exercise on 50 different computers, it takes a lot of time to grab the files and/or screenshots from each station. Optimal Sort, the web-based sort application can solve this problem since everything is managed from their website. You can also resort to sampling a portion of your class or program to reduce the numbers. Another option is having your IT folks use something like Apple’s Remote Management software that allows you to access an entire computer lab from one master location–you can install the software and collect all resulting sort files from one chair. Pretty nice. Remember, you won’t be doing this every week, but rather perhaps once or twice a semester, or even as an annual outcomes assessment tool. Start small to try it out and get a feel for it, then see how it may help you on a larger scale.
