Effects of mental model quality on collaborative system performance

Abstract

As the tasks humans perform become more complicated and the technology manufactured to support those tasks becomes more adaptive, the relationship between humans and automation transforms into a collaborative system. In this system each member depends on the input of the other to reach a predetermined goal beneficial to both parties. Studying the human/automation dynamic as a social team provides a new set of variables affecting performance previously unstudied by automation researchers. One such variable is the shared mental model (Mathieu, Heffner, Goodwin, Salas, & Cannon-Bowers, 2000). This study examined the relationship between mental model quality and collaborative system performance within the domain of a navigation task. Participants navigated through a simulated city with the help of a navigational system performing at two levels of accuracy; 70% and 100%. Participants with robust mental models of the task environment identified automation errors when they occurred and optimally navigated to destinations. Conversely, users with vague mental models were less likely to identify automation errors, and chose inefficient routes to destinations. Thus, mental model quality proved to be an efficient predictor of navigation performance. Additionally, participants with no mental model performed as well as participants with vague mental models. The difference in performance was the number and type of errors committed. This research is important as it supports previous assertions that humans and automated systems can work as teammates and perform teamwork (Nass, Fog, & Moon, 2000). Thus, other variables found to impact human/human team performance might also affect human/automation team performance just as this study explored the effects of a primarily human/human team performance variable, the mental model. Additionally, this research suggests that a training program creating a weak, inaccurate, or incomplete mental model in the user is equivalent to no training program in terms of performance. Finally, through a qualitative model, this study proposes mental model quality affects the constructs of user self confidence and trust in automation. These two constructs are thought to ultimately determine automation usage (Lee & Moray, 1994). To validate the model a follow on study is proposed to measure automation usage as mental model quality changes.