Toward an understanding of optimal performance within a human-automation collaborative system: Effects of error and verification costs

Abstract

Automated products, especially automated decision aids, have the potential to improve the lives of older adults by supporting their daily needs. Although automation seems promising in this arena, there is evidence that humans, in general, tend to have difficulty optimizing their behavior with a decision aid, and older adults even more so. In a human-automation collaborative system, the ability to balance costs involved in relying on the automation and those involved in verifying the automation is essential for optimal performance and error minimization. Thus, this study was conducted to better understand the processes associated with balancing these costs and also to examine age differences in these processes. Cost of reliance on automation was evaluated using an object counting task. Participants were required to indicate the number of circles on a display, with support coming from a computer estimate decision aid. They were instructed to rely on the aid if they believed its answer or verify the aid by manually counting the circles on the screen if they did not believe the aid to be correct. Manipulations in this task were the cost of a wrong answer, either -5, -10, -25, or -50 points and the cost of verification, either high or low. It was expected that participants would develop a general pattern of appropriate reliance across the cost conditions, but would not change their reliance behavior enough to reach optimality. Older adults were expected to rely on the decision aid to a lesser extent than younger adults in all conditions, yet rate the automation as being more reliable. It was found that older and younger adults did not show large differences in reliance, although older adults tend to be more resistant to changing their reliance due to costs than younger adults. Both age groups significantly underutilized the computer estimate, yet overestimated its reliability. The results are important because it may be necessary to design automated devices and training programs differently for older adults than for younger adults, to direct them towards an optimal strategy of reliance.