I Know What You've Been Looking At! Testing Visual Recognition of Photographs to Retroactively Track Visual Attention in Online Visual Search Tasks

PAIS Building, Colloquium Room

Tracking eye movements is a useful tool for understanding the relationship between what we seek and what attracts our attention. Without eye-tracking, we might know that certain tasks are more difficult without knowing why or where the bottlenecks are in visual processing. When the pandemic forced us to shift to remote data collection on visual search tasks (akin to Where's Waldo), we developed a resourceful way to track visual attention without recording eye movements. By leveraging the fact that people are more likely to remember objects they spend more time looking at, we measured recognition memory for images as a residual trace of what people paid attention to. In two experiments using realistic photos, observers looked for groups of "target" images. We measured recognition of "distracting" images at the end of the experiment to see how the number and variety of sought-after target images influenced observers' ability to focus their attention. Experiment One revealed that a benefit to distractor recognition comes from searching for multiple, unrelated images (a puppy, a car, and a wallet, for example). This suggests that when you simultaneously look for many unrelated objects your ability to ignore irrelevant visual information is impaired. Experiment Two probed the generality of this effect by including distractor images that were from the same category, or categories, as the targets. Images from random categories were used as distractors in equal proportion to images from the target category(ies). Although multiple- and single-category searches were equally slow and promoted high recognition of within-category distractors, recognition of random distractors reduced in single-category search. This confirmed prior research showing attention is drawn to images that resemble targets. It appears that single-category search permits observers to narrowly focus their attention by ignoring random distractors but that distinguishing targets from distractors within the same category is easier when searching for only one image per category. Testing visual recognition might prove useful in evaluating the ease of use for applications, websites, or physical spaces when fine-grained measurements such as eye-tracking could be impractical. Recognition of random details within a website might indicate an opportunity to reduce user difficulty.