Optimal visual search strategy with inter-saccade response correlations

Humans process the visual world with varying resolution across the visual field and sample information using eye movements to point the high-resolution central vision (fovea) to regions of interest. What eye movements optimize decisions in tasks such as target detection and localization? In 2005, Najemnik & Geisler proposed the Bayesian Ideal searcher (IS) that takes into account the foveated properties of human visual and employs the optimal fixation selection strategy to maximize visual search performance. In addition, they proposed a computationally simpler model, entropy limit minimization (ELM), that approximates the IS searcher. One limitation of these models is that they were developed with the assumption that the visual system's internal responses across fixations are statistically independent. This assumption will not hold for search tasks such as in 2D medical images for which the external noise and anatomical noise results in correlations in internal responses across saccadic fixations (inter-saccade response correlations). In this work, we present image-computable foveated IS and ELM models that accommodate inter-saccade response correlations. We demonstrate that for static images, the optimal searchers that account for the inter-saccade correlations (i.e., IS-COR and ELM-COR) significantly outperform the traditional methods that ignore such correlations. Moreover, the novel ELM-COR achieves a similar performance as the IS-COR but runs about 18 times faster. Together, the IS-COR and ELM-COR extend the optimal searcher framework to evaulate human fixations for more realistic search tasks with inter-saccadic correlations.