Hostname: page-component-78c5997874-j824f Total loading time: 0 Render date: 2024-11-02T22:55:28.094Z Has data issue: false hasContentIssue false
  • English
  • Français

Image statistics at the point of gaze during human navigation

Published online by Cambridge University Press:  01 January 2009

CONSTANTIN A. ROTHKOPF  and
DANA H. BALLARD
CONSTANTIN A. ROTHKOPF*
Affiliation:
Center for Visual Science, Department of Brain and Cognitive Sciences, University of Rochester, Rochester, New York Frankfurt Institute for Advanced Studies, Johann Wolfgang Goethe University, Frankfurt, Germany
DANA H. BALLARD
Affiliation:
Department of Computer Science, University of Texas at Austin, Austin, Texas
*
*Address correspondence and reprint requests to: Constantin A. Rothkopf, Frankfurt Institute for Advanced Studies, Johann Wolfgang Goethe University, Ruth-Moufang-Str. 1, 60438 Frankfurt, Germany. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Theories of efficient sensory processing have considered the regularities of image properties due to the structure of the environment in order to explain properties of neuronal representations of the visual world. The regularities imposed on the input to the visual system due to the regularities of the active selection process mediated by the voluntary movements of the eyes have been considered to a much lesser degree. This is surprising, given that the active nature of vision is well established. The present article investigates statistics of image features at the center of gaze of human subjects navigating through a virtual environment and avoiding and approaching different objects. The analysis shows that contrast can be significantly higher or lower at fixation location compared to random locations, depending on whether subjects avoid or approach targets. Similarly, significant differences in the distribution of responses of model simple and complex cells between horizontal and vertical orientations are found over timescales of tens of seconds. By clustering the model simple cell responses, it is established that gaze was directed toward three distinct features of intermediate complexity the vast majority of time. Thus, this study demonstrates and quantifies how the visuomotor tasks of approaching and avoiding objects during navigation determine feature statistics of the input to the visual system through the combined influence on body and eye movements.

Keywords

Active visionImage statisticsGaze selectionOptimal coding
Type
Natural Scene Statistics and Natural Tasks
Information
Visual Neuroscience , Volume 26 , Issue 1 , January 2009 , pp. 81 - 92
Copyright
Copyright © Cambridge University Press 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Attneave, F. (1954). Some informational aspects of visual perception. Psychological Review 61, 183–193.CrossRefGoogle ScholarPubMed
Baddeley, R., Abbott, L.F., Booth, M.J.A., Sengpiel, F., Freeman, T., Wakeman, E.A. & Rolls, E.T. (1997). Responses of neurons in primary and inferior temporal visual cortices to natural scenes. Proceedings of the Royal Society London B 264, 1775–1783.CrossRefGoogle ScholarPubMed
Balboa, R.M. & Grzywacz, N.M. (2003). Power spectra and distribution of contrasts of natural images from different habitats. Vision Research 43, 2527–2537.CrossRefGoogle ScholarPubMed
Ballard, D.H. (1991). Animate vision. Artificial Intelligence Journal 48, 57–86.CrossRefGoogle Scholar
Ballard, D.H., Hayhoe, M.M. & Pelz, J.B. (1995). Memory representations in natural tasks. Journal of Cognitive Neuroscience 7, 68–82.CrossRefGoogle ScholarPubMed
Ballard, D.H., Hayhoe, M.M., Pook, P. & Rao, R. (1997). Deictic codes for the embodiment of cognition. Behavioral and Brain Sciences 20, 723–767.Google Scholar
Barlow, H.B. (1961). Possible principles underlying the transformation of sensory messages. In Sensory Communication, ed. Rosenblith, W.A., pp. 217–234. Cambridge, MA: MIT Press.Google Scholar
Bell, A.J. & Sejnowski, T.J. (1997). The ‘independent components’ of natural scenes are edge filters. Vision Research 37, 3327–3338.CrossRefGoogle ScholarPubMed
Clifford, C.W.G., Webster, M.A., Stanley, G.B., Stocker, A.A., Kohn, A., Sharpee, T.O. & Schwartz, O. (2007). Visual adaptation: Neural, psychological and computational aspects. Vision Research 47, 3125–3131.CrossRefGoogle ScholarPubMed
Daugman, J.G. (1985). Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America A 2, 1160.CrossRefGoogle ScholarPubMed
David, S.V., Vinje, W.E. & Gallant, J.L. (2004). Natural stimulus statistics alter the receptive field structure of V1 neurons. Journal of Neuroscience 24, 6991–7006.CrossRefGoogle ScholarPubMed
Dayan, P. & Abbott, L.F. (2001). Theoretical Neuroscience. Cambridge, MA: The MIT Press.Google Scholar
Field, D.J. (1987). Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America A 4, 2379–2394.Google Scholar
Findlay, J.M. & Gilchrist, I.D. (2003). Active Vision: The Psychology of Looking and Seeing. Oxford: Oxford University Press.CrossRefGoogle Scholar
Frazor, R.A. & Geisler, W.S. (2006). Local luminance and contrast in natural images. Vision Research 46, 1585–1598.CrossRefGoogle ScholarPubMed
Freeman, W.T. & Adelson, E.H. (1991). The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 13, 891–906.Google Scholar
Harel, A., Ullman, S., Epshtein, B. & Bentin, S. (2007). Mutual information of image fragments predicts categorization in humans: Electrophysiological and behavioral evidence. Vision Research 47, 2010–2020.CrossRefGoogle ScholarPubMed
Hayhoe, M. & Ballard, D. (2005). Eye movements in natural behavior. Trends in Cognitive Science 9, 188–194.CrossRefGoogle ScholarPubMed
Hayhoe, M., Shrivastava, A., Mruczek, R. & Pelz, J. (2003). Visual memory and motor planning in a natural task. Journal of Vision 3, 49–63.CrossRefGoogle Scholar
Henderson, J.M., Brockmole, J.R., Castelhano, M.S. & Mack, M. (2007). Visual saliency does not account for eye movements during visual search in real-world scenes. In Eye Movements: A Window on Mind and Brain, ed. van Gompel, R., Fischer, M., Murray, W. & Hill, R., pp. 537–562. Oxford: Elsevier.CrossRefGoogle Scholar
Henderson, J.M. & Hollingworth, A. (1999). High-level scene perception. Annual Review of Psychology 50, 243–271.Google Scholar
Itti, L. (2005). Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Visual Cognition 12, 1093–1123.CrossRefGoogle Scholar
Itti, L., Koch, C. & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 1254–1259.CrossRefGoogle Scholar
Johansson, R.S., Westling, G., Bäckström, A. & Flanagan, J.R. (2001). Eye-hand coordination in object manipulation. The Journal of Neuroscience 21, 6917–6932.Google Scholar
Koch, C. & Ullman, S. (1985). Shifts in selective visual attention: Towards the underlying neural circuitry. Human Neurobiology 4, 219–227.Google Scholar
Land, M. & Hayhoe, M. (2001). In what ways do eye movements contribute to everyday activities? Vision Research, Special Issue on Eye Movements and Vision in the Natural World 41, 3559–3566.Google ScholarPubMed
Land, M.F. & Lee, D.N. (1994). Where we look when we steer. Nature 369, 742–744.CrossRefGoogle ScholarPubMed
Laughlin, S.B. (1981). A simple coding procedure enhances a neuron’s information capacity. Zeitschrift für Naturforschung 36c, 910–912.CrossRefGoogle Scholar
Lippert, J. & Wagner, H. (2002). Visual depth encoding in populations of neurons with localized receptive fields. Biological Cybernetics 87, 249–261.CrossRefGoogle ScholarPubMed
Mallot, H.A. (2000). Computational Vision. Cambridge, MA: MIT Press, Bradford Books.Google Scholar
Mannan, S.K., Ruddock, K.H. & Wooding, D.S. (1996). The relationship between the location of spatial features and those of fixations made during visual examination of briefly presented images. Spatial Vision 10, 165–188.Google Scholar
Najemnik, J. & Geisler, W.S. (2005). Optimal eye movement strategies in visual search. Nature 434, 387–391.CrossRefGoogle ScholarPubMed
Navalpakkam, V. & Itti, L. (2007). Search goal tunes visual features optimally. Neuron 53, 605–617.CrossRefGoogle ScholarPubMed
Olshausen, B.A. & Field, D.J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research 37, 3311–3325.CrossRefGoogle Scholar
O’Regan, J.K. & Noe, A. (2001). A sensorimotor approach to vision and visual consciousness. Behavioral and Brain Sciences 24, 939–973.Google Scholar
Parkhurst, D.J. & Niebur, E. (2003). Scene content selected by active vision. Spatial Vision 16, 125–154.Google Scholar
Rao, R. & Ballard, D.H. (1999). Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience 2, 79–87.Google Scholar
Reinagel, P. & Zador, A.M. (1999). Natural scene statistics at the centre of gaze. Network: Computations in Neural Systems 10, 341–350.Google Scholar
Ringach, D.L. (2002). Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. Journal of Neurophysiology 88, 455–463.Google Scholar
Ruderman, D.L. & Bialek, W. (1994). Statistics of natural images: Scaling in the woods. Physical Review Letters 73, 814–817.CrossRefGoogle ScholarPubMed
Ruderman, D.L., Cronin, T.W. & Chiao, C.-C. (1998). Statistics of cone responses to natural images: Implications for visual coding. Journal of the Optical Society of America A 15, 2036–2045.Google Scholar
Schwartz, O. & Simoncelli, E.P. (2001). Natural signal statistics and sensory gain control. Nature Neuroscience 4, 819–825.Google Scholar
Simoncelli, E.P. & Olshausen, B.A. (2001). Natural image statistics and neural representation. Annual Review of Neuroscience 24, 1193–1216.CrossRefGoogle ScholarPubMed
Stoica, P. & Moses, R.L. (1997). Introduction to Spectral Analysis. Upper Saddle River, NJ: Prentice Hall.Google Scholar
Tadmor, Y. & Tolhurst, D.J. (2000). Calculating the contrasts that retinal ganglion cells and LGN neurones encounter in natural scenes. Vision Research 40, 3145–3157.CrossRefGoogle ScholarPubMed
Ullman, S., Vidal-Naquet, M. & Sali, E. (2002). Visual features of intermediate complexity and their use in classification. Nature Neuroscience 5, 682–687.Google Scholar
van Hateren, J.H. & van der Schaaf, A. (1998). Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings of the Royal Society London B 265, 359–366.CrossRefGoogle ScholarPubMed
Yarbus, A. (1967). Eye Movements and Vision. New York, NY: Plenum Press.CrossRefGoogle Scholar
Zhu, S.-C. (2003). Statistical modeling and conceptualization of visual patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 691–712. doi: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2003.1201820.Google Scholar