Hostname: page-component-78c5997874-m6dg7 Total loading time: 0 Render date: 2024-11-15T09:21:20.590Z Has data issue: false hasContentIssue false
  • English
  • Français

Sounds can affect visual perception mediated primarily by the parvocellular pathway

Published online by Cambridge University Press:  08 February 2010

PHILIP M. JAEKL  and
LAURENCE R. HARRIS
PHILIP M. JAEKL*
Affiliation:
Centre for Vision Research, York University, Toronto, Ontario, Canada
LAURENCE R. HARRIS
Affiliation:
Centre for Vision Research, York University, Toronto, Ontario, Canada
*
†Address correspondence and reprint requests to: Philip Jaekl, Centre for Brain and Cognition, Universitat Pompeu Fabra, C/Roc Boronat, 138, 08018 Barcelona, Spain. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

We investigated the effect of auditory–visual sensory integration on visual tasks that were predominantly dependent on parvocellular processing. These tasks were (i) detecting metacontrast-masked targets and (ii) discriminating orientation differences between high spatial frequency Gabor patch stimuli. Sounds that contained no information relevant to either task were presented before, synchronized with, or after the visual targets, and the results were compared to conditions with no sound. Both tasks used a two-alternative forced choice technique. For detecting metacontrast-masked targets, one interval contained the visual target and both (or neither) intervals contained a sound. Sound–target synchrony within 50 ms lowered luminance thresholds for detecting the presence of a target compared to when no sound occurred or when sound onset preceded target onset. Threshold angles for discriminating the orientation of a Gabor patch consistently increased in the presence of a sound. These results are compatible with sound-induced activity in the parvocellular visual pathway increasing the visibility of flashed targets and hindering orientation discrimination.

Keywords

Auditory–visualMultisensory integrationContrast detectionPerceptual enhancementVentral stream
Type
Research Articles
Information
Visual Neuroscience , Volume 26 , Issue 5-6 , November 2009 , pp. 477 - 486
Copyright
Copyright © Cambridge University Press 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alpern, M. (1953). Metacontrast. Journal of the Optical Society of America 43, 648–657.CrossRefGoogle ScholarPubMed
Ansorge, U., Klotz, W. & Neumann, O. (1998). Manual and verbal responses to completely masked (unreportable) stimuli: Exploring some conditions for the metacontrast dissociation. Perception 27, 1177–1189.CrossRefGoogle ScholarPubMed
Berger, T.D., Martelli, M. & Pelli, D.G. (2003). Flicker flutter: Is an illusory event as good as the real thing? Journal of Vision 3, 406–412.CrossRefGoogle ScholarPubMed
Bertelson, P. & Aschersleben, G. (2003). Temporal ventriloquism: Crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment. International Journal of Psychophysiology 50, 147–155.CrossRefGoogle ScholarPubMed
Bolognini, N., Frassinetti, F., Serino, A. & Ladavas, E. (2005). “Acoustical vision” of below threshold stimuli: Interaction among spatially converging audiovisual inputs. Experimental Brain Research 160, 273–282.CrossRefGoogle ScholarPubMed
Bookheimer, S.Y., Zeffiro, T.A., Blaxton, T.A., Gaillard, W.D., Malow, B. & Theodore, W.H. (1998). Regional cerebral blood flow during auditory responsive naming: Evidence for cross-modality neural activation. Neuroreport 9, 2409–2413.CrossRefGoogle ScholarPubMed
Brainard, D.H. (1997). The psychophysics toolbox. Spatial Vision 10, 433–436.Google Scholar
Breitmeyer, B. (1984). Visual Masking: An Integrative Approach. Oxford: Oxford University Press.Google Scholar
Breitmeyer, B.G. & Ganz, L. (1976). Implications of sustained and transient channels for theories of visual pattern masking, saccadic suppression, and information processing. Psychological Review 83, 1–36.CrossRefGoogle ScholarPubMed
Breitmeyer, B.G. & Ogmen, H. (2006). Visual Masking: Time Slices through Conscious and Unconscious Vision (2nd ed.). New York: Oxford University Press.CrossRefGoogle Scholar
Breitmeyer, B.G., Ro, T. & Singhal, N.S. (2004). Unconscious color priming occurs at stimulus- not percept-dependent levels of processing. Psychological Science 15, 198–202.Google Scholar
Bridgeman, B. (1980). Temporal response characteristics of cells in monkey striate cortex measured with metacontrast masking and brightness discrimination. Brain Research 196, 347–364.CrossRefGoogle ScholarPubMed
Burr, D.C., Ross, J. & Morrone, M.C. (1986). Seeing objects in motion. Proceedings of the Royal Society of London. Series B 227, 249–265.Google Scholar
Busse, L., Roberts, K.C., Crist, R.E., Weissman, D.H. & Woldorff, M.G. (2005). The spread of attention across modalities and space in a multisensory object. Proceedings of the National Academy of Sciences of the United States of America 102, 18751–18756.CrossRefGoogle Scholar
Callaway, E.M. (2005). Neural substrates within primary visual cortex for interactions between parallel visual pathways. Progress in Brain Research 149, 59–64.CrossRefGoogle ScholarPubMed
Calvert, G.A., Spence, C. & Stein, B.E. (2004). The Handbook of Multisensory Processes. Cambridge, MA: MIT.CrossRefGoogle Scholar
Calvert, G.A. & Thesen, T. (2004). Multisensory integration: Methodological approaches and emerging principles in the human brain. Journal of Physiology, Paris 98, 191–205.CrossRefGoogle ScholarPubMed
Celesia, G.G. (1976). Organization of auditory cortical areas in man. Brain 99, 403–414.CrossRefGoogle ScholarPubMed
Clark, V.P. & Hillyard, S.A. (1996). Spatial selective attention affects early extrastriate but not striate components of the visual evoked potential. Journal of Cognitive Neuroscience 8, 387–402.CrossRefGoogle Scholar
Clavagnier, S., Falchier, A. & Kennedy, H. (2004). Long-distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive, Affective & Behavioral Neuroscience 4, 117–126.Google Scholar
Colombo, M. & Gross, C.G. (1994). Responses of inferior temporal cortex and hippocampal neurons during delayed matching to sample in monkeys (Macaca fascicularis). Behavioral Neuroscience 108, 443–455.Google Scholar
Cornsweet, T.N. & Pinsker, H.M. (1965). Luminance discrimination of brief flashes under various conditions of adaptation. The Journal of Physiology 176, 294–310.CrossRefGoogle ScholarPubMed
De Gelder, B. & Bertelson, P. (2003). Multisensory integration, perception and ecological validity. Trends in Cognitive Sciences 7, 460–467.Google Scholar
Dehaene, S. (2003). The neural basis of the Weber-Fechner law: A logarithmic mental number line. Trends in Cognitive Sciences 7, 145–147.CrossRefGoogle ScholarPubMed
Desimone, R., Albright, T.D., Gross, C.G. & Bruce, C. (1984). Stimulus-selective properties of inferior temporal neurons in the macaque. The Journal of Neuroscience 4, 2051–2062.Google Scholar
Desimone, R., Fleming, J. & Gross, C.G. (1980). Prestriate afferents to inferior temporal cortex: An HRP study. Brain Research 184, 41–55.CrossRefGoogle ScholarPubMed
Distler, C. & Hoffmann, K.P. (1993). Visual receptive field properties in kitten pretectal nucleus of the optic tract and dorsal terminal nucleus of the accessory optic tract. Journal of Neurophysiology 70, 814–827.CrossRefGoogle ScholarPubMed
Driver, J. & Noesselt, T. (2008). Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57, 11–23.CrossRefGoogle ScholarPubMed
Driver, J. & Spence, C. (2000). Multisensory perception: Beyond modularity and convergence. Current Biology 10, R731–R735.CrossRefGoogle ScholarPubMed
Enns, J.T. & Di Lollo, V. (2000). What’s new in visual masking? Trends in Cognitive Sciences 4, 345–352.Google Scholar
Falchier, A., Clavagnier, S., Barone, P. & Kennedy, H. (2002). Anatomical evidence of multimodal integration in primate striate cortex. The Journal of Neuroscience 22, 5749–2759.CrossRefGoogle ScholarPubMed
Fendrich, R. & Corballis, P.M. (2001). The temporal cross-capture of audition and vision. Perception & Psychophysics 63, 719–725.Google Scholar
Foxe, J.J. & Schroeder, C.E. (2005). The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16, 419–423.Google Scholar
Foxe, J.J. & Simpson, G.V. (2002). Flow of activation from V1 to frontal cortex in humans. A framework for defining “early” visual processing. Experimental Brain Research 142, 139–150.CrossRefGoogle ScholarPubMed
Frassinetti, F., Bolognini, N. & Ladavas, E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Experimental Brain Research 147, 332–343.Google Scholar
Gattass, C.R., Ghobrial, I. & Bunn-Moreno, M.M. (1988). Specific inhibition of OKT8 binding to peripheral blood mononuclear cells by jacalin. Immunology Letters 17, 133–138.Google Scholar
Gescheider, G.A. (1997). Psychophysics: The Fundamentals (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Gibson, J.R. & Maunsell, J.H. (1997). Sensory modality specificity of neural activity related to memory in visual cortex. Journal of Neurophysiology 78, 1263–1275.CrossRefGoogle ScholarPubMed
Goodale, M.A. & Milner, A.D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences 15, 20–25.CrossRefGoogle ScholarPubMed
Griebel, U. & Schmid, A. (1997). Brightness discrimination ability in the West Indian manatee (Trichechus manatus). The Journal of Experimental Biology 200, 1587–1592.CrossRefGoogle ScholarPubMed
Iwai, E., Aihara, T. & Hikosaka, K. (1987). Inferotemporal neurons of the monkey responsive to auditory signal. Brain Research 410, 121–124.Google Scholar
Klotz, W. & Wolff, P. (1995). The effect of a masked stimulus on the response to the masking stimulus. Psychological Research 58, 92–101.Google Scholar
Kondo, H. & Komatsu, H. (2000). Suppression on neuronal responses by a metacontrast masking stimulus in monkey V4. Neuroscience Research 36, 27–33.CrossRefGoogle ScholarPubMed
Kopinska, A. & Harris, L.R. (2004). Simultaneity constancy. Perception 33, 1049–1060.CrossRefGoogle ScholarPubMed
Leo, F., Bertini, C., Di Pelligrino, G. & Ladavas, E. (2008). Multisensory integration for orienting responses in humans requires the activation of the superior colliculus. Experimental Brain Research 186, 67–77.Google Scholar
Livingstone, M. & Hubel, D.H. (1988). Segregation of form, color, movement, and depth: Anatomy, physiology, and perception. Science 240, 740–749.CrossRefGoogle ScholarPubMed
Manjarrez, E., Mendez, I., Martinez, L., Flores, A. & Mirasso, C.R. (2007). Effects of auditory noise on the psychophysical detection of visual signals: Cross-modal stochastic resonance. Neuroscience Letters 415, 231–236.CrossRefGoogle ScholarPubMed
Miller, G.A. (1947). Sensitivity to changes in the intensity of white and it’s relation to loudness and masking. The Journal of the Acoustical Society of America 19, 609–619.Google Scholar
Mishkin, M. & Ungerleider, L.G. (1982). Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys. Behavioural Brain Research 6, 57–77.Google Scholar
Molholm, S., Ritter, W., Murray, M.M., Javitt, D.C., Schroeder, C.E. & Foxe, J.J. (2002). Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Research Cognitive Brain Research 14, 115–128.CrossRefGoogle ScholarPubMed
Morein-Zamir, S., Soto-Faraco, S. & Kingstone, A. (2003). Auditory capture of vision: Examining temporal ventriloquism. Brain Research. Cognitive Brain Research 17, 154–163.CrossRefGoogle ScholarPubMed
Mullen, K.T., Dumoulin, S.O., Mcmahon, K.L., de Zbicaray, G.I. & Hess, R.F. (2007). Selectivity of human retinotopic visual cortex to S-cone-opponent, L/M-cone-opponent and achromatic stimulation. The European Journal of Neuroscience 25, 491–502.CrossRefGoogle ScholarPubMed
Neumann, O. & Klotz, W. (1994). Motor responses to non-reportable, masked stimuli: Where is the limit of direct parameter specification? In Attention and Performance XV: Conscious and Nonconscious Information Processing, ed. Umiltà, C. & Moskovitch, M. Cambridge, MA: MIT Press.Google Scholar
Perry, V.H., Oehler, R. & Cowey, A. (1984). Retinal ganglion cells that project to the dorsal lateral geniculate nucleus in the macaque monkey. Neuroscience 12, 1101–1123.Google Scholar
Recanzone, G.H. (2003). Auditory influences on visual temporal rate perception. Journal of Neurophysiology 89, 1078–1093.Google Scholar
Ringo, J.L. & O’Neill, S.G. (1993). Indirect inputs to ventral temporal cortex of monkey: The influence of unit activity of alerting auditory input, interhemispheric subcortical visual input, reward, and the behavioral response. Journal of Neurophysiology 70, 2215–2225.CrossRefGoogle ScholarPubMed
Ro, T., Singhal, N.S., Breitmeyer, B.G. & Garcia, J.O. (2009). Unconscious processing of color and form in metacontrast masking. Attention, Perception & Psychophysics 71, 95–103.Google Scholar
Rockland, K.S. & Van Hoesen, G.W. (1994). Direct temporal-occipital feedback connections to striate cortex (V1) in the macaque monkey. Cerebral Cortex 4, 300–313.CrossRefGoogle ScholarPubMed
Rogowitz, B.E. (1983). Spatial/temporal interactions: Backward and forward metacontrast masking with sine-wave gratings. Vision Research 23, 1057–1073.CrossRefGoogle ScholarPubMed
Romanski, L.M. (2007). Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. Cerebral Cortex 17(Suppl. 1), i61–i69.CrossRefGoogle ScholarPubMed
O’Scalaidhe, S.P., Rodman, H.R., Albright, T.D. & Gross, C.G. (1997). The effects of combined superior temporal polysensory area and frontal eye field lesions on eye movements in the macaque monkey. Behavioural Brain Research 84, 31–46.CrossRefGoogle Scholar
Schneider, T.R., Engel, A.K. & Debener, S. (2008). Multisensory identification of natural objects in a two-way crossmodal priming paradigm. Experimental Psychology 55, 121–132.Google Scholar
Shipp, S. & Zeki, S. (1985). Segregation of pathways leading from area V2 to areas V4 and V5 of macaque monkey visual cortex. Nature 315, 322–325.CrossRefGoogle ScholarPubMed
Slutsky, D.A. & Recanzone, G.H. (2001). Temporal and spatial dependency of the ventriloquism effect. Neuroreport 12, 7–10.CrossRefGoogle ScholarPubMed
Smiley, J.F. & Falchier, A. (2009). Multisensory connections of monkey auditory cerebral cortex. Hearing Research 258, 37–46.Google Scholar
Stein, B.E., London, N., Wilkinson, L.K. & Price, D.D. (1996). Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis. Journal of Cognitive Neuroscience 8, 497–506.Google Scholar
Stein, B.E. & Stanford, T.R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews. Neuroscience 9, 255–266.CrossRefGoogle ScholarPubMed
Stewart, A.L. & Purcell, D.G. (1974). Visual backward masking by a flash of light: A study of u-shaped detection functions. Journal of Experimental Psychology 103, 553–566.Google Scholar
Suied, C., Bonneel, N. & Viaud-Delmon, I. (2009). Integration of auditory and visual information in the recognition of realistic objects. Experimental Brain Research 194, 91–102.CrossRefGoogle ScholarPubMed
Tranel, D., Damasio, H., Eichhorn, G.R., Grabowski, T., Ponto, L.L. & Hichwa, R.D. (2003). Neural correlates of naming animals from their characteristic sounds. Neuropsychologia 41, 847–854.Google Scholar
Tranel, D., Grabowski, T.J., Lyon, J. & Damasio, H. (2005). Naming the same entities from visual or from auditory stimulation engages similar regions of left inferotemporal cortices. Journal of Cognitive Neuroscience 17, 1293–1305.Google Scholar
Vroomen, J. & de Gelder, B. (2004). Temporal ventriloquism: Sound modulates the flash-lag effect. Journal of Experimental Psychology. Human Perception and Performance 30, 513–518.CrossRefGoogle ScholarPubMed
Watkins, S., Shams, L., Josephs, O. & Rees, G. (2007). Activity in human V1 follows multisensory perception. NeuroImage 37, 572–578.Google Scholar
Watkins, S., Shams, L., Tanaka, S., Haynes, J.D. & Rees, G. (2006). Sound alters activity in human V1 in association with illusory visual perception. NeuroImage 31, 1247–1256.Google Scholar
Watson, A.B. & Pelli, D.G. (1983). QUEST: a Bayesian adaptive psychometric method. Perception & Psychophysics 33, 113–120.CrossRefGoogle Scholar