In his target article, Morin claims that we cannot find general-purpose codes entirely consisting of images that refer directly to ideas without being translated into spoken language (ideography). He maintains that these codes cannot be used to encode information on a broad range of topics, but only in highly specialized domains and claims that whenever visual codes become generalist, this is because (as in writing) visual images are translated into spoken language.
In the present commentary I will dwell on this issue, drawing on the use of ideography to improve the communicative abilities of patients with severe language disorders. The first attempt in this direction was made by Glass, Gazzaniga, and Premack (Reference Glass, Gazzaniga and Premack1973), who trained global aphasic patients with a pictorial system originally developed by Premack for chimpanzees, and showed that aphasic patients could learn to use this system producing simple pictorial constructions. Similar results were obtained by Johannsen-Hornbach, Cegla, Mager, and Schempp (Reference Johannsen-Hornbach, Cegla, Mager and Schempp1985) who treated global aphasics with Blissymbols and observed that some of them could benefit from therapy. Funnell and Allport (Reference Funnell and Allport1989) showed, however, that in these patients pictorial symbols provided no communicative advantage compared with their processing of written language. Analogous conclusions were reached by McCall, Shelton, Weinrich, and Cox (Reference McCall, Shelton, Weinrich and Cox2000), who showed that, after several years of repeated practice with computerized visual communication, global aphasic patients produced a restricted set of lexical items and of simple syntactic frames, but did not improve their communication.
The failure of pictorial systems to improve the communicative abilities of patients with global aphasia matches the inability shown by ideography to encode information on a broad range of topics and could be due to the fact that pictorial codes cannot replace the inner language that in severely aphasic patients is impaired as much as explicit communicative language.
Investigations exploring the nonverbal cognitive abilities of aphasic patients (recently reviewed by Gainotti, Reference Gainotti2021) have, indeed, confirmed Head's (Reference Head1926) statement that these patients are not impaired on cognitive tasks that can be performed with simple perceptual activities, but are defective when intermediate (explicit or implicit) verbal formulations are required by the task. For instance, Baldo, Paulraj, Curran, and Dronkers (Reference Baldo, Paulraj, Curran and Dronkers2015) showed a dissociation in performance of aphasic patients between two nonverbal tasks (picture completion vs. picture arrangement tasks of the Wechsler Adult Intelligence Scale) that require differing degrees of inner verbal reasoning. Aphasic patients were, indeed, usually not impaired on the first task, which can be solved with a simple perceptual activity, whereas they were severely impaired on the second task, in which subjects must detect critical differences between pictures to form with them a reasonable and meaningful story.
On the contrary, the reliance of written language on oral language has been confirmed by cognitive neuroscience studies that have shown that structures involved in oral language processing are activated during silent reading. In particular, Perrone-Bertolotti et al. (Reference Perrone-Bertolotti, Kujala, Vidal, Hamame, Ossandon, Bertrand and Lachaux2012) have shown that the temporal voice areas (TVAs), which are selectively activated by human voice perception, are also activated during silent reading and that this activation is increased when participants are reading attentively
Taken together, all these findings seem to show that ideographic codes cannot be used to encode information on a broad range of topics because they are not converted into spoken language, which is the communication device biologically grounded in the human species. Pictorial codes can, therefore, be used to encode general information only if they are converted into spoken language.
More in general, it could be said that ideographic codes are appealing for cognitive authors who maintain that concepts are represented in the brain in a formal, abstract manner, totally unrelated to the brain processing of sensory-motor functions. They are, on the contrary, much less appealing for authors who prefer a more embodied approach (e.g., Barsalou, Reference Barsalou2008) assuming that the organization of semantic representations may reflect the manner in which the information most relevant for their development has been acquired
You have
Access
In his target article, Morin claims that we cannot find general-purpose codes entirely consisting of images that refer directly to ideas without being translated into spoken language (ideography). He maintains that these codes cannot be used to encode information on a broad range of topics, but only in highly specialized domains and claims that whenever visual codes become generalist, this is because (as in writing) visual images are translated into spoken language.
In the present commentary I will dwell on this issue, drawing on the use of ideography to improve the communicative abilities of patients with severe language disorders. The first attempt in this direction was made by Glass, Gazzaniga, and Premack (Reference Glass, Gazzaniga and Premack1973), who trained global aphasic patients with a pictorial system originally developed by Premack for chimpanzees, and showed that aphasic patients could learn to use this system producing simple pictorial constructions. Similar results were obtained by Johannsen-Hornbach, Cegla, Mager, and Schempp (Reference Johannsen-Hornbach, Cegla, Mager and Schempp1985) who treated global aphasics with Blissymbols and observed that some of them could benefit from therapy. Funnell and Allport (Reference Funnell and Allport1989) showed, however, that in these patients pictorial symbols provided no communicative advantage compared with their processing of written language. Analogous conclusions were reached by McCall, Shelton, Weinrich, and Cox (Reference McCall, Shelton, Weinrich and Cox2000), who showed that, after several years of repeated practice with computerized visual communication, global aphasic patients produced a restricted set of lexical items and of simple syntactic frames, but did not improve their communication.
The failure of pictorial systems to improve the communicative abilities of patients with global aphasia matches the inability shown by ideography to encode information on a broad range of topics and could be due to the fact that pictorial codes cannot replace the inner language that in severely aphasic patients is impaired as much as explicit communicative language.
Investigations exploring the nonverbal cognitive abilities of aphasic patients (recently reviewed by Gainotti, Reference Gainotti2021) have, indeed, confirmed Head's (Reference Head1926) statement that these patients are not impaired on cognitive tasks that can be performed with simple perceptual activities, but are defective when intermediate (explicit or implicit) verbal formulations are required by the task. For instance, Baldo, Paulraj, Curran, and Dronkers (Reference Baldo, Paulraj, Curran and Dronkers2015) showed a dissociation in performance of aphasic patients between two nonverbal tasks (picture completion vs. picture arrangement tasks of the Wechsler Adult Intelligence Scale) that require differing degrees of inner verbal reasoning. Aphasic patients were, indeed, usually not impaired on the first task, which can be solved with a simple perceptual activity, whereas they were severely impaired on the second task, in which subjects must detect critical differences between pictures to form with them a reasonable and meaningful story.
On the contrary, the reliance of written language on oral language has been confirmed by cognitive neuroscience studies that have shown that structures involved in oral language processing are activated during silent reading. In particular, Perrone-Bertolotti et al. (Reference Perrone-Bertolotti, Kujala, Vidal, Hamame, Ossandon, Bertrand and Lachaux2012) have shown that the temporal voice areas (TVAs), which are selectively activated by human voice perception, are also activated during silent reading and that this activation is increased when participants are reading attentively
Taken together, all these findings seem to show that ideographic codes cannot be used to encode information on a broad range of topics because they are not converted into spoken language, which is the communication device biologically grounded in the human species. Pictorial codes can, therefore, be used to encode general information only if they are converted into spoken language.
More in general, it could be said that ideographic codes are appealing for cognitive authors who maintain that concepts are represented in the brain in a formal, abstract manner, totally unrelated to the brain processing of sensory-motor functions. They are, on the contrary, much less appealing for authors who prefer a more embodied approach (e.g., Barsalou, Reference Barsalou2008) assuming that the organization of semantic representations may reflect the manner in which the information most relevant for their development has been acquired
Financial support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Competing interest
None.