Bimodal bilingualism refers to a type of bilingualism that employs two different input-output channels, one involving spoken language and the other involving sign language. Until the second half of the twentieth century, sign language was not recognized as a fully-fledged language and there was very little research devoted to bilingual sign language speakers (Grosjean, 1992). In the last two decades, however, interest in the study of bimodal bilingualism, including the cognitive effects of bimodal bilingualism and the neural organization of spoken and sign languages, has increased considerably. Furthermore, the particular significance of studies of bimodal bilingualism for understanding bilingual language representation and processing more generally has been properly recognized. Bilingualism: Language and Cognition has not been silent or inactive on this front. Two years ago we specifically added the study of bimodal bilingualism to the journal's core areas of interest, and we are glad to see an increased number of research articles on bimodal bilingualism (e.g., Giezen & Emmorey, 2016; Rinaldi & Caselli, 2014; Williams & Newman, 2015; Morford et al., in press; Kaufmann & Philipp, in press).

Bimodal bilinguals, fluent in a signed and a spoken language, exhibit a unique form of bilingualism because their two languages access distinct sensory-motor systems for comprehension and production. Differences between unimodal and bimodal bilinguals have implications for how the brain is organized to control, process, and represent two languages. Evidence from code-blending (simultaneous production of a word and a sign) indicates that the production system can access two lexical representations without cost, and the comprehension system must be able to simultaneously integrate lexical information from two languages. Further, evidence of cross-language activation in bimodal bilinguals indicates the necessity of links between languages at the lexical or semantic level. Finally, the bimodal bilingual brain differs from the unimodal bilingual brain with respect to the degree and extent of neural overlap for the two languages, with less overlap for bimodal bilinguals.

In 1939, NYU Professor of German, Murat Roberts warned readers about the potentially harmful effects of societal bilingualism: “When two languages come to be spoken by the same society for the same purposes, both of these languages are certain to deteriorate. The sense of conflict disturbs in both of them the basis of articulation, deranges the procedure of grammar, and imperils the integrity of thought. The representation of the mind is divided into incongruous halves; and the average speaker, being no linguistic expert, finds it difficult to keep the two media apart. Confusion follows. The contours of language grow dim as the two systems collide and intermingle” (23). Roberts’ warnings about the threat of bilingualism are a thin cloak over the assumption that monolingualism is the norm. But even without dire predictions, conceptions of language representation and use derived from the study of bilinguals have been slow to enter the mainstream. The relation between language representation and control (Abutalebi & Green, 2007) and the dynamic nature of grammatical knowledge across the lifespan (Linck, Kroll & Sunderman, 2009) should change the way we conceptualize all language processing, whether monolingual, bilingual or multilingual.

Bimodal bilinguals, the fluent users of a spoken and a signed language, occupy only a small proportion of the bilingual population. However, they provide a unique window into the cognitive and neural structures of language processing. In an article in this issue, Emmorey and colleagues carefully examined the recent empirical evidence from hearing bimodal bilinguals and presented clear and interesting discussion on the implications to our understanding of bilingual language organization and processing (Emmorey, Giezen & Gollan, Emmorey, Giezen & Gollan). Here I would like to add discussion on code-blending and language control in bimodal bilinguals.

Emmorey, Giezen and Gollan (Emmorey, Giezen & Gollan) in their Key Note article review data bearing on language control in bimodal bilinguals and provoke questions critical to theoretical advance. I consider here two interrelated questions: one on multimodal synchrony and one on the control of serial order.

In the recent swell of research on bilingualism and its consequences for the mind and the brain, there has been a warning that we need to remember that not all bilinguals are the same (e.g., Green & Abutalebi, 2013; Kroll & Bialystok, 2013; Luk & Bialystok, 2013). There are bilinguals who acquired two languages in early childhood and have used them continuously throughout their lives, bilinguals who acquired one language early and then switched to another language when they entered school or emigrated from one country to another, and others who only acquired a second language (L2) as an adult. Among these forms of bilingualism there are differences in both the context and amount of time spent in each language and differences in the status of the languages themselves. The L2 may be a majority language, spoken by almost everyone in the environment, or a minority language, spoken only by a few. The native or first language (L1) may also be the dominant language or may have been overtaken by the influence of the L2 given the circumstances imposed by the environment. Likewise, the L1 and L2 may vary in how similar they are structurally, whether they share the same written script, or whether one language is spoken and the other signed.

Emmorey, Giezen and Gollan (Emmorey, Giezen & Gollan) address the fascinating question of what can be learnt about language, cognition and the brain from the unique group of people who have grown up learning both a signed and a spoken language. The focus of their review is hearing individuals – referred to as hearing bimodal bilinguals. The review presents an excellent overview of research in this field and highlights the unique insights that this population can provide.

In their review, Emmorey, Giezen and Gollan (Emmorey, Giezen & Gollan) contrast bimodal bilinguals (individuals who are fluent in a signed and a spoken language) and unimodal bilinguals (individuals fluent in two spoken languages) to highlight the implications of bimodal bilingualism for language processing, the cognitive effects of bilingualism, and the neural organization of languages. For this purpose, the authors focus on the evidence for language mixing in bimodal bilinguals (so-called ‘code-blends’) by hearing children of deaf parents and explore how language co-activation and control differentially impacts the processing of languages compared to unimodal bilinguals. The sustained controlling of two languages from differing modalities in bimodal bilinguals, according to the authors, may lead to modality-specific cognitive advantages in contrast to unimodal bilinguals.

Recent years have seen increasing research into bimodal bilingualism from a variety of paradigms such as bilingual acquisition, language processing, neural systems, and cognitive skills, with the underlying assumption that successful bimodal bilingualism entails the knowledge representations and processing of two grammars each of which via a distinct modality, auditory-oral versus visual-gestural. As such, it opens up an arena of cutting-edge research enabling comparisons of the linguistic and cognitive effects of monolingualism versus bilingualism, as well as unimodal bilingualism versus bimodal bilingualism.

The commentaries on our Keynote article “Psycholinguistic, cognitive, and neural implications of bimodal bilingualism” were enthusiastic about what can be learned by studying bilinguals who acquire two languages that are understood via distinct perceptual systems (vision vs. audition) and that are produced with distinct linguistic articulators (the hands vs. the vocal tract). The authors also brought out several new ideas, extensions, and issues related to bimodal bilingualism, which we discuss in this reply.

We used picture–word interference (PWI) to discover a) whether cross-language activation at the lexical level can yield phonological priming effects when languages do not share phonological representations, and b) whether semantic interference effects occur without articulatory competition. Bimodal bilinguals fluent in American Sign Language (ASL) and English named pictures in ASL while listening to distractor words that were 1) translation equivalents, 2) phonologically related to the target sign through translation, 3) semantically related, or 4) unrelated. Monolingual speakers named pictures in English. Production of ASL signs was facilitated by words that were phonologically related through translation and by translation equivalents, indicating that cross-language activation spreads from lexical to phonological levels for production. Semantic interference effects were not observed for bimodal bilinguals, providing some support for a post-lexical locus of semantic interference, but which we suggest may instead reflect time course differences in spoken and signed production in the PWI task.

To examine how task demands influence bilingual advantage in executive control over monolinguals, we tested 32 Chinese monolinguals and 32 Chinese–English bilinguals with four versions of a color-shape switching task. During switching trials, the task required participants to suppress one set of conflicting (or non-conflicting) responses and simultaneously to activate another set of conflicting (or non-conflicting) responses. The results showed that compared to monolinguals, (i) when suppressing conflicting responses or (ii) activating non-conflicting responses, bilinguals had significantly smaller switching costs though similar mixing costs; (iii) when suppressing one set of conflicting responses and simultaneously activating another set of conflicting responses, bilinguals had significantly smaller switching costs though larger mixing costs; and (iv) when suppressing one set of non-conflicting responses and simultaneously activating another set of non-conflicting responses, bilinguals had similar switching costs and mixing costs. These findings indicate that task demands affect bilingual advantage in executive control.

We investigate the ‘gender-congruency’ effect during a spoken-word recognition task using the visual world paradigm. Eye movements of Italian–Spanish bilinguals and Spanish monolinguals were monitored while they viewed a pair of objects on a computer screen. Participants listened to instructions in Spanish (encuentra la bufanda / ‘find the scarf’) and clicked on the object named in the instruction. Grammatical gender of the objects’ name was manipulated so that pairs of objects had the same (congruent) or different (incongruent) gender in Italian, but gender in Spanish was always congruent. Results showed that bilinguals, but not monolinguals, looked at target objects less when they were incongruent in gender, suggesting a between-language gender competition effect. In addition, bilinguals looked at target objects more when the definite article in the spoken instructions provided a valid cue to anticipate its selection (different-gender condition). The temporal dynamics of gender processing and cross-language activation in bilinguals are discussed.

Behavioural studies investigating word processing in bilinguals generally report faster response times (RTs) for first (L1) than for second (L2) language words. To examine the locus of this language effect, this study used behavioural data and event-related potentials (ERPs) collected from bilinguals while performing a semantic categorisation task on visual word pairs. RTs revealed both language and semantic relatedness effects. Spatio-temporal analysis of ERP map series showed that the semantic effect was explained by a condition-specific map segment occurring during the N400 component. The language effect was primarily explained by a map segment that started at ~170 ms and covered the period of the P2 component, that was longer in L2 than in L1 and whose duration correlated with RTs. Source localisation showed that this early segment involved the bilateral occipito-temporal regions including the fusiform area. These findings indicate that ERPs differentiated L1 and L2 during early word recognition steps.

Auditory phonological processing skills are critical for successful reading development in English not only in native (L1) speakers but also in second language (L2) learners. However, the neural deficits of auditory phonological processing remain unknown in English-as-the-second-language (ESL) learners with reading difficulties. Here we investigated neural responses during spoken word rhyme judgments in typical and impaired ESL readers in China. The impaired readers showed comparable activation in the left superior temporal gyrus (LSTG), but reduced activation in the left inferior frontal gyrus (LIFG) and left fusiform and reduced connectivity between the LSTG and left fusiform when compared to typical readers. These findings suggest that impaired ESL readers have relative intact representations but impaired manipulation of phonology and reduced or absent automatic access to orthographic representations. This is consistent with previous findings in native English speakers and suggests a common neural mechanism underlying English impairment across the L1 and L2 learners.

We recorded eye movements during natural reading to explore the influence of sentence context on bilingual word recognition. English monolinguals and Spanish–English bilinguals read sentences in English that biased either the English or the Spanish meaning of interlingual homographs. Shortly after encountering the homograph, the groups showed equivalent implausibility effects when its English meaning was incongruent with the preceding sentence context. No evidence for immediate homograph interference emerged during this period in the bilingual group. Only in later processing measures did group and congruency interact. Bilinguals may have initially accessed and selected the language appropriate meaning of the homograph to integrate into the sentence. Later, bilinguals accessed their first language lexicon and integrated the Spanish meaning into the sentence when semantically appropriate. Rather than always experiencing cross-language competition, proficient bilinguals may dynamically adapt to contextual cues and selectively access information associated with the contextually cued language under certain conditions.

We evaluate whether bilinguals activate the phonology of their first language (L1) when they speak in their second language (L2). In Experiment 1, Spanish–English (L1–L2) bilinguals performed a color naming task in which they named the color of pictures in L2 while the phonological relationship between the color names and picture names in L1 was evaluated. The bilinguals were slower when color names and picture names were phonologically related in L1 relative to a control condition in which they were unrelated. The same pattern of results was obtained in Experiment 2 when bilinguals with less L2 proficiency were evaluated. These results suggest that bilinguals coactivated the phonology of their two languages and that the phonology in L1 influenced the production of speech in L2. This finding is explained in terms of cascade models of bilingual language production.

We investigated whether speaking in one language affects cross- and within-language activation when subsequently switching to a task performed in the same or different language. English–French bilinguals (L1 English, n = 29; L1 French, n = 28) were randomly assigned to a prior language context condition consisting of a spontaneous production task in English (the no-switch group) or in French (the switch group). Participants then performed an English spoken language comprehension task using the visual world method. The key result was that the switch group showed less evidence of cross-language competition than the no-switch group, consistent with the notion of an active inhibition of a prior language in the switch group. These data suggest that proficient bilinguals can globally suppress a non-target language, whether it is L1 or L2, though doing so requires cognitive resources that may be diverted from other demands, such as controlling within-language competition.

Context and prosody are the main cues native-English speakers rely on to detect and interpret sarcastic irony within spoken discourse. The importance of each type of cue for detecting sarcasm has not been fully investigated in native speakers and has not been examined at all in adult English learners. Here, we compare the extent to which native-English speakers and Arabic-speaking English learners rely on contextual and prosodic cues to identify sarcasm in spoken English, situating these findings within current cross-linguistic effects literature. We show Arabic speakers utilize the cues to a different extent than native speakers: they tend not to utilize prosodic information, focusing on contextual semantic information. These results help clarify the relative weight of contextual and prosodic cues in native-English speakers and support theories that suggest that prosody and emotion could transfer separately in second language learning such that one could transfer while the other does not.

Much research on language control in bilinguals has relied on the interpretation of the costs of switching between two languages. Of the two types of costs that are linked to language control, switching costs are assumed to be transient in nature and modulated by trial-specific manipulations (e.g., by preparation time), while mixing costs are supposed to be more stable and less affected by trial-specific manipulations. The present study investigated the effect of preparation time on switching and mixing costs, revealing that both types of costs can be influenced by trial-specific manipulations.