The influence of three-gendered grammatical systems on simultaneous bilingual cognition: The case of Ukrainian-Russian bilinguals

Abstract

This paper examines the linguistic relativity principle (Whorf, 1956) by investigating the impact of grammatical gender on cognition in simultaneous bilinguals of three-gendered Ukrainian and Russian. It examines whether speakers of three-gendered languages show grammatical gender effects on categorisation, empirically addressing claims that such effects are insignificant due to the presence of the neuter gender (Sera et al., 2002). We conducted two experiments using a similarity judgement paradigm while manipulating the presence of neuter gender stimuli (Phillips & Boroditsky, 2003). Experiment 1, including neuter gender, revealed no significant effects, compatible with earlier studies on three-gendered languages. Conversely, Experiment 2, excluding neuter gender stimuli, showed significant language effects. Bilingual participants rated pairs as more similar when grammatical genders in both languages were congruent with the biological sex of a character. Significant effects were also found for pairs with mismatching grammatical genders in Ukrainian and Russian. Participants with higher proficiency in Ukrainian rated pairs as more similar when the grammatical gender of a noun in Ukrainian was congruent with the character’s biological sex, and incongruent in Russian. Our findings thus provide the first empirical demonstration that the exclusion of neuter gender online induces grammatical gender effects in speakers of three-gendered languages.

1. Introduction

The majority of studies investigating linguistic relativity effects typically concentrate on the question ‘Does language influence our thoughts?’ (Athanasopoulos & Casaponsa, 2020). While this question has been asked in a number of disciplines, such as philosophy, linguistics, anthropology and psychology, modern versions of the question can be traced to Whorf (1956) and more recent transdisciplinary scholarly activity (Lucy, 1997; Gentner & Goldin-Meadow, 2003; Athanasopoulos et al., 2016), which has placed the question at the forefront of cognitive science. Various domains have been used as a testbed for the hypothesis, such as spatio-temporal metaphors (Athanasopoulos & Bylund, 2023), colour (Athanasopoulos, 2009; Winawer et al., 2007) and grammatical gender (Boroditsky & Schmidt, 2000; Boutonnet et al., 2012; Sato & Athanasopoulos, 2018).

The latest surge of attention led to more detailed explanations of the effects languages may have on cognitive processes, by including various experimental conditions, such as verbal interference, differentiating stimuli based on their perceptual characteristics or manipulating the complexity of experimental design (Athanasopoulos & Casaponsa, 2020). Therefore, posing the aforementioned question as one that requires a binary answer seems outdated. Instead, the focus is moving away from providing evidence to a ‘yes-no’ question towards investigating what circumstances lead to emerging language effects on cognitive processes (e.g., memory or categorisation), as well as how and why language-specific features form the groundwork for individual perceptual judgement, including multilingual speakers (Bassetti & Filipović, 2022; Casasanto, 2016). An illustrative example of the latter in our study pertains to the emergence of grammatical gender effects in speakers of three-gendered languages. Previous research on linguistic relativity (Sera et al., 2002; Vigliocco et al., 2005) has reported the absence of such effects, while more recent studies yield mixed results (Pavlidou & Alvanoudi, 2019). The primary factor contributing to these mixed or non-emergent outcomes has been hypothesised to be the presence of the neuter gender in these languages, which is thought to diminish the prominence of gender effects. Consequently, our research seeks to determine whether grammatical gender effects on cognitive processes, such as categorisation, are confined to two-gendered languages or can also be observed in speakers of three-gendered languages, and under what specific conditions these effects manifest.

We also focus on bilingual speakers who have two partially conflicting grammatical systems (where some nouns have matching and others mismatching grammatical gender in Ukrainian and Russian). Specifically, the impact two grammatical gender systems have on perception and categorisation, even when participants are not actively engaging with either language, as the testing was conducted entirely in English, which, unlike Russian and Ukrainian, does not have a grammatical gender system.

Generally, research on language and cognition in bilinguals continues to be an important endeavour of the linguistic relativity theory complex, as Whorf (1956) himself pointed out that if language affects our thoughts, then learning other languages can free people from the shackles of their own language. Employing Ukrainian-Russian simultaneous bilinguals is of interest because the representation of two grammatical gender systems within an individual’s mind and their effects on bilinguals’ cognitive processes, such as memory or categorisation, have received little attention (e.g., the study by Bassetti, 2007). It remains unclear whether language effects would emerge only when grammatical gender matches in both languages or if they would also occur when grammatical gender mismatches, depending on the more proficient language. Additionally, there is uncertainty whether any effects would appear at all, given that both languages include a neuter gender in their grammatical system.

Here, we attempt to investigate the effects that two partially contrasting three-gendered grammatical systems (e.g., Ukrainian as L1 and Russian as 2 L1) have on categorisation, as well as introduce simultaneous bilinguals with two distinct grammatical gender systems into linguistic relativity research. In addition, at a theoretical level, we aim to explore whether the presence of neuter grammatical gender mitigates language effects, as suggested previously (Sera et al., 2002; Vigliocco et al., 2005). To do so we employed a similarity judgement paradigm while manipulating stimuli with neuter gender (Experiment 1) and without neuter gender (Experiment 2). Such manipulation would also allow us to investigate further into the nature of the gender effects, particularly whether (if found) the effects of grammatical gender arise online (in the moment of testing) or offline (entrenched in previous language experience) (Lupyan et al., 2020). If the effects arose online (Lupyan, 2012; Sato & Athanasopoulos, 2018), we anticipated observing more pronounced effects in Experiment 2, whereas if the effects were offline, comparable effects were expected across both Experiment 1 and Experiment 2.

1.1. Grammatical gender in language and mind

The empirical evidence of linguistic relativity effects can be found across various domains, such as colour categorisation/discrimination (Athanasopoulos, 2009; Roberson et al., 2005; Winawer et al., 2007), time and space (Athanasopoulos & Bylund, 2023; Boroditsky, 2001; Casasanto et al., 2004), motion (Athanasopoulos & Bylund, 2013), grammatical number and object classification (Athanasopoulos, 2006; Lucy, 1992), tactile perception (Miller et al., 2018) and even olfaction (Cao et al., 2024; Speed & Majid, 2019; Vanek et al., 2021). This evidence supports the idea that the structure of language can shape non-linguistic cognition, offering a compelling testbed for investigating how grammatical features, such as gender, influence thought.

Grammatical gender has been used as a subject of analysis by linguistic relativity researchers because of two primary reasons. Firstly, when grammatical gender is absent, no other lexicalisation pattern can replace it (Boutonnet et al., 2012). Secondly, the assignment of grammatical gender to inanimate nouns, and certain animals in the case of Ukrainian and Russian, is usually unpredictable and semantically illogical (Elpers et al., 2022). For instance, ‘parrot’ in Ukrainian takes the feminine grammatical gender, while in Russian it is masculine. Besides, even though grammatical gender is superfluous for interaction in the case of many languages (e.g., English), for speakers of various languages such as Russian and Ukrainian it cannot be ignored. In such languages, the gender of objects is mandatorily marked in a range of morphosyntactic constructions, such as demonstratives, pronouns, singular adjectives and verbs in the past tense (Mitrofanova et al., 2018). Such morphosyntactic consequences of grammatical gender make it an ideal candidate for examining whether grammatical categories influence cognitive processes beyond lexical features (Sato & Athanasopoulos, 2018).

Despite extensive research, a notable gap exists in understanding the cognitive effects of grammatical gender across different grammatical systems, particularly three-gendered languages. Most studies have focused on German (Bassetti, 2007; Pavlidou & Alvanoudi, 2019; Sera et al., 2002; Vigliocco et al., 2005), which may yield less significant results due to inconsistencies in gender assignment (e.g., ‘das Mädchen’ [a girl] being neuter) and the use of articles that do not always differentiate between genders (e.g., the dative case where both masculine and neuter use ‘dem’). In contrast, Ukrainian and Russian, both three-gendered, indicate gender primarily through noun endings, providing a more consistent gender-marking system. By extending research to these under-represented languages, this study aims to offer new insights into how three-gendered grammatical systems influence cognitive processes.

A wide range of behavioural tasks has been developed to study the impact of grammatical gender on the cognitive representation of concepts, with the most common one being the voice attribution task (i.e., asking participants to assign either a male or female voice to objects; see Samuel et al., 2019). Other methods include a sex assignment task (Belacchi & Cubelli, 2012), an object-name memory task (Boroditsky & Schmidt, 2000) and a similarity judgement task (Phillips & Boroditsky, 2003). The current study employs the similarity judgement task, where participants rate the similarity between pairs of depicted objects and characters with a clear biological sex using a Likert scale. The choice of this paradigm is rooted in its unique strengths, such as it requires using unlabelled stimuli that minimise active language processing that is a key element in testing whether language shapes non-linguistic representations (Casasanto, 2016). This methodology was first implemented in linguistic relativity research in the seminal work of Phillips and Boroditsky (2003), who argued that Spanish-English and German-English sequential bilinguals perceived object-personified character pairs as more similar when the biological sex of the character and the grammatical gender of the object in their L1 were congruent, even when tested in English. This suggests that grammatical gender influences object categorisation even when grammatical gender is not explicitly used. Overall, the research has shown that when making gender-related judgements, individuals often take into account the object’s grammatical gender (Flaherty, 2001; Konishi, 1993). Despite more recent studies that produced contrasting results and highlighted the issue of a replication crisis, including a failed replication by Elpers et al. (2022) and mixed findings by Sedlmeier et al. (2016), the study by Phillips and Boroditsky (2003) has nonetheless made a significant impact on the field.

One possible explanation for the mixed findings might be linked to the type of grammatical gender system present in a language, particularly the distinction between two-gendered and three-gendered systems. For instance, Sera et al. (2002) found that, unlike Spanish and French monolingual children, German children did not use grammatical gender to assign voices to objects during categorisation tasks, instead aligning their responses more closely to Spanish gender. The study suggests that two-gendered languages have a stronger association between grammatical and natural gender, leading to overgeneralisation of masculine and feminine traits to inanimate objects. In contrast, speakers of languages with a three-gender system, such as German, appear to rely less on gender and more on other conceptual distinctions when categorising objects. Similarly, Vigliocco et al. (2005) found significant gender effects in Italian but not in German during a similarity judgement task, arguing that the weaker link between grammatical gender and semantic properties in three-gender systems results in reduced gender effects on perception. Inconsistencies in gender assignment and a lack of clear correspondence with the sex of referents likely contribute to this difference. The authors suggest that the mapping between grammatical gender and semantic properties is weaker in three-gender systems like German compared to two-gender systems like Italian. They argue that three-gendered languages do not exhibit the same grammatical gender effects because the correspondence between gender and the sex of referents is less transparent. To address these criticisms and further examine the role of grammatical gender in three-gender systems, Pavlidou and Alvanoudi (2019) conducted a sex-attribution task (adapted from Sera et al., 2002) with speakers of German and Greek (both three-gendered languages). Participants were asked to assign names to depicted nouns for a preschool play, with nouns having masculine, feminine or neuter gender. Their analysis revealed significant effects of grammatical gender on sex-attribution in both languages, challenging earlier claims by Sera et al. (2002) and Vigliocco et al. (2005).

Similarly, Bassetti (2007) – the only study to our knowledge that examined simultaneous bilinguals when looking at grammatical gender effects in linguistic relativity research – investigated how grammatical gender influences categorisation and representations of concepts in Italian-German simultaneous bilingual and Italian monolingual children using a voice attribution task. This is particularly relevant to the current study because objects were also chosen with opposite genders in Italian and German. Results showed that grammatical gender effects were only present in Italian monolinguals, echoing Sera et al. (2002), suggesting that Italian gender assignment may be more intuitive or ‘natural’ compared to German. The study also noted that bilinguals, who navigate two languages with mismatched grammatical gender systems, develop unique cognitive frameworks, integrating elements from both languages. Consequently, bilinguals may think differently from monolinguals, not because of bilingualism itself, but due to the specific characteristics of the grammatical systems embedded in the languages they speak, such as mismatching grammatical genders in Italian and German. This observation is particularly relevant to our study, as we also examine partially mismatching grammatical gender systems, albeit within two three-gendered languages.

1.2. Online vs offline nature of the grammatical gender effects

A central question in this line of research is whether grammatical gender effects operate online (as real-time, context-sensitive influences) or offline (as enduring impacts of long-term linguistic experience). According to Lupyan et al. (2020), online effects occur when language actively modulates perception and decision-making at the moment, often shaped by top-down feedback from linguistic labels and grammatical structures. Offline effects, in contrast, reflect long-term, habitual patterns ingrained by extensive language use that influence perception even outside linguistic contexts.

This study draws on two complementary theoretical frameworks to address this distinction. The label-feedback hypothesis (Lupyan, 2012) proposes that even when no explicit labels are presented, internal labelling processes may still influence perception and categorisation in real-time. This reflects a top-down influence, where prior language knowledge actively shapes what features are noticed or emphasised during perception. Extending this idea, the structural-feedback hypothesis (Sato & Athanasopoulos, 2018) posits that the influence of grammatical gender extends beyond specific labels, stemming from the broader habitual patterns ingrained by the grammatical system itself. According to this hypothesis, grammatical gender activates unconsciously during the online categorical perception, and by doing so, it modulates perception by emphasising the features associated with it.

The current study aims to directly engage with the online vs offline debate by designing two similar experiments with the main difference being that Experiment 1 includes objects of all three grammatical genders (masculine, feminine and neuter), while Experiment 2 excludes neuter gender. This allowed us to test whether the presence of neuter stimuli dilutes the salience of masculine-feminine distinctions, potentially weakening online grammatical gender effects. If the effects are online, we expect stronger effects in Experiment 2, as removing the neuter gender heightens the binary masculine-feminine distinction. Conversely, if the effects are offline, results should remain consistent across both experiments, reflecting the enduring impact of long-term linguistic patterns rather than immediate task context.

To sum up, given the mixed results demonstrated in studies involving speakers of three-gendered languages, it is important to note that no previous research has directly compared the strength of grammatical gender effects using the same task with and without the inclusion of neuter gender. The present study uniquely investigates the cognitive effects of bilingualism in two conflicting three-gendered languages, a topic that has not been previously explored. Besides, we extend research beyond typically used German to other three-gendered languages (Ukrainian and Russian). This approach provides a more comprehensive understanding of how grammatical gender influences cognition across diverse linguistic contexts.

1.3. Case of Ukrainian simultaneous bilingualism and typological differences in Ukrainian and Russian languages

Simultaneous bilingualism in Ukraine presents unique challenges and insights into the cognitive processing of language, particularly when the languages involved have distinct grammatical systems. This study focuses on Ukrainian-Russian bilingualism, specifically the typological differences between the languages, especially regarding grammatical gender.

Ukraine has a deep-rooted history of multilingualism (Poftak & Shykula, 2022), and the status of the Russian language has long been a subject of debate (Eberhard et al., 2019). According to the 2001 census, out of Ukraine’s then-population of 48.5 million, 78% identified as Ukrainians and 17% identified as Russians when asked to choose one ethnic affiliation. However, linguistic preferences differed, with 68% selecting Ukrainian as their native language and 30% opting for Russian (Bilaniuk & Melnyk, 2008). Despite the historical stigmatisation of bilingualism even prior to the war (Pavlenko, 2012), it is clear that societal bilingualism is inherent in Ukraine (Csernicskó & Máté, 2017; Shumlianskyi, 2010).

The onset of the war in February 2022 dramatically altered these linguistic landscapes. There has been a sharp increase in the proportion of respondents who, according to self-reported questionnaires, speak predominantly Ukrainian in everyday life and a corresponding decrease in Russian speakers. The most recent poll from December 2022 indicates that 41% of respondents claimed to communicate only in Ukrainian, another 17% reported using Ukrainian ‘in most situations’, while only 6% speak only in Russian and 9% predominantly in Russian, another 24% said they use both languages ‘equally’ (Kulyk, 2023). Compared to 2017, the proportion of exclusive and predominant Ukrainian speakers increased by 8%, and the proportion of Russian speakers decreased by 11% (Kulyk, 2023). Given the fluid language attitudes and shifting language use among bilingual individuals in Ukraine, it is worth examining which languages have the most significant impact on the cognitive processes of such speakers. It has been proposed in linguistic relativity research that language effects are found for the dominant native language, rather than for the second language (Bassetti, 2007; Phillips & Boroditsky, 2003). However, these assessments often relied on participants’ self-evaluations of their language dominance and language proficiency. To address this issue, the current study includes proficiency tests for English (language of testing), Ukrainian and Russian, as well as a self-rated Bilingual Linguistic Profile (BLP, Gertken et al., 2014) to comprehensively assess the proficiency differences.

Typologically, the two languages are linguistic cousins, both belonging to the East Slavic branch of the Indo-European language family (Kortmann & Auwera, 2011), which shares significant historical, lexical and grammatical similarities. They have a considerable overlap in vocabulary, grammar and pronunciation characteristics, setting them apart from other Slavonic languages. Various studies indicate that Ukrainian and Russian share about 55%–62% of their vocabulary, a lexical distance akin to that between Portuguese and French (Steinback, 2015). Like other Indo-European languages, Ukrainian and Russian incorporate grammatical gender, categorising nouns as feminine, masculine or neuter. These languages are highly inflectional with overt gender systems, where gender influences noun declension and adjective endings (Budzhak‐Jones, 1997). In Ukrainian, nouns are divided into three genders, with syntactic agreement indicating gender, except for invariably gender-neutral plural nouns (Rusanivskyj et al., 2004). Russian follows a similar division but with an uneven distribution: 46% of nouns are masculine, 41% feminine and 13% neuter. The masculine gender, being most prevalent, is often considered the default (Corbett, 1991; 2007). The lack of extensive research on Ukrainian gender distribution leaves the question of whether it follows a similar pattern open.

Although nouns in Ukrainian and Russian neither change according to genders nor have gendered articles, grammatical gender affects the declension of nouns and endings in both languages. In Ukrainian language, masculine gendered animate and inanimate nouns typically have consonant endings (e.g., дім [dim] – house), while feminine gender is predicted by -a / - я endings (e.g., кава [kava] – coffee, iсторiя [istoriia] – history). Most abstract nouns are feminine (Pugh & Press, 1999), regardless of the ending (e.g., радiсть [radist’] – joy, тиша [tysha] – quiet). Neuter nouns have three possible endings: -o, −e, −ння / -ття (дерево [derevo] – tree, сонце [sontse] – sun, кохання [kokhannia] – love) (Bezpoiasko et al., 1993; Gorpynyč, 2004).

Similarly, in Russian, the endings of nouns suggest their grammatical gender: masculine nouns end with a consonant or -й, feminine nouns end with -а or -я, while neuter nouns have -o / -e endings. There is also a large number of exceptions, such as nouns ending with a soft sign -ь, that can refer either to masculine or feminine nouns. In both languages, grammatical gender is semantically and morphologically assigned, affecting adjectives, pronouns and determiners (Basova, 2014), and is a mandatory feature for nouns except in plural forms (Gorpynyč, 2004). The described grammatical gender distribution in Ukrainian and Russian provides a well-suited setting for investigating grammatical gender effects on cognitive processes. It presents an opportunity to go beyond investigating a three-gendered grammatical system, but analysing language effects when 2 L1s have contrasting three-gendered systems.

2. Aims and the scope of the current study

This study aims to shed light on what (if any) effects two partially contrasting three-gendered grammatical systems have on cognitive processes of simultaneous bilinguals. While research has examined the impact of single three-gendered systems (Konishi, 1993; Pavlidou & Alvanoudi, 2019; Sera et al., 2002), little is known about the cognitive implications of simultaneously acquiring two languages with differing grammatical features (Bassetti, 2007). We hypothesised that simultaneous bilinguals would demonstrate a language effect similar to that of sequential bilinguals – specifically, they would demonstrate the influence of grammatical gender on categorisation, despite prior research suggesting that gender effects are limited to speakers of two-gendered languages because the binary nature of the system makes grammatical gender more salient (Sera et al., 2002). We expect to observe a grammatical gender effect, by employing a more rigorous stimuli design encompassing grammatical genders both matching and mismatching across languages, coupled with the inclusion of languages where grammatical gender is manifested through diverse grammatical features rather than articles. Additionally, we aim to investigate whether the presence of neuter gender in the stimuli (Experiment 1) would affect the observed grammatical gender effects, compared to Experiment 2, where it was absent. If grammatical gender effects have an online nature, as shown in previous studies (Sato & Athanasopoulos, 2018), we would expect stronger effects in Experiment 2, compared to Experiment 1, as the absence of neuter gender would amplify the contrast between masculine and feminine gender, enhancing the observed effects in the real-time of task completion.

To investigate our hypothesis, we adapted a similarity judgement paradigm where participants rated the similarity of pairs of stimuli, comprising depicted conceptually neutral nouns (e.g., a notebook), presented alongside a picture of a male or female character (e.g., a ballerina) on a 9-point Likert scale (Phillips & Boroditsky, 2003). The tasks in both experiments were conducted in English (starting with the participant’s information sheet in the first email until debriefing). This was done to prevent the participant from actively using either of their L1s. The current paradigm was chosen for several reasons. Firstly, it has been used many times, yielding mixed results with speakers of three-gendered languages. However, it has never been used to our knowledge with a three-gendered language omitting the neuter gender as presented in Experiment 2. Using the same task ensures that any effects observed can be attributed to our experimental manipulation rather than any potential confounds of the task itself. Secondly, it was employed due to the high salience of gender/sex in the task (Samuel et al., 2019), laying the groundwork for subsequent exploration of more subtle, implicit effects of gender on cognitive processes.

Experiment 1 aims to provide an initial understanding of the grammatical gender effects of Ukrainian and Russian on categorisation, in contrast to English monolingual controls. In the first part of this experiment, we look at the interaction between group (Ukrainian-Russian bilinguals vs English monolinguals) and condition (whether the noun’s grammatical gender matches or mismatches the character’s biological sex) and whether it had any influence on similarity ratings (Likert scores). Here we anticipate that Ukrainian-Russian bilinguals will show stronger effects of condition on the similarity ratings compared to English monolinguals. The stimuli include nouns with matching grammatical genders in Ukrainian and Russian (e.g., ‘pencil’ – masculine in both, ‘candle’ – feminine in both, ‘tree’ – neutral in both). Confirming this prediction would reaffirm the original findings by Phillips and Boroditsky (2003) and demonstrate that the presence of neuter gender does not negate the language effects. In the second part, when looking at the results of the bilingual group only, we analyse ratings based on participants’ most proficient language (Ukrainian or Russian). Stimuli were chosen to include noun-character pairs with contrasting grammatical genders in Ukrainian and Russian languages (e.g., ‘a basket’ – masculine in Ukrainian, feminine in Russian – paired with a ballerina [female character]; ‘an iron’ – masculine in Russian, feminine in Ukrainian – paired with a king [male character]). We predict that bilinguals will rate pairs as more similar when the grammatical gender of the object (masculine or feminine) in their more proficient language is congruent with the character’s biological sex (male or female).

Experiment 2 contains only masculine and feminine nouns, investigating whether excluding neuter gender strengthens the grammatical gender effects. The manipulation here directly addresses a central question in the field regarding the possibility that the presence of neuter gender impairs language effects. The question is whether this happens at a general or a local level. In other words, does the presence of the neuter gender in the grammatical system of a language attenuate the effects of gender on categorisation across the board, or are such attenuating effects only observable when the neuter gender is used as part of the similarity judgements that participants are asked to perform. Similar to Experiment 1, we anticipate to find grammatical gender effects on similarity ratings in the Ukrainian-Russian bilingual group but not in the English monolingual group. Within the Ukrainian group, the impact of language proficiency on ratings is also explored.

Overall, we expect to find a significant effect of grammatical gender on categorisation of simultaneous bilinguals, irrespective of the contrasting three-gendered systems of Ukrainian and Russian. The outcomes of this study are expected to highlight the influence grammatical gender has on cognitive processes, shedding more light on how complex and contrasting linguistic systems shape human cognition.

3. Method

Materials and analysis codes can be found on the Open Science Framework (OSF): https://osf.io/3xgaw/?view_only=d061634113d14fa098fb8c2eacb4d81e.

3.1. Experiment 1

Participants. 63 Ukrainian-Russian simultaneous bilingual speakers (with English as a foreign language) and 37 English monolingual speakers completed the study online in exchange for time compensation in the form of a £10 Amazon voucher. After examining their linguistic profiles and responses, 51 Ukrainian-Russian bilinguals (48 females; Mean_age = 32, SD_age = 10) and 24 English monolinguals (9 females; Mean_age = 30, SD_age = 13) were included in the analysis. Exclusion criteria encompassed speaking other gendered languages (n = 22) or consistently selecting a ‘1’ rating on the Likert scale, indicating inattention to instructions or lack of engagement (n = 3). Among the bilingual group, 66.7% (n = 34) had a postgraduate degree, 23.5% (n = 12) had an undergraduate degree, 2% (n = 1) had a college degree and 7.8% (n = 4) had high school education or less. In contrast, among the monolingual group, 50% of participants (n = 12) had a postgraduate degree, 25% (n = 6) had an undergraduate degree and 25% of participants (n = 6) had a college degree, with no participants having only finished high school.

The bilingual participants’ proficiency in Ukrainian, Russian and English was assessed using standardised language tests. For Ukrainian and Russian, advanced ZNO Tests (External Independent Assessment) were used (Ukrainian Center for Educational Quality Assessment, 2020). These standardised university entrance examinations evaluate participants’ language skills up to the C2 proficiency level, thereby mitigating potential ceiling effects of L1 proficiency in our study. English proficiency was determined through the Oxford Quick Placement Test (OQPT) (Oxford University Press, 2001) or existing International English Language Testing System (IELTS) certification (Cambridge University Press, 2021). Acceptable scores were set at 67% for the OQPT and 5.5 for the IELTS, both equivalent to the B2 (Upper-Intermediate) level. ZNO tests classify Ukrainian and Russian proficiency levels between C1 (advanced) and C2 (proficient).

The bilingual participants reported an average age of 8.68 years (SD = 3.21) for acquiring English as a foreign language (L2), with a minimum proficiency level of Upper-Intermediate. The majority of participants demonstrated higher proficiency scores in Ukrainian (57.38%, n = 29), as opposed to Russian (22.95%, n = 12), or equal proficiency in both (19.67%, n = 10). The proficiency scores ranged widely, indicating no ceiling effects (see Table 1).

Table 1. Proficiency scores and distribution of Ukrainian-Russian bilingual participants in Experiment 1

Participants completed the study online, after being recruited through social media or through posters at [ANONYMISED]. The gender imbalance in bilingual participants, predominantly female, resulted from the data collection occurring after the onset of the war in Ukraine. However, as Flaherty (2001) notes, such a discrepancy in participants’ gender is unlikely to significantly affect the responses. Besides, we used separate cumulative link mixed models (CLMMs) for each experiment to investigate whether there was an effect of participants’ gender (see Supplementary Materials for full analysis and results). However, the absence of a significant three-way interaction between the group (Ukrainian-Russian bilingual vs English monolingual), participant’s gender (male vs female) and grammatical gender (masculine vs feminine vs neuter) suggested that the gender imbalance in the bilingual group did not appear to disproportionately affect the main findings of the study.

3.1.1. Materials

Pre-test. A pre-test was conducted to select conceptually gender-neutral items for the main experiment, following the approach of Sato and Athanasopoulos (2018). Ten Ukrainian-Russian-English speakers (5 females; Mean age = 26, SD age = 4) and ten English monolinguals (4 females; Mean age = 31, SD age = 10) were recruited. None of the participants took part in the main study. Participants were shown 137 black-and-white object images one by one and asked to rate each picture on a 7-point Likert scale ranging from ‘very feminine’ (1) to ‘very masculine’ (7). The objects were divided into five groups based on their grammatical genders in Ukrainian and Russian: (1) 20 nouns masculine in Russian and feminine in Ukrainian, (2) 24 nouns feminine in Russian and masculine in Ukrainian, (3) 31 nouns feminine in both languages, (4) 31 nouns masculine in both languages and (5) 31 nouns neutral in both languages. All images, presented against a greyscale and white background to avoid colour biases, were sourced from the Bank of Standardised Stimuli (Brodeur et al., 2014).

The pre-test yielded 50 conceptually neutral items (Mean = 4.01; SD = 0.13), which were then divided into the five categories (see Table 2): (1) nouns with masculine grammatical gender in both Russian and Ukrainian languages, (2) feminine grammatical gender in both Russian and Ukrainian, (3) feminine in Russian, masculine in Ukrainian, (4) feminine in Ukrainian and masculine in Russian, and (5) neutral in both. A slight imbalance between stimuli (3) and (4) is not anticipated to impact our results, as they will be analysed collectively. This will yield a total of 20 nouns with matching grammatical gender in both languages, 20 nouns with mismatching grammatical gender and 10 neuter fillers.

Table 2. Example of stimuli used for both Experiment 1 and 2

Main testing. In the main experiment, participants were presented with 100 pairs, each consisting of one of the 50 selected conceptually neutral unlabelled black-and-white objects and one of the16 characters: 8 female images (a queen, a bride, a witch, a smurf, a ballerina, a girl, a pensioner, an ogre) and 8 male images (a king, a groom, a giant, a smurf, an architect, a boy, a man, an ogre). Each depicted noun was presented once with a male character and once with a female character, resulting in 100 pairs. Pairs were presented in a randomised order. Each participant had to provide a similarity rating on the Likert scale from 1 (not similar) to 9 (very similar) with each pair displaying the object on the left and the character on the right of the screen.

3.1.2. Procedure and design

To conduct the experiment, we utilised the Gorilla Experiment Builder software. Upon registration, participants received an introductory email containing the participant information sheet and a link to the experiment. After signing a consent form, they were redirected to the main task, which they accessed on their personal laptops or computers.

Both groups undertook the same experimental task in English. The instructions were similar to those from Phillips and Boroditsky (2003), p. 929): ‘In this study, you will see pairs of pictures appear on the screen. In each pair, there will be a picture of a person on the left and a picture of an object or animal on the right. You will see a scale where 1 = not similar and 9 = very similar. For each pair of pictures, please choose a number between 1 and 9 to indicate how similar you think the two pictures are. Try to use the whole scale (give some 1’s and some 9’s and some of all the numbers in-between). Please respond with the first answer that comes to mind’.

Each object-person pair remained on the screen until participants selected ‘Next’. Once they moved on to the next pair, they could not change their answer. After completing the task, participants were asked what criteria were used to rate the pairs to determine whether they detected the experiment’s aim and used grammatical gender as a task-solving strategy. None of the participants reported reliance on grammatical gender or language in general. Instead, responses were reported to be influenced by associations with films or cartoons, shapes or random guesses. Ukrainian-Russian bilingual participants then completed a Bilingual Language Profile (BLP, Gertken et al., 2014) questionnaire and two proficiency tests (Oxford University Press, 2001; Ukrainian Centre for Educational Quality Assessment, 2020). The monolingual group only completed the BLP to identify any gendered language knowledge potentially affecting results. Additionally, we monitored the real-time completion of the experiment. In those instances where participants substantially exceeded the expected average response times or stopped during the task, their participation was manually excluded (6 bilingual and 9 monolingual participants), given the importance of capturing responses on the first-impression basis.

3.1.3. Analysis

For each experiment, data analysis involved CLMMs in RStudio (version 2022.07.22, R Core Team, 2022), using the ‘ordinal’ package (Christensen, 2019), with similarity ratings as the dependent variable. Previous study that replicated the original experiment by Phillips and Boroditsky (2003) employed linear mixed-effects models (Elpers et al., 2022), highlighting their advantages, such as incorporating both fixed and random effects and analysing non-averaged data (Baayen et al., 2008; Vasishth & Broe, 2011). However, as the analysis includes Likert scale and ordinal data, we used CLMMs instead. Similar to linear mixed effects models, CLMMs also accommodate multiple sources of error variance as random variables, such as participant variability and the gender of depicted characters (Bross, 2019). Yet, CLMMs are more suited for analysing ordinal data, as they account for the possibility of varying distances between levels of the rating scale (Ackerman, 2018).

We divided the analysis into two parts. The first part involved a comparative analysis of responses from both Ukrainian-Russian bilingual and English monolingual participants. We focused on how the interaction between grammatical gender congruence of the pairs (grammatical gender of the object was congruent or incongruent in both Russian and Ukrainian with the biological sex of the character) and the participant group (Ukrainian-Russian or English) influenced the Likert scores. The maximal model that converged included random intercepts for participants and items. The detailed analysis is available on OSF (https://osf.io/3xgaw/?view_only=d061634113d14fa098fb8c2eacb4d81e).

Secondly, to investigate deeper the effects of two contrasting three-gendered languages, we conducted an analysis comparing Ukrainian-Russian bilinguals only, based on their most proficient language. In the current study, we approached bilingualism as a continuum and measured it as a continuous variable by subtracting Russian proficiency from Ukrainian proficiency scores, resulting with the scale −100 being only proficient in Russian and + 100 being only proficient in Ukrainian. Participants with equal proficiency scores were included in the analysis with the coefficient score 0. Here, we examined how the congruence of an object’s grammatical gender in L1 with the character’s biological sex (and its incongruence in 2 L1) interacted with language proficiency to affect similarity ratings. A maximal model in this part also included random intercepts for both participants and items.

3.1.4. Results

Comparing the Ukrainian-Russian bilingual and English monolingual participants. In this analysis, we included stimuli where the grammatical gender of nouns was either congruent or incongruent with the character’s biological sex in both Ukrainian and Russian. An example of this would be ‘a ballerina’ (female) and ‘a pen’ (feminine in both Ukrainian and Russian) or ‘a ballerina’ and ‘an almond’ (masculine in both). Our expectation was that Ukrainian-Russian bilinguals would show stronger grammatical gender effects compared to English monolinguals. Specifically, we predicted that congruent pairs, where the character’s biological sex is congruent with the object label’s grammatical gender in 2 L1s, would receive higher similarity ratings. For instance, Ukrainian-Russian bilinguals were anticipated to rate a congruent pair, such as ‘a ballerina’ and ‘a pen’, as more similar than incongruent pairs like ‘a king’ and ‘a pen’. English monolinguals were not expected to show any significant trends.

Comparing the mean responses of Ukrainian-Russian bilingual participants in the congruent (Mean = 3.22, SD = 2.41) and incongruent (Mean = 3.28, SD = 2.41) pairs revealed nearly identical ratings, contrary to our predictions (see Figure 1). Notably, bilingual participants displayed slightly higher, but not statistically significant, average responses for stimuli with neuter grammatical gender (Mean = 4.00, SD = 2.55). In contrast, English monolingual participants consistently assigned similar ratings across all conditions (congruent: Mean = 4.24, SD = 2.38; incongruent: Mean = 4.30, SD = 2.17; neuter: Mean = 4.30, SD = 2.38), indicating that condition type did not notably influence their judgements of object-character similarity.

Figure 1. Comparison of Likert scores across conditions for Ukrainian-Russian Bilinguals and English Monolinguals: mean (dotted line) and median (solid line) differences in congruent, incongruent and neuter stimuli pairs in Experiment 1.

We built a CLMM to compare two groups of participants looking at the interaction between the group (Ukrainian-Russian bilinguals vs English monolinguals) and pair congruency in both L1s (congruent vs incongruent vs neutral), as a predictor for similarity ratings (Likert scores). Random intercepts were included for participants and items to account for variations specific to each.

The results revealed a statistically significant Group effect, with Ukrainian-Russian bilinguals exhibiting lower similarity ratings compared to the English controls (SE = 0.3318, z = −2.771, p = 0.006). However, there were no statistically significant main effects for pair congruency (SE = 0.2194, z = 0.165, p = 0.869) or for the interaction between the two variables. Specifically, the lack of significant group-condition interaction (SE = 0.1376, z = 0.888, p = 0.3744) demonstrated that, in contrast to our hypothesis, Ukrainian-Russian bilinguals did not rate incongruent pairs as less similar compared to the congruent pairs.

Comparing Ukrainian-Russian simultaneous bilinguals based on the Language proficiency in L1 and 2 L1. To compare the results of Ukrainian-Russian bilinguals only and investigate the effect of the more proficient first language (L1 or 2 L1) on similarity ratings, we conducted a separate analysis with different stimuli. This included noun pairs where grammatical gender matched the character’s biological sex in one language but not the other. For example, ‘a queen’ and ‘an onion’ (masculine in Russian, feminine in Ukrainian) were congruent in Ukrainian but incongruent in Russian. Conversely, ‘a king’ and ‘a sock’ (feminine in Ukrainian, masculine in Russian) were congruent in Russian and incongruent in Ukrainian.

Ukrainian-Russian bilinguals assigned ratings to pairs congruent in Ukrainian (Mean = 3.50, Range = 2.92–4.08) and pairs congruent in Russian (Mean = 3.17, Range = 2.59–3.74) when their proficiency was higher in Ukrainian (Figure 2). However, the differences in ratings were minimal and statistically non-significant, against our expectations.

Figure 2. Mean Likert Scale Responses from Experiment 1 per participant (Ukrainian-Russian bilinguals only) by Language Proficiency for pairs of stimuli where characters’ biological sex and objects’ grammatical gender are (a) congruent in Ukrainian and incongruent in Russian, and (b) congruent in Russian and incongruent in Ukrainian.

In the second CLMM, we explored whether Likert scores were influenced by the interaction between condition (biological sex and grammatical gender congruent in Ukrainian and incongruent in Russian vs congruent in Russian and incongruent in Ukrainian) and language proficiency (−100 to 100, with −100 being only Proficient in Russian, to 100 – only proficient in Ukrainian). The maximum convergence model included random intercepts for participants and items to account for participant-specific and item-specific variations. Contrary to our predictions, we found no significant effects for the condition-proficiency interaction (SE = 0.005, z = −0.784, p = 0.433), demonstrating that bilingual participants with higher proficiency in Russian did not assign higher ratings to the pairs that were congruent in Russian and incongruent in Ukrainian. Furthermore, no significant main effects for condition (SE = 0.3241, z = −0.741, p = 0.459) or language proficiency (SE = 0.0104, z = 0.725, p = 0.468) were found. Overall, our findings for the stimuli with mismatching grammatical gender in 2 L1s suggest that neither the individual variables nor their interaction significantly contributed to participants’ similarity ratings.

In summary, Experiment 1 revealed that gender congruence of noun-character pairs had no statistically significant impact on similarity ratings. Moreover, an unexpected pattern emerged, as Ukrainian-Russian bilinguals consistently rated objects as less similar than their English monolingual counterparts across all conditions. Our findings in this experiment align with the claims by Sera et al. (2002) that the presence of neuter grammatical gender may negate grammatical gender effects in speakers of three-gendered languages.

3.2. Experiment 2

The results from Experiment 1 suggest that including a neutral gender may have mitigated the significance of the language effect by diminishing the salience of grammatical gender. This raised the possibility that excluding neutral gender from the study design could affect the findings, particularly if the grammatical gender effects are online in nature and arise from real-time language effects. Therefore, in this study, we largely retained the methodology used in Experiment 1 but excluded the neuter gender from the stimuli.

3.2.1. Participants

40 English monolinguals and 70 Ukrainian-Russian bilinguals were recruited. After analysing their linguistic profile and responses, 64 bilinguals (44 females; Mean age = 30, SD age = 12) and 34 monolinguals (18 females; Mean age = 26, SD age = 6) were included in the analysis. Exclusions were due to participants either knowing other gendered languages (n = 6) or consistently using a single value on the Likert scale (n = 6), suggesting a potential lack of engagement or failure to follow instructions. The demographic distribution of the bilingual group in Experiment 2 was consistent with that of Experiment 1. As in the previous experiment, the largest proportion of bilingual participants held postgraduate degrees: 42.2% (n = 27). This was followed by 31.3% (n = 20) with undergraduate degrees, 18.8% (n = 12) with a high school diploma and 7.8% (n = 5 s) with a college degree. For the monolingual group, the distribution shifted slightly from Experiment 1. While postgraduate degrees remained the most common (35.3%, n = 12), the proportions for college and undergraduate degrees changed. In Experiment 1, college and undergraduate diplomas were equally represented, but in Experiment 2, 32.4% (n = 11) had a college diploma, 23.5% (n = 8) held an undergraduate degree and 8.8% (n = 3) had a high school education. Similar to Experiment 1, no effects of participants’ gender on their ratings were found (see Tables 4 and 5 in Supplementary Materials).

Analogously to the first experiment, we assessed bilingual participants’ linguistic profiles and proficiency in Ukrainian, Russian and English. Participants were recruited online and via posters at [ANOMYMISED]. The bilingual participants reported acquiring English (L2) at an average age of 9 years (Range = 4–20) and had at least an upper-intermediate proficiency level. Among them, 72% of participants demonstrated higher proficiency scores in Ukrainian and 28% in Russian. None of the participants reported using grammatical gender as a conscious strategy. The proficiency scores varied widely (see Table 3), demonstrating that ceiling effects were absent.

Table 3. Proficiency scores of Ukrainian-Russian bilinguals in Experiment 2

3.2.2. Materials

As with Experiment 1, participants were asked to rate object-character pairs using a 1 (not similar) to 9 (very similar) Likert scale. The stimuli consisted of 40 conceptually neutral black-and-white objects, categorised as follows: 10 masculine in both Russian and Ukrainian, 10 feminine in both languages, 8 feminine in Russian but masculine in Ukrainian and 12 feminine in Ukrainian but masculine in Russian. In addition, 16 characters (8 male, 8 female; the same as in Experiment 1) were used. To compensate for the reduction in stimuli due to the exclusion of neutral grammatical gender, we adjusted the number of trials in this experiment. Specifically, we paired each object with every character (rather than just one male and one female pairing per item as in Experiment 1), resulting in 640 unique pairs. This adjustment was made for two main reasons. First, the exclusion of neuter gender reduced the overall number of stimuli, which could have impacted the statistical power of the study, while increasing the number of trials helped to counterbalance this reduction. Second, in Experiment 1, pairings were pseudorandomised to minimise the risk of semantic associations (e.g., avoiding obvious pairings like ‘a broom’ with ‘a witch’). In Experiment 2, to eliminate this potential confound entirely, each object was paired with every character, thus increasing variability and reducing the chance of unintended semantic associations. The trial order was randomised for each participant, with objects presented on the left and characters on the right of the screen.

To ensure the validity of the data, we adopted enhanced measures, including comprehensive guidelines detailing the necessary procedures and environment for successful task completion. Additionally, participants were observed during the experiment. Any participant observed becoming distracted or communicating in their native languages was excluded from the analysis (13 bilingual and 11 monolingual speakers).

3.2.3. Procedure and design

The approach for Experiment 2 closely followed that of Experiment 1 but with the inclusion of participant observation conducted via Zoom. An experimenter monitored each session to ensure that participants were focused, free from distractions and not using their native language during the task. All interactions were done in English and if participants needed clarifications, they did so in English as well. In Experiment 2, we also modified the verbal instructions to emphasise the use of the entire response scale (1 to 9). This adjustment was made based on observations from Experiment 1, where some participants tended to limit their responses to a narrower range of the scale. The experimenter used intonation to explicitly highlight this request during the verbal instructions while maintaining the original instructions from Experiment 1. The modified instructions, given in English, were as follows: ‘In this study, you will see pairs of pictures appear on the screen. In each pair, there will be a picture of a person on the left and a picture of an object or animal on the right. You will see a scale where 1 = not similar and 9 = very similar. For each pair of pictures, please choose a number between 1 and 9 to indicate how similar you think the two pictures are. Try to use the WHOLE scale (give some 1’s and some 9’s and some of all the numbers in-between). Please respond with the first answer that comes to mind. Please try not to be distracted and avoid communicating with anyone (unless necessary) until the experiment is complete’. The final sentence, instructing participants to avoid distractions and communication, was added specifically for experiment 2 to help maintain task focus.

The analytical approach remained consistent with that of Experiment 1, employing a similar structure for the CLMMs. The analysis comprised two parts. In the first part, we compared the responses of English monolinguals and Ukrainian-Russian bilinguals. This comparative analysis explored the effects of pair congruence (congruent vs incongruent in both Russian and Ukrainian) and group (Ukrainian-Russian bilinguals vs English monolinguals) interaction on Likert scores. The second part focused on examining responses from Ukrainian-Russian bilinguals only, assessing the effect of pair congruence (congruent in Ukrainian/incongruent in Russian vs congruent in Russian/incongruent in Ukrainian) and language proficiency (−100 to 100, with −100 being only Proficient in Russian, to 100 – only proficient in Ukrainian) interaction on similarity ratings. In both parts of the analysis, the maximum convergence models included random intercepts for participants and items.

3.2.4. Results

Comparing the Ukrainian-Russian bilingual and English monolingual participants. Consistent with our predictions, bilinguals assigned significantly higher ratings to pairs with congruent biological sex and grammatical gender in both L1 and 2 L1 (Mean = 5.8, SD = 2.0), as opposed to the incongruent pairs (Mean = 3.4, SD = 1.73). Besides, as confirmed by pairwise comparison, bilinguals rated congruent pairs significantly higher than monolingual participants (Mean = 4.12, SD = 2.4). As for the incongruent pairs (Figure 3), Ukrainian-Russian bilinguals tended to rate them significantly lower (Mean = 3.4, SD = 1.74) than English controls (Mean = 4.13, SD = 2.4). For the English monolingual group, there was no significant difference between the ‘congruent’ and ‘incongruent’ conditions (estimate = 0.0115, SE = 0.0245, z = 0.47, p = 0.639).

Figure 3. Comparison of Likert scores across conditions for Ukrainian-Russian Bilinguals and English Monolinguals: mean (dotted line) and median (solid line) differences in congruent and incongruent stimuli pairs in Experiment 2.

* p < .05. ** p < .01. *** p < .001.

Analogously to the first experiment, a CLMM examined the interaction between the group (bilingual vs monolingual) and condition (congruent vs incongruent in both L1s), as a predictor for similarity ratings. The results revealed a statistically significant group effect for Ukrainian-Russian bilinguals (SE = 0.0888, z = 16.38, p < 0.001). We also found significant effects for the bilingual group-condition interaction, indicating that bilinguals assigned a significantly lower rating to the incongruent pairs (SE = −1.9301, z = −55.15, p < 0.001) than English monolinguals. These findings confirmed our hypothesis that matching grammatical gender in both languages of bilinguals significantly affects their categorisation once neutral gender is excluded from the testing conditions.

Comparing Ukrainian-Russian simultaneous bilinguals based on the language proficiency in L1 and 2 L1. Figure 4 illustrates a clear difference in ratings, in line with our expectations. Ukrainian-Russian bilingual participants who were more proficient in the Ukrainian language gave significantly higher similarity ratings to object-character pairs where the object’s grammatical gender in Ukrainian was congruent to the character’s biological sex (Mean = 4.99, SD = 2.26), compared to pairs congruent in Russian (Mean = 4.56, SD = 2.38). Conversely, those with higher proficiency in Russian tended to give significantly higher ratings to pairs congruent in Russian (Mean = 5.12, SD = 2.24) than to incongruent ones (Mean = 4.54, SD = 2.26).

Figure 4. Mean Likert Scale Responses from Experiment 2 per participant (Ukrainian-Russian bilinguals only) by Language Proficiency for pairs of stimuli where characters’ biological sex and objects’ grammatical gender are (a) congruent in Ukrainian and incongruent in Russian, and (b) congruent in Russian and incongruent in Ukrainian.

The designed CLMM tested the impact of the interaction between condition (congruent with Ukrainian language and incongruent with Russian vs congruent with Russian language and incongruent with Ukrainian) and language proficiency (−100 to 100). While no significant main effect for Condition (SE = 0.0292, z = 0.475, p = 0.635), a significant main effect of Proficiency (SE = 0.0022, z = −1.960, p = 0.05) was observed. Besides, as predicted, a significant interaction was found between condition and proficiency (SE = 0.0013, z = 8.622, p < 0.001). This suggests that the interaction between the most proficient L1 of a simultaneous bilingual and condition had a significant impact on categorisation, and those bilingual participants that were more proficient in Ukrainian rated pairs that were congruent in Ukrainian and incongruent in Russian as more similar, and vice versa for those more proficient in Russian.

4. Discussion

The current study aimed to explore how language, grammatical gender in particular, affects cognitive processes of Ukrainian-Russian bilinguals in an all-English context. The group was chosen for several reasons. First, Ukrainian and Russian grammatical systems have nouns with both matching and contrasting grammatical gender across languages. Secondly, both languages have three grammatical genders (masculine, feminine and neuter). Incorporating Ukrainian and Russian languages is beneficial for linguistic relativity research because, unlike previously studied languages such as Italian, Spanish, French or German, they lack articles that could conflict with the biological sex of the referent. Instead, grammatical gender in Ukrainian and Russian is predominantly marked through noun, adjective and sometimes verb endings. This distinct morphosyntactic feature – where gender is conveyed directly through morphological changes rather than through articles or fixed gender markers – has often been overlooked in existing research focused on languages with different gender-marking strategies.

Moreover, one of our research interests in the present study was to contribute to the discussion of whether gender effects arise online or offline, by examining whether having neuter gender embedded in the grammatical systems of both Ukrainian and Russian would lead to diminished grammatical gender effects. Therefore, we adapted one of the seminal studies on grammatical gender (Phillips & Boroditsky, 2003), while manipulating grammatical gender in Ukrainian and Russian, as well as presence (Experiment 1) and absence (Experiment 2) of neuter gender in testing conditions.

In Experiment 1, we observed a lack of significant effects of grammatical gender and group, as well as their interaction, when comparing the ratings of bilingual and monolingual participants. Additionally, we found no effects of the interaction between language proficiency and grammatical gender in Ukrainian-Russian bilinguals, indicating that their more proficient language had little to no effect on similarity judgements. Such findings align with previous research that reported a lack of grammatical gender effects on speakers of three-gendered languages, such as German (Sera et al., 2002; Vigliocco et al., 2005).

However, after excluding neuter gender in Experiment 2, a significant interaction between group and condition was found when comparing bilingual and monolingual groups, indicating that Ukrainian-Russian bilinguals rated higher in those pairs where grammatical gender of an object in both Ukrainian and Russian was congruent with biological sex of a character, compared to the incongruent pairs. Additionally, a significant interaction between condition and language proficiency was observed, when only simultaneous bilinguals’ results were analysed. The latter demonstrated that bilinguals with higher proficiency in Ukrainian rated those pairs as more similar where grammatical gender and biological sex were congruent in Ukrainian and incongruent in Russian. The analoguous effect was observed for speakers more proficient in Russian, as they perceived the pairs congruent in Russian to be more similar than those congruent in Ukrainian.

Before discussing differences between the experiments, we should first explore the possible reasons for the null results in Experiment 1. The absence of significant results in the first experiment might be attributed to several factors. Firstly, as suggested by Sera et al. (2002) and Vigliocco et al. (2005), three-gendered grammatical systems may not show effects as strong as those in two-gendered languages with more direct and intuitive associations between grammatical gender and natural gender, which can lead to stronger perceptual biases. In contrast, three-gendered systems which include a neuter gender, introduce a level of grammatical complexity that may obscure the relationship between gender and categorisation. The neuter gender, in particular, could have reduced the salience of masculine and feminine distinctions, thereby weakening potential gender effects. Secondly, the broader lack of support for findings using this paradigm may reflect ongoing issues related to the replication crisis in linguistic relativity research. As mentioned earlier, most previous attempts to replicate Phillips and Boroditsky (2003) have not yielded significant results, except for Pavlidou and Alvanoudi (2019). For instance, Elpers et al. (2022), even with an increased sample size, failed to provide significant results using the linear mixed effects models, though analysis using the t-tests showed significance. This issue is exacerbated by methodological variations and by the use of different statistical analyses across studies that employ the same paradigm, which makes it challenging to compare results consistently. Finally, the unique linguistic profiles of participants, which often differ across research contexts, add another layer of complexity. Previous studies that used a similarity judgement task also focused on bilingual participants, but there is limited consistency in how those participants were selected or their linguistic profiles were characterised. Key details, such as whether participants spoke other gendered languages and the criteria used for proficiency self-assessment are often not reported in sufficient detail. This variability makes it difficult to draw meaningful comparisons across studies, as differences in participant characteristics could significantly influence the observed effects – or the lack thereof – of grammatical gender. However, a key unifying factor between our study and those conducted by Sera et al. (2002) and Vigliocco et al. (2005) is the inclusion of neuter gender in the stimuli. This suggests that the presence of neuter gender may have influenced the absence of grammatical gender effects observed across these studies.

The discrepancy in language effects between Experiments 1 and 2 could be attributed to variations in experimental design, such as increased number of stimuli, variation in instructions or participant observations in Experiment 2, as well as lack of neuter gender in the task. While we initially hypothesised that the absence of neuter gender would be primarily driving the observed differences, it is important to consider that other methodological changes may also have contributed. First, the increased number of stimuli in Experiment 2 likely enhanced statistical power, providing a clearer picture of language effects that might have been less detectable in Experiment 1. Besides, the increased number of pairs allowed us to account for the possible semantic associations in Experiment 2 that could have emerged in Experiment 1 (e.g., pairing ‘a broom and ‘a witch’ together). To examine the potential outcomes of using only the stimuli from Experiment 1 within the context of Experiment 2, an additional analysis was conducted with this subset. This analysis, which included 72 pairs of stimuli from Experiment 1, confirmed a robust and significant effect for both types of stimuli, consistent with the results obtained from the full stimuli set in Experiment 2. These findings strengthen the interpretation that the absence of neuter-gender stimuli in Experiment 2 may be a driving factor behind the observed grammatical gender effects, further validating our findings. Detailed analysis has been included in the Supplementary Materials (pp. 9–10). Second, the modified verbal instructions emphasised the use of the entire scale (1 to 9), which may have influenced participants to use a broader range of responses. Third, the addition of participant observation via Zoom allowed the experimenter to ensure that participants remained focused and did not revert to their native language.

However, it is also possible that the observed differences in the results were primarily due to the absence of the neuter grammatical gender, as hypothesised. This effect may be explained by considering the distinction between online and offline language processing discussed in the literature. According to both the label-feedback (Lupyan et al., 2020) and structural-feedback hypotheses (Sato & Athanasopoulos, 2018), online effects occur when language actively modulates perception and decision-making in real-time, influenced by top-down feedback from specific linguistic labels and broader structural patterns, respectively. In Experiment 1, the inclusion of neuter gender may have diluted the salience of masculine and feminine categories, reducing the immediate impact of gender cues on participants’ judgements. Neuter nouns might have introduced a neutral, less distinctive category that disrupted the online processing of gender, as it did not align with the binary masculine-feminine distinction. This aligns with findings from previous research, which suggest that the presence of a third, neuter category can weaken the perceptual link between grammatical and natural gender – not in the offline manner as claimed by Sera et al. (2002), but during the process of task completion. In Experiment 2, by excluding neuter gender, the task environment emphasised over the course of the experiment the binary masculine-feminine distinction, creating a feedback loop where the structure of the gender system becomes more entrenched and influences real-time (online) processing more strongly. Without the neutral baseline provided by neuter nouns, participants were more inclined to use the salient gendered cues actively, resulting in more pronounced effects. This suggests that the grammatical gender effects observed in Experiment 2 were primarily driven by the immediate, context-sensitive use of gender information (i.e., online effects), but also by the reinforcing influence of the underlying linguistic structure on cognitive processing (i.e., a structural feedback effect).

In sum, our study shows that such an effect does not have its roots in the mere presence of the neuter gender in a language’s grammatical system, but rather arises online, as a function of the absence of the neuter gender in the task. Such an interpretation is compatible with modern accounts of the mechanisms underpinning linguistic relativity effects, such as the label-feedback hypothesis (Lupyan, 2012) and the structural-feedback hypothesis (Sato & Athanasopoulos, 2018). These findings also align with earlier research and demonstrate that three-gendered languages do indeed impact cognitive processes, such as categorisation. Furthermore, the language effects are present even when grammatical genders do not match in the two languages of simultaneous bilinguals, as they rely on the grammatical gender of their more proficient language.

The complexity of our findings underscores the necessity for more nuanced research methodologies. The similarity judgement task is merely the first step in analysing gender effects within our new group of participants. We suggest that future research employ more rigorous methodologies to further investigate these effects. For instance, incorporating neurophysiological measures, such as event-related potentials (ERPs), to better elucidate the effects of grammatical gender on bilingual cognition. This could be done by adapting previously used paradigms by Sato, et al. (2020) or Boutonnet et al. (2012) to investigate whether grammatical gender primes conceptual or semantic representations (looking at N300 or Left Anterior Negativity respectively) in speakers of three-gendered languages compared to speakers of two-gendered languages that were used in these two studies. Additionally, we recommend expanding the range of stimuli used to test speakers of multiple three-gendered languages. For example, future research could include nouns that have masculine or feminine grammatical gender in one language (L1) and neuter gender in the second language (2 L1). This expansion would provide further insights into the influence of grammatical gender on bilingual cognition, grammatical gender representation in simultaneous/early bilingual’s mind and contribute to the broader field of linguistic relativity.