Lemma navigation by excellent secondary school students of Ancient Greek

Abstract

Under the high cognitive pressure of a translation task, secondary school Classics students often make dictionary mistakes. Research indicates that successful students use a lemma's meta-information to engage in a feedback loop between text and dictionary. Using eye-tracking data of expert learners, this study examined how successful students employ the different types of lemma information while limiting their cognitive load. We found that students primarily manage their cognitive load by using boldface definitions as stepping-stones, but navigate through meta-information when it (1) can effectively reduce the amount of definitions and (2) refers to (morphological) features directly visible in the text. Alternatively, they use meta-information as a backup plan, when they are stuck or want to confirm a hypothesis.

1. Introduction

Secondary school Classics students often produce awkward, incoherent translations because of the way they use their dictionaries. Research in the Netherlands and Germany shows that students consult the lexicon excessively and tend to lose sight of the syntax of the original sentence. They mechanically replace all Greek and Latin words with their respective modern language dictionary equivalents and start to build a translation from there (Eikeboom, 1967; Van Krieken, 1981; Florian, 2017; Bartelds, 2018).

A considerable number of translation mistakes seem to be the result of ‘semantic tunnel vision’, i.e. the tendency to focus exclusively on the definitions in a lemma, while ignoring all other (meta-)information. Students apparently concentrate on the first translation possibility in a lemma, thinking this possibility is the meaning (Florian, 2017, p.152).

Research on dictionary behaviour while translating a text suggests that successful secondary school students of Ancient Greek engage in a feedback loop, of actively moving back and forth between text and dictionary (Bartelds, 2021). These ‘expert learners’ specifically employ meta-information in a lemma to return to the text in an informed manner. A label with subj., for example, can prompt a student to check whether there is in fact a subjunctive in the sentence. By using meta-information, semantic tunnel vision can be avoided.

This article reports on an eye-tracking experiment in which we explored in more detail how expert learners of Ancient Greek navigate through a lemma¹. The goal of this qualitative study was to investigate how expert learners deal with different types of meta-information that organise a lemma. Because meta-information often offers shortcuts to the relevant (sub)section of a lemma, it can facilitate efficient navigation. Expert learners, however, are still learners and employing meta-information may cost them a lot of cognitive effort. We were specifically interested, therefore, to explore what type of meta-information they decided to use and in which circumstances. In addition, we compared their usage (or ‘affordance’, see section 1.1) to the lexicographical design intentions. From the results we hope to draw both pedagogical and lexicographical lessons.

In the following two paragraphs, we elaborate on two concepts that are central to the design and analytical framework of this study: affordances and cognitive load theory.

1.1 The affordances of lemma-information

To better understand the possible uses of the different types of lemma-information, the concept of affordances is helpful. The term, first coined by Gibson (1966), was defined by Norman (1988) as a possibility to act. Thus, an affordance of a chair is ‘to sit on’. An affordance is a function of the relationship between (the characteristics of) the object and (the characteristics of) the person perceiving the object. Consequently, a certain object can ‘afford’ different actions to different persons. This means that the intended affordance can differ from the affordance that a particular person perceives. From the perspective of a small child, a chair may rather afford ‘to hide under’. To refer to a certain affordance in relation to a specific person, the term ‘perceived affordance’ is used.

Level of expertise can determine whether someone will be aware of an affordance. An indoor climbing wall, for example, with numerous coloured hold types of various sizes and shapes, offers different affordances, depending on the experience of the perceiver (Seifert et al., 2017).

The design of a lemma can be regarded as a collection of affordances intended for a dictionary user. Below we will illustrate four different types of lemma-information by means of a lemma taken from the lexicon used in this study and discuss their affordances.

1. (1) Definitions are the translation possibilities, rendered in boldface. The affordance of definitions is to form a semantic field of the various senses of the word. The bold typeface has the affordance of ‘stepping-stones’: it invites notice and allows a focus exclusively on the definitions, ignoring other information.

The boldface definitions are surrounded by the following types of meta-information (see also Figure 1).

1. (2) Signposts (in italics, often abbreviated) are placed at the start of a (sub)section, and define a semantic, morphological or syntactic condition governing the whole (sub)section. By imposing hierarchical structure on a lemma, they afford to quickly select or exclude parts of the lemma.

2. (3) Labels (in italics, often abbreviated) are placed anywhere on, but never at the start of, the lowest hierarchical level, directly preceding and defining a definition or example. Because labels further specify the (semantic, morphological or syntactic) usage, they allow users to assess the relevance of a translation option in more detail than a signpost.

3. (4) Examples consist of a Greek quote, a translation and a reference to the author and work in which it occurs. An example offers an illustration, which invites making an analogy with the context to be translated.

Figure 1. Examples of the types of meta-information.

The question is whether secondary school students attend to the above affordances and whether their perceived affordances accord with them. Moreover, lemmata typically differ in complexity and hierarchical structure, and thus in the types of affordances present. Dictionary users, therefore, need to adapt their navigational strategies to the lemma at hand. Are such adaptations in strategies visible in lemma navigations of expert learners? We expected that expert learners would indeed perceive affordances and would adapt their strategies. We were specifically interested in the extent to which they would do this, and expected that this could be connected to cognitive load theory.

1.2 Cognitive load management

When we investigate the navigational behaviour of secondary school students, it is important to consider the cognitive pressure involved in the translation task. Secondary school students are learners and translating Ancient Greek is a complex task, which makes it necessary to carefully manage their cognitive load. In that respect, we can agree that semantic tunnel vision is an ineffective attempt to do so, but what constitutes a successful, indeed, a strategic attempt?

According to Cognitive Load Theory (Sweller, 1988), task-directedness is crucial in effectively accepting cognitive load: activities that directly contribute to performing a task are beneficial (germane), whereas activities that are not (extraneous) should be avoided. Meta-information in a complex lemma can be seen as a collection of anchor points that enables the user to connect a word in the text to the relevant section of a lemma. In such lemmata, meta-information provides a shortcut to the relevant definition. Thus, employing this type of lemma-information is an example of germane cognitive load. It facilitates engagement in a feedback loop between text and dictionary.

The fact that using meta-information is a form of germane cognitive load, however, does not mean that it is in all circumstances an efficient strategy for students. Successfully decoding and applying meta-information can be a difficult and time-consuming process for secondary school students. This means that we can expect that learners will perform some kind of cognitive cost-benefit analysis while navigating, which influences their selective attention to the available information. In our study, we wanted to find out how expert learners deal with this trade-off and how they assess the various types of meta-information in terms of this analysis.

2. Method

Eye-tracking has been used as a method to investigate the look-up process for modern language learning (e.g., Tono, 2011) and, within Classics, to examine general translating behaviour (e.g., Luger, 2018), but our study is, to our knowledge, the first to investigate lemma navigation for Ancient Greek. For our expert-learner study we recorded the eye movements of 14 excellent secondary school students while studying a lemma during a translation task. Subsequently, we asked them to reflect on their behaviour, using the eye-tracking video as a prompt (stimulated recall).

2.1 Recruitment and selection procedure

Teachers of six Gymnasia in the Netherlands helped to recruit candidates with excellent translation skills from the penultimate (fifth) or final (sixth) year. Candidates were informed that the study would examine their translation behaviour; but not that the focus of the experiment was their dictionary behaviour.

First, candidates were selected on the basis of an unseen translation of Plato's Laches 179c2-d5, a concrete text not previously studied by any of the participants, although they were familiar with genre and dialect. The Dutch national examination authority validated the test's design, score model and level of difficulty.

Out of 48 respondents we selected the 14 participants with the highest scores (i.e., all candidates with a score of 8 or higher on a scale of 1–10)². This number allowed for the in-depth qualitative analysis befitting our research question. The number is comparable to the number of participants in other studies on translation habits in Classics (e.g., Eikeboom, 1967, Van Krieken, 1981, Florian, 2017, Luger, 2018) and we expected that it would yield enough data to reach saturation.

2.2 Materials

Every task sheet had three fields: the Greek sentence, the glosses (aantekeningen) and the ‘dictionary’ (woordenboek), i.e., a lemma corresponding to a target word in the sentence (Figure 2).

Figure 2. Translation task example sheet.

The new Greek-Dutch dictionary by Sluiter, Kessels and Rijksbaron was used as a source³. This lexicon was chosen because it is new and secondary school students are among its intended users. The alternative school lexicon is less elaborate and therefore less suitable for investigating lemma navigation. Team members of the lexicon were consulted on the rationale behind the different types of meta-information and confirmed our analysis of the intended affordances.

In the tasks, some of the lengthier lemmata were trimmed to fit into the corresponding frame. Most of the other words in the sentence were glossed, so the target word would be the only translation problem to solve. The six tasks consisted of short sentences from prose texts in the Attic dialect.

Because of the explorative nature of this study, we aimed to confront the participants with a variety of lemma-information. We therefore selected lemmata with relatively complex architectures, rich meta-information and multiple, distinct senses. The tasks were presented to the participants in increasing order of difficulty of the signposts describing the hierarchical structure of the lemmata involved.

We started with a purely semantic distinction in πλοῦς; the morphologically divided συμ-βάλλω was used for the second and third tasks; the fourth, καθ-ίστημι, features long and complex morphological and syntactic signposts; the fifth, δια-τελέω, has an asymmetric threefold division; the sixth, πρίν, was included as an example of a very complex lemma of a word that should be too elementary to look up at all. Moreover, in four of the six lemmata a part of the task sentence was given as an example. This allowed us to investigate whether students would notice such information at the lowest hierarchical level in the lemma.

2.3 Experiment setup

The experiment was conducted in an eye-tracking lab. Participants were seated at a desk in an enclosed booth, their head in a chin rest and facing a monitor equipped with the Eyelink 1000 system (Figure 3). The researcher was in the adjacent room and could communicate with the participant through a microphone. He was able to watch the participant's eye movements via a live feed on his screen.

Figure 3. Setup of the booth.

2.4 Procedure

After some preliminary technical steps, (e.g., setting the height of chair and chin rest), an example sheet was shown to familiarise the participant with the default layout of the translation tasks. In the actual experiment, the six translation tasks were presented (to all participants in the same order). The researcher orally introduced the context for each translation task; this information could be repeated at any time during the task at the request of the participant. During the whole experiment, participants could not move from their chin rest, which meant they could not take notes or write out their translation. Instead, voice recordings were made of their translations and they were asked to start their definitive translation formally by saying ‘the translation of number x is…’. No time limit was imposed, but participants were asked to treat the experiment as if it were a school translation test.

Directly after the last translation task, a stimulated-recall interview was conducted in which participants were shown a video of their eye movements during the task (Figure 4). The researcher invited the participants to reflect on their video, asking questions such as ‘Do you remember what you were thinking here?’, ‘Why were you switching between these two words?’, ‘Why did you choose this translation?’ etc. The transcriptions of these interviews formed the main data set for the study.

Figure 4. Video still of eye tracking data (purple dot indicating fixation).

3. Experiment

In this section we report on our experiment. For the reader's convenience, we present the findings of each translation task together with the materials used for it. For each translation task, we first discuss the available lemma-information for finding the right definition, then the task's sheet and introduction are shown, after which we present a brief overview of the results in numbers and conclude with a more detailed qualitative analysis⁴. In this analysis, we refer to the terms we used to distinguish the various types of lemma-information and the concepts of affordances and cognitive load as our theoretical framework. The main question is how the participants use the available lemma-information to arrive at the desired definition. We illustrate our insights with key quotes from the stimulated recall interviews.

3.1 One sentence, two meanings of πλοῦς

The first translation task (Figure 5) is different from the following tasks, because the lemma does not contain any signposts. The entry of πλοῦς offers two distinct meanings, both of which are needed for the task. The first instance of πλοῦς corresponds to the second meaning, ‘the time or favourable circumstances to sail’ (tijd of gelegenheid om te varen), while the second requires the first: ‘sailing, sea voyage’ (vaart, zeereis). The available lemma-information consists of boldface definitions and several examples, sometimes preceded by the label ‘figurative’ (overdr.). We were specifically interested to see whether our expert learners would look further than the first definition, avoiding semantic tunnel vision.

Figure 5. Translation task 1.

3.1.1 Translation task

3.1.2 Results

The second instance of πλοῦς was translated correctly by everyone, but three participants also translated the first as ‘voyage’, instead of ‘time to sail’. Regarding the type of information, ten reported that they limited their attention to the boldface definitions, while the eye-tracking data of four participants showed that they also scanned (some of) the examples.

3.1.3 Qualitative analysis

In this task, the dominant role of the boldface definitions in the students’ navigational behaviour is evident. The definitions have the affordance of stepping-stones: to easily jump from one translation option to the next, without paying attention to the other information. Student number 38 (S38) illustrates this approach:

The students who did examine the examples reported that they were scanning the Greek words in the lemma to look for a possible one-to-one match with the Greek in the sentence.

These reports indicate the perceived affordance of an example for these students: they are not looking for an opportunity to draw an analogy – the intended affordance of an example – but they use examples only to check whether the (exact) Greek words of the task are included in the lemma. From the perspective of managing cognitive load, this is an understandable approach: this is not a very demanding process. It would be much more demanding to check whether an analogy could be made. The fact that no student takes account of the label figurative (overdr.), which defines the examples given in 1.a., fits this analysis. To decide whether a word is used figuratively requires a more comprehensive evaluation.

What about the three participants who mistakenly selected ‘voyage’ instead of ‘time to sail’ for the first instance of πλοῦς? In two cases eye-tracking suggests no attention at all was paid to the second section of the lemma. Here semantic tunnel vision seems to be in play: they believe that the first definition is adequate and they stop looking for other options (even though they end up with an awkward translation). The third student did briefly look at the second section but thought that the corresponding meaning was ‘time’ and therefore discarded it quickly. They had stopped reading immediately after the word ‘time’ (tijd) – not realising that the definition was not finished yet. The fact that the boldface is interrupted by the word or (of) in italics obstructed their reading eye. This illustrates the strong impact of the boldface on processing lemma-information.

3.2 An intimidating lemma συμ-βάλλω (1): morphological signposts

The lemma συμ-βάλλω, used in tasks 2 and 3, is relatively long and complex, which students often find intimidating (Figure 6). However, the signposts provided do allow to reduce the amount of information by eliminating irrelevant sections. The main division is marked by the morphological signposts act. (active) and med. (middle). Section 2.a. includes the meaning relevant for this task: ‘to contribute’ (bijdragen). This subsection features two examples, the second of which matches the sentence of the task.

Figure 6. Translation task 2.

3.2.1 Translation task (1)

3.2.2 Results

All 14 participants chose the correct translation. Although all participants recognised the active/middle division, ten actually limited their attention to the second section of the lemma. A total of six noticed the matching example in 2.a. Only three used its syntactic label.

3.2.3 Qualitative analysis

It is clear that the affordance of the middle-signpost to exclude the first section of the lemma was realised by many participants. It appears to have a positive outcome in terms of a cost-benefit analysis. Below we can see that S43 is initially intimidated by the length of the lemma, but then gratefully (‘oh, -οντο, that's middle’) dismisses half of it. For S43 it is evidently not very cognitively costly to apply the signpost.

From students’ reflections on how they found the matching example in the entry, two different views emerge. First, S6 detects the label εἰς and connects it to the εἰς in the sentence. This is one of the few cases where a label is used by a student. S6 explains that they studied section 2.a. more closely because they already suspected they were at the right spot (‘Of course I was thinking already that ‘contribute’ was a good translation’) and was actively looking for more correspondence between sentence and lemma (‘I want it to match as much as possible’).

By contrast, S22 reports that they took a closer look at the information in 2.a. because they were struggling to integrate the word in the translation of the sentence as a whole (‘at the moment I get a bit stuck or can't really think of a good translation’).

3.3. An intimidating lemma συμ-βάλλω (2): morphological and semantic signposts

The second task involving the lemma συμ-βάλλω features the word συμβαλεῖν, which means only the ‘active’ part of the lemma is relevant, and from this part the third subsection (1.c.) is needed, marked with the semantic signpost of mental processes (van mentale processen) (Figure 7). There, the students will require the second set of definitions: ‘to understand, interpret’ (begrijpen, interpreteren). Two examples illustrate possible objects; the second example (τὴν μαντείαν) matches the task. This task specifically tests what students do with a semantic (instead of a morphological) signpost that is placed on the second hierarchical level.

Figure 7. Translation task 3.

3.3.1 Translation task

3.3.2 Results

All participants limited their attention to the first part of the lemma. 13 chose the correct translation, six noticed the matching example τὴν μαντείαν, and only two used mental processes as a signpost to navigate to the desired definition.

3.3.3 Qualitative analysis

Students did indeed attend to the affordance of the signpost on the first hierarchical level (act.), but then mostly reverted to their default strategy using boldface translations as stepping-stones. The other meta-information, the example and semantic signpost, is evidently considered much less attractive as a way to navigate to the right definition.

With respect to using examples, S33 offers a telling quote. They explain how they tried out each translation option in the first section with ‘oracle’ as an object. When asked whether they had noticed the example τὴν μαντείαν they reveal their ‘policy’: ‘No, not at all. I don't really read on any further!’.

By contrast, we learn why S11, who also finds the correct definition by scanning the boldface translations, does spot the example. For them, finding a promising translation means they are stimulated to scrutinise the corresponding part of the lemma.

With regard to the semantic signpost, two participants successfully used mental processes to navigate to the relevant section. S37, for instance, reports that they had already hypothesised the relevant meaning from the context and decided to scan the first section to find it. Because the first translations did not match their expectation, they decided to change their strategy and use the semantic signposts (‘the bits that say what it is about’).

Most participants, however, did not (successfully) realise the affordance of this signpost as a shortcut to select one of the three subsections. Reports indicate that students had difficulties in correctly applying it to the translation task. S6 points out that initially they did try applying the semantic signposts, but these did not offer them enough clarity. It is telling that they even discard the relevant signpost: ‘it's also not really a mental process’. S6 then reverts to examining the boldface translations: ‘it's also just about what is logical’ and thus ends up at the correct translation.

S32 also attempts to navigate following the semantic signposts. They think that ‘of things’ is the most appropriate signpost and then choose a wrong translation. The eye-tracking data, however, showed that they were gazing at ‘understand, interpret’, and S32 confirmed that they thought ‘understand’ was the best translation. It turned out that they rejected this option on the basis of the label with ὅτι-clause (met ὅτι-zin). The student interpreted this label as a necessary condition⁵.

After three translation tasks we can observe that the default strategy to navigate through a lemma is determined by the boldface translations. This can be explained by cognitive load management. It appears that the decision to attend to the surrounding meta-information depends on the possibility to reduce the number of translations to check. This means that signposts, because they are conditional, are most profitable. Nevertheless, when a signpost is cognitively costly to apply, students quickly revert to the main navigation strategy of jumping on stepping-stones.

The fact that the active/middle signposts were popular to use can be explained by the fact that these refer to readily discernible concrete characteristics of the Greek text. Most participants realised that the signposts allow them to reduce the lemma συμ-βάλλω by half: low cognitive costs, high gain. In order to profit from the semantic signposts, however, students needed to choose between not two, but three options, which all required a higher level of abstraction to determine the relevant entity of the object.

In addition, we have observed behaviour that can be regarded as a ‘workaround’ when meta-information is costly to apply. Some students (for example S11 in quote 5 above) have a habit of focusing first on boldface definitions, but then, after having found a promising candidate, widen their scope again for meta-information. This strategy is not so much a navigational, but a confirmational approach. This can be seen as an effective way to overcome cognitive pressure: it may be too costly to employ meta-information to systematically navigate, but it can still be a powerful tool to locally verify a preliminary chosen translation. A related strategy can be observed in the quote of S22 in the second translation task, who says that they decide to pay attention to meta-information as soon as they get stuck. Both approaches have in common that they come as a backup plan, after students have tried – either successfully or unsuccessfully – to navigate with semantic stepping-stones.

3.4 καθ-ίστημι: dealing with long, complex signposts

The signposts distinguishing the two main sections of καθ-ίστημι are relatively long and make use of two grammatical categories: syntax and morphology. Section 1 is about transitive usage (met acc.); in this usage, the verb has a sigmatic aorist. Section 2 describes intransitive usage, with root aorist (stamaor.). The translation task contained the active κατέστησεν, and the correct meaning is found in 1.c.: ‘to put in a certain state’ (in een bepaalde toestand brengen) (Figure 8). The sentence is listed as an example again. We wondered whether our students would use the signposts and if so, which type of information, syntactic or morphological, would be preferred.

Figure 8. Translation task 4.

3.4.1 Translation task

3.4.2 Results

Of the 11 participants who chose the correct translation, six students made use of the signposts: three relied on the morphological and three on the syntactic distinction. The fiveremaining successful students just scanned the lemma from the top to find a suitable translation. The three students who chose the wrong translation all used the distinction based on morphology, but incorrectly identified κατέστησεν as a root aorist. Nine students noticed that the phrase was included as an example in section 1.c.

3.4.3 Qualitative analysis

When we compare the signposts in καθ-ίστημι to those in συμ-βάλλω, we see that fewer students used them and fewer were successful in doing so. It appears that, although the same amount of reduction is at stake (ca. 50%), the cognitive investment required to employ the signposts is often estimated to be too costly to divert students from the boldface translation ‘hopping’ (S38 illustrates this point nicely).

Closer inspection indicates that the term ‘intrans.’ is particularly hard to process. S24 is puzzled by it (‘intrans-something’) and then decides to focus on the first section, but not on syntactic or morphological grounds, but merely because they think that ‘active’ is a discriminating term in this lemma as well. For this participant the signposts were too difficult to decode.

In the data of this task, we also see cases of the confirmational approach explained in 3.3.3. Participant S13 consults the lemma with the meaning ‘to bring’ already in mind and is first mainly helped by the label ‘in a certain state’ (in een bepaalde toestand) at the beginning of 1.c. They then proceed to find definitive confirmation in the morphological signpost, by checking the type of aorist.

3.5 The asymmetrical division of δια-τελέω

The lemma δια-τελέω has a rather asymmetrical syntactic division marked with the signposts ‘with accusative’ (1.), ‘with participle’ (2.a.) and ‘with adjective’ (2.b.). In addition, the header of section 2 offers the translation instruction ‘verb to be translated as ‘continuously’’ (ww. te vertalen als ‘voortdurend’). The example illustrating the right definition (under 2.b.) is not a literal match with the target sentence (Figure 9).

Figure 9. Translation task 5.

We wanted to know how the students would deal with this kind of signposts and the general translation instruction. The signpost ‘with accusative’ allows discarding the first part, which is rather long and filled with different examples and labels. The glosses provided for ἀνυπόδητος and ἀχίτων (containing the abbreviation ‘adj.’) could facilitate using the corresponding signpost of 2.b.

3.5.1 Translation task

3.5.2 Results

All 14 participants chose the correct translation, nine using the signpost ‘with adj.’ and five focusing on the boldface definitions. Only three students used ‘with acc.’ to discard the first section of the lemma. Of the students who used the signpost ‘with adj.’, only three did so as a navigational strategy and six as a way to confirm their assumption.

Interestingly, many participants gratefully used the instruction to translate the verb as ‘continuously’, although this was not in boldface; three of them did not even look deeper into the second section and S11 reports that ‘it was kind of nice in the sentence: just translating it as ‘continuously’.

3.5.3 Qualitative analysis

In this translation task signposts are used even less navigationally than in the task with καθ-ίστημι. S31 rather clinically dissects the lemma to find their way to the relevant section, whereas S29 more or less stumbles upon the relevant meta-information once they found a promising lead.

Cognitive costs may offer one explanation: the asymmetry between ‘with acc’., ‘with ptc.’ and ‘with adj.’ may discourage students regarding them in relation to each other. The term ‘adjective’ also seems to be rather demanding. In fact, the two students who successfully employed the signpost both struggle to formulate what the abbreviation stands for. If the abbreviation had not occurred in the glosses, would they have been able to use it? Another reason for ignoring the signposts is that there is not a lot to gain this time, since there are only few boldface definitions to start with.

Finally, we discuss the only instance in the experiment of a student using an example to make an analogy instead of merely looking for a literal match. S33 explains that they consulted the example for ‘to stay willing’ (bereidwillig blijven) because they found the phrase problematic, realising this was the only time they decided to ‘keep reading’. This is again an instance of the strategy to shift the attention to meta-information when one is stuck (‘I thought it was an annoying bit’). They then consciously made an effort to look for an illustration of the usage of the example: ‘how do they think that we should use this?’ and then fabulously define the analogy-affordance: ‘if this is the way they use it, I can use it the same way with a different adjective’.

3.6 πρίν and knowing when not to use the dictionary

The lemma πρίν describes one of those words more easily memorised than looked up in a dictionary. It has two sections, the second of which has four subsections. The highest division is marked by the morphological signposts ‘adverb’ and ‘conjunction’ (voegw.); on the second level syntactic signposts are used. The second subsection of 2 (2.b. ‘with indic.’) is relevant for the translation task, which further defines the difference between the usage of ‘before’ (voordat) and ‘until’ (totdat). Two rather long examples are provided in this subsection, the second of which matches the translation task (Figure 10).

Figure 10. Translation task 6.

We wanted to know how the participants would deal with a fairly complex lemma of a straightforward word. The question is, first, whether they consult the lemma at all, and secondly, if so, whether they would use the multi-levelled signposts to navigate to the right usage.

3.6.1 Translation task

3.6.2 Results

13 students correctly translated πρίν with ‘before’ (voordat) and one chose the less appropriate ‘until’ (totdat); two students used both the conjunction (voegw.) signpost and ‘with indicative (met indic.); four used only the latter; three did not use the signposts and only tried out the boldface definitions. The remaining four reported that they already knew the meaning of the word; three of them only briefly checked the lemma for confirmation of their hypothesis and one did not look at the lemma at all⁶. A total of eight students noticed that the sentence was included in the lemma.

3.6.3 Qualitative analysis

Πρίν is or should be familiar to students and this is reflected in their lookup behaviour: a considerable number only briefly checked the lemma to see whether the translation ‘before’ is indeed listed. One student did not look at the lemma at all, although it is so prominent on the screen.

Reflecting on the use of the signposts, we see that ‘with indicative’, which is a signpost on the second level, is preferred to those on the first level (indicating part of speech). This is understandable, because the form of πρίν simply does not reveal whether it is an adverb or conjunction, whereas the verbal mode can be determined by its form. Students who struggled with the first-level signposts thus found an opening into the lemma which worked for them.

Below, S29 explains their difficulties with the signpost ‘conjunction’. They appear to say that they do know, inductively, by trying out the definitions in the sentence, that a conjunction is needed, and not an adverb. This does not mean, however, that they could use ‘conjunction’ in a deductive manner. In terms of the cost-benefit analysis, ‘conjunction’ seems too abstract, with too high cognitive costs, while the reduction in boldface translations is relatively small: there are only three general definitions to consider anyway.

4. Conclusion

At the outset of this article we argued that using meta-information stimulates the engagement in a feedback loop between text and dictionary, which can help students to avoid dictionary mistakes and awkward translations. The use of meta-information is, however, not easy, especially not for learners. Translating Ancient Greek is a cognitively demanding task and successfully using a dictionary requires careful cognitive load management.

What do we learn from our expert learners’ navigational behaviour? First and foremost, that their default strategy is to focus on boldface definitions. We need to recognise that this in itself is a way to limit cognitive load: the bold typeface has the affordance of stepping-stones, ignoring other information in the lemma. By critically testing the translations in context, most participants avoided mistakes due to semantic tunnel vision.

We did, however, find diversions from this default strategy. These can be understood as the outcome of an implicit cognitive cost-benefit analysis: the investment of employing meta-information is weighed against the reduction of boldface translation options that can be gained. Consequently, we understand that signposts were the most popular type of meta-information, because they provide a condition for use and thus can discard whole sections of the lemma. The participants attend to this select/exclude-affordance, but only when these signposts are not too costly to apply. They showed a preference for (morphological) meta-information which is directly relatable to formal features of the text. Even these excellent students experienced difficulties with more abstract terms like ‘intransitive’, ‘conjunction’ and even ‘adjective’.

Labels and examples, which are on a lower hierarchical level and do not have the same ‘shortcut benefits’ as signposts, received less attention when participants were navigating to the relevant section. We did observe, however, that some students employed these types of meta-information, as well as the more costly signposts, in a later stage of their decision-making process. These participants first focused on a promising boldface definition and then included meta-information in their scope as a way to confirm their hypothesis. Or, alternatively, they decided to shift their focus to meta-information because they were stuck. These are also examples of effectively dealing with cognitive pressure and show that expert learners have the flexibility to use more than one route to the desired outcome.

Examples are seen by our participants as a possibility to hit the jackpot: they could be a literal match with the text and provide 100% certainty (and often even a free translation). The affordance of drawing an analogy from an example was only realised once. This could indicate that the analogy-affordance belongs to a different level of expertise.

4.1 Methodological reflections

The main limitation of our study design pertains to the laboratory setting of the experiment, which does not resemble the normal classroom situation in which students perform translation tasks. In other words, the ecological validity was low. Furthermore, the design of the translation task required inclusion of many glosses: in most of the tasks, the only word left to be solved was the word that was to be looked up. This facilitated completing the task. We also observed a learning effect: in some earlier tasks, parts of the sentence matched a dictionary example. Some students reported this made them look at the examples more actively in the later tasks.

However, combining eye-tracking with a stimulated recall proved to be a fruitful method to record the unconscious dictionary behaviour of students and their conscious reflections on it. This combination yielded a rich data set which we could not have achieved in a classroom setting. Without the reports of the students, we could not have interpreted the eye movements, and, in turn, students would not have been able to report as informatively on their behaviour without the stimulus of their eye movements. This was a successful method to triangulate the data.

4.2 Pedagogical implications

We think that it is paramount that secondary school students are introduced into the rich landscape of affordances a lemma has to offer them. The most important pedagogical recommendation, therefore, is to raise ‘lemma awareness’ under students: explicate different ways in which lemmata are organised, discuss the various types of meta-information, and let the students experience the benefits that can be gained by attending to available affordances. Moreover, the participants of this study show that a lemma offers more than one route to a satisfactory outcome. Teachers can model these various approaches for their students. Furthermore, we need to invest in familiarising students with the more abstract terms that are used in a lemma (e.g., ‘figurative’, ‘intransitive’ or ‘conjunction’) to decrease the cognitive costs of using them. By tweaking the outcome of their internal cost-benefit analysis, we can prompt students to avail themselves of the benefits meta-information has to offer.

4.3 Lexicographical implications

To facilitate cognitive load management, lexicographers should be aware of the impact of typography on the reading eye of a student. The fact that one participant stopped reading prematurely when a definition was interrupted by the word ‘or’ in italics is telling. Furthermore, meta-information is now often rendered in the same, easily-overlooked typography, although affordances between the various types can differ greatly. Electronic dictionaries can offer the possibility to toggle between two versions of a lemma, e.g., with or without examples, or to open/close a subsection. In choosing signposts and labels, it can be advisable to use terms that feature characteristics visible in the text, instead of requiring a level of abstraction (e.g., ‘with accusative’ instead of ‘transitive’).