A New Frontier: AI and Ancient Language Pedagogy

Abstract

In November 2022, ChatGPT 3.5 was released on a public research preview, gaining notoriety for its ability to pull from a vast body of information to create coherent and digestible bodies of text that accurately respond to queries (OpenAI, 2022). It is able to recognise the grammar and vocabulary of ancient languages, translate passages, and compose texts at an alarmingly accurate and rapid rate. For teachers, this AI has had mixed reviews. Some fear its ability to produce well-written work effortlessly, while others are excited by its abilities to push the boundaries of current teaching practices. This paper explores how well ChatGPT explains grammatical concepts, parses inflected forms, and translates Classical Latin, Ancient Greek, and Classical Sanskrit. Overall, ChatGPT is rather good at working with Classical Latin and Sanskrit, but its abilities with Ancient Greek are deeply problematic. Although it is quite flawed at this time, ChatGPT, when used properly, could become a useful a tool for ancient language study. With proper guiding phrases, students could use this AI to practise vocabulary, check their translations, and rephrase grammatical concepts.

Introduction

Over the past year, Artificial Intelligence (AI) models, such as Dall-E and ChatGPT, have captured the public eye for their ability to accurately produce images and bodies of text according to recognised patterns at an alarmingly rapid rate.¹ Since the 3.5 model of ChatGPT was released on a public research preview in November 2022, it has gained notoriety for its ability to pull from a vast body of information to create coherent and digestible bodies of text that accurately respond to queries (OpenAI, 2022). Responses in the humanities community are fairly mixed. To some, ChatGPT instils fear because the AI is able to mimic entry-level prose at a high level quickly and effectively (Mathias, 2023). To others, this is an amazing technological advancement that will make work easier in a variety of fields (Jarus, 2023). In Classics, there is a combination of these views. There is a similar fear that students could try to pass off AI-generated work as their own. However, it is also a significant development for the machine translation of, and composition in, ancient languages (Gellar-Goad, 2022). As Classics teachers in the United Kingdom, our teaching should aim to follow the UK Professional Standards Framework for Teaching and Supporting Learning in Higher Education (UKPSF) (Advance HE, 2023). Following Core Knowledge dimensions 4² and 6³ and Professional Values 3⁴ and 4⁵, we as teachers need to continue to learn and develop with the arising of new technologies so that we can provide the best teaching practice to ancient language students in this evolving, technical world. Although it is quite flawed at this time, ChatGPT, when used properly, could become as useful a tool for ancient language study as the Perseus Word Study Tool and Whitaker's Words.

This paper will discuss the possible uses that ChatGPT will have for students studying ancient languages at the GCSE or University level. In particular, I will explore how ChatGPT handles explaining grammatical concepts, parsing the grammar of all forms of words in Classical Latin, Ancient Greek, and Classical Sanskrit, translating from these three ancient languages into English, and translating from English into the same three ancient languages. To match the level of Ancient Greek, Latin, and Sanskrit that students are expected to encounter, I will draw examples from the OCR Greek to GCSE (Taylor, 2016a, 2016b) and Latin to GCSE (Cullen and Taylor, 2016a, 2016b) books and the Goldmans' Introduction to Sanskrit Language (2004). In order to test the accuracy of ChatGPT, I will compare its answers to the relevant parsing and translation software, including Whitaker's Words, the Perseus Word Study Tool, SanskritDictionary.com, and Google Translate.

This essay is divided into three major parts: the first discusses how ChatGPT works. The second part will then discuss the results of my investigations into how effectively ChatGPT describes grammar and works with Classical Latin, Ancient Greek, and Classical Sanskrit. Each language section will first discuss the popular programs and applications available to aid parsing and translation for the relevant language. I will then discuss how effectively it can work as a parsing device, a machine translator into English, and a machine translator from English into the ancient language. The third part discusses the implications of ChatGPT for ancient language pedagogy, some possible uses that it could have for entry-level ancient language students in its current form, and some ways that ancient language teachers can introduce ChatGPT to their students. Throughout, however, I will discuss the benefits and pitfalls a language student could meet using ChatGPT. Before analysing the grammar and translation, I will discuss how ChatGPT works in broad strokes.

ChatGPT 3.5: A Conversational AI Model

ChatGPT is an Artificial Intelligence (AI) model created by OpenAI that responds to requests and interacts with a user in a conversational way (OpenAI, 2022). OpenAI advertises that this model has the capacity to copywrite, summarise, parse, classify, and translate texts (OpenAI, n.d.-a). It was built on GPT-3, which was released in 2020, so although Generative Pre-trained Transformers (GPTs) are not new, ChatGPT is the first model released in such a public-friendly format (OpenAI, 2021). The major attraction of the 3.5 model is that it responds following human speech patterns and draws from a vast corpus for each response. It can answer questions, quiz you, have a chat, and compose bodies of work according to given parameters.

For example, if you asked ChatGPT to write a short essay discussing how AI can improve ancient language pedagogy, it writes an answer like this (Figure 1).

Figure 1. ChatGPT produced essay discussing how AI can improve ancient language pedagogy.

The text produced is clear and coherent, but it tends to answer queries with longer answers rather than shorter answers for the sake of coherence (OpenAI, 2022). Despite its length, it discusses the topic in superficial detail and does not list sources. If ChatGPT is asked to provide the sources used to write the essay or list sources that discuss the same topic, it produces this (Figure 2).

Figure 2. ChatGPT responses to requests for sources and citations.

This is quite representative of the limitations of the current ChatGPT model. OpenAI notes, and ChatGPT also reiterates when prompted, that the AI model was trained using a blend of filtered Common Crawl data, WebText, ebooks, and Wikipedia articles from before Q4 2021, so it is unable to answer questions or access information during or after that time (OpenAI, n.d.-b). Despite OpenAI's transparency in indicating what percentages of certain websites were used as training data for ChatGPT, they only list the major websites used to collect data, not all websites used (Brown et al., 2020, p. 9). This leaves much of the language training data ambiguous, so I cannot determine what specific language sources were used to train the model, at this time. However, it is highly likely that the most easily-searchable websites and sources may have been used to train ChatGPT's Classical Latin, Ancient Greek, and Classical Sanskrit corpora.

Although it does not occur in the essay example, one major limitation that OpenAI is working to correct is ChatGPT's tendency to assume the missing details of an input and generate an answer that sounds correct, even if it is based on non-sense information (OpenAI, 2022). These answers may be quite verbose to make them appear more comprehensive, despite being nonsensical. Furthermore, ChatGPT tends to react very differently to minor changes in input. Broad questions tend to output generalised answers, while very specific requests output specific answers. That being said, if certain words are used, it may provide a disclaimer or no answer at all. This is usually connected to OpenAI's ‘efforts to make the model refuse inappropriate requests’ which target certain words as inappropriate or ambiguous (OpenAI, 2022). I will explore this issue more when discussing ChatGPT's machine translation abilities for each language. Before I delve into the specifics of each ancient language, let's first take a look at how well ChatGPT can describe and explain difficult grammatical concepts.

Describing Grammatical Concepts

Just like how it responded to the request to write an essay, ChatGPT draws from its extensive training data to answer a variety of grammatical questions. If a student was confused about a particular grammatical form and wanted to ask follow-up questions, would they be able to use ChatGPT as a resource?

Fascinatingly, when asked to explain the Ablative Absolute and provide examples in Classical Latin, ChatGPT outputs a coherent description with consistent examples to support its work (Figure 3).

Figure 3. ChatGPT explains the ablative absolute with a Classical Latin example.

The discussion is quite good, but it does use some grammatical jargon that may be confusing to a student with little experience. However, they can ask ChatGPT to expand on the discussion or explain certain words that it used, and it will provide more coherent and accurate descriptions. The Latin example is also quite accurate, using a simple, grammatically-sound sentence with helpful commentary. This will be an excellent way for students to supplement their ancient language learning while they are practising their homework. It even can work through difficult grammatical forms in Ancient Greek and Classical Sanskrit.

When I prompted ChatGPT to describe an indirect statement and use an Ancient Greek example, it was able to describe the grammatical form quite well after several attempts. This is because ChatGPT will start from zero each time a new conversation starts, no matter the repetition, but it can be trained within a conversation with extremely specific requests regarding format and source bases (Figure 4).

Figure 4. ChatGPT explains the indirect statement with an Ancient Greek example.

Unfortunately, because the outputs are inconsistent, ChatGPT appears to be less reliable with Ancient Greek grammar examples than with Latin. Firstly, ChatGPT will not write Greek words with accents unless prompted, but it will do so from that point onwards once it is requested. Furthermore, the ‘Ancient Greek’ examples it outputs actually use Modern Greek words and forms rather than Ancient Greek ones. For example, instead of using an infinitive verb to indicate ‘to rule’, it uses the Modern Greek preposition να and the indicative verb βασιλευσει. This appears to be an issue with ChatGPT's recognition of words and lexical corpora, but I will discuss this problem further when looking at ChatGPT's machine translation abilities. Overall, a student should be apprehensive when asking for Ancient Greek grammatical commentary.

With Classical Sanskrit grammar concepts, ChatGPT is able to describe the gerundive quite accurately, but it is unable to produce a grammatically or semantically-sound Classical Sanskrit sentence as an example (Figure 5).

Figure 5. ChatGPT explains the gerundive with a Classical Sanskrit example.

This issue persists after several different test phrases. In the example above, ChatGPT provides the present active verb paṭhati instead of the expected future passive participle paṭhitavyaḥ and still translates it like a gerundive. Furthermore, semantically, the sentence is odd; although you can read a house, it is not a commonly expected object of the verb ‘to read’. So, like with Ancient Greek, ChatGPT can be used to describe how certain types of grammar work in Classical Sanskrit, but the examples will largely not match the explained form. I will discuss how deep this Sanskrit translation issue goes when reviewing its machine translation abilities.

After reviewing these tests, it is quite clear that ChatGPT is able to isolate grammatical concepts and describe them in a digestible format, but the main issues lie in the examples it claims come from the ancient languages. Although it generally outputs grammatically sound Latin phrases, ChatGPT is unable to reproduce grammatical forms in Ancient Greek or Classical Sanskrit properly. It is unclear why this occurs, but I will explore some possibilities in the next three sections. Despite this, the AI at this stage could be used by students to re-explain grammatical concepts, but any examples should be ignored until ChatGPT's abilities are refined. Now, let's investigate how ChatGPT responds to specific queries about the translation and forms of Classical Latin.

Classical Latin Translation and Composition

At this time, the most popular parsing program available for Latin is Whitaker's Words. Whitaker's Words is a computer program developed by Col. William Whitaker (1936–2010) that parses inflected forms of Latin words and provides the gloss translations from the Oxford Latin Dictionary and Lewis and Short lexicon (Whitaker, n.d.).⁶ It is an excellent tool for identifying all possible grammatical interpretations a Latin word may have out of context. For example, if you put the word laetis into the program, it will list all possible grammatical interpretations of the word and a gloss translation from the Lewis and Short Latin Dictionary (Figure 6).

Figure 6. Whitaker's Words output for laetis.

When asking ChatGPT to parse the same word, it responds vaguely and with some difficulty (Figure 7).

Figure 7. ChatGPT parsing laetis incorrectly.

Though it does parse the word incorrectly, it does provide some supporting information about how it could be used in a sentence. This appears to be an issue related to wording. If the request is changed to the more specific ‘Provide all grammatical information about the Latin word laetis and it is given some guiding phrases beforehand, it does output one correct option. When pressed further about other possible grammatical interpretations, it cannot fathom that laetis could be interpreted as the identically-spelled ablative plural form (Figure 8).

Figure 8. ChatGPT parsing laetis in one way.

The AI does not have the ability to discern information and critically think about the similarities between words like humans can. In essence, it draws from the wide corpus that it was trained with and makes connections, but it is unable to read between the lines and see similarly spelled words like Whitaker's Words can. ChatGPT does make a note that it is difficult to determine the grammatical form of the word without context, so let's turn to its machine translation abilities.

Although new neural network models are working to develop machine translations of Latin texts, there are few accessible applications that can translate Classical Latin to a good standard (Fischer et al., 2022). Google Translate (n.d.), however, has made great strides in the past decade.⁷ It is quite useful for discerning the general idea of a Latin sentence, but we are a long way away from a trustworthy Latin machine translator. For the sentence ancilla perterrita ‘hostes nunc adsunt, domine!’ subito clamavit (Cullen and Taylor, 2016a, p. 201),⁸ Google Translate has quite a bit of trouble recognising the clauses and relationships between the words (Figure 9).

Figure 9. Google Translate translates a Classical Latin sentence into English.

It can translate each of the words and build a sentence from them, but it has no recognition of the general context of a sentence. ChatGPT, on the other hand, outputs a surprisingly accurate translation (Figure 10).

Figure 10. ChatGPT translates a Classical Latin sentence into English.

With very specific instructions, ChatGPT produces a sound translation of the Latin and describes the grammatical purposes of each word in the context of the sentence. Of course, more information could be provided for each word, but this output would be an acceptable translation for an entry-level Classical Latin student.

When asked to translate longer paragraphs, the outputs are largely the same. Google Translate struggles to build the sentences within their context while ChatGPT is able to translate the paragraphs in good grammatical form. I took this one step further and played the role of a lazy student who did not wish to type out the entire passage. I asked ChatGPT to translate a text simply by providing the name and line numbers, and it was able to find the relevant text, provide a decently accurate translation, and comment on the content of the text. Taking this a step further, I chose a particularly explicit passage, Catullus 16, in order to test the limits of ChatGPT's content policy (Figure 11).

Figure 11. ChatGPT translates “Catullus 16.”

Surprisingly, after several attempts with different wordings, the AI would not provide a translation of any portion of Carmen 16; instead, it produces translations of other Catullus poems of varying lengths and claims it is Catullus 16 with no notice or disclaimer. This is quite problematic for students who do not know the full context of the work they are translating. If they took ChatGPT's output at face value, the student would develop significant misconceptions about the text. When the Latin for Catullus 16 is inputted into ChatGPT, it provides a full translation, but the output is flagged as violating the content policy. It is unclear if this is the reason why ChatGPT will not locate the proper passage, but it appears to be a likely possibility. Though other models, like DAN ChatGPT, are working to produce models that work around these restrictions, they are not yet widely available (King, 2023). This cements the fact that any translation provided by ChatGPT, at this point, must be taken with apprehension and further researched.

Although ChatGPT's ability to translate given passages from Latin to English is quite good, it appears to have less skill when translating from English into Latin. For the sentence ‘The woman does not trust her own brother’ (Cullen and Taylor, 2016a, p. 167),⁹ Google Translate and ChatGPT both produce quite sound sentences but seemingly draw from different lexicons (Figures 12 and 13).

Figure 12. Google Translate translates an English sentence into Classical Latin.

Figure 13. ChatGPT translates an English sentence into Classical Latin.

It appears that Google Translate draws from an internal lexicon that associates words with certain terms, while ChatGPT crunches all the information it has trained with and uses the first possible option that matches the parameter and does not continue searching for a ‘closer’ translation. So, it is crucial to be sceptical of any ChatGPT's Latin outputs.

When looking at longer passages of English to Latin translation in ChatGPT, the AI follows the same paradigms as with shorter sentences, but it also offers a disclaimer that the translation is based on certain dialects and may appear differently in another location. If I prod further and ask ChatGPT to compose a work of prose or poetry, it can quickly create a work that is generally grammatically sound and broadly follows expected writing paradigms. It can even write novel outputs of creative works based on the style of other authors (Figure 14).

Figure 14. ChatGPT composes a Classical Latin poem in the style of Catullus.

This particular composition was meant to follow the style of Catullus. The themes of the output poem are similar to Catullus' love poetry, although very vague. There are some grammatical errors, such as the use of cresco instead of crescit, and it does not appear to follow Catullus' common meters. However, these are all issues that can eventually be corrected once the model is trained with a more comprehensive training data set. The fact that ChatGPT can quickly produce novel creative works is astounding and could be used by teachers and students to produce practice exercises.

For students of Classical Latin, ChatGPT appears to have the capacity to become an exceedingly useful study tool. Although its parsing abilities, compared to Whitaker's Words, are exceedingly poor, its ability to translate from Latin to English and English to Latin is quite superior to Google Translate. A student could input Latin or English sentences into ChatGPT to check their work immediately. Furthermore, ChatGPT's ability to compose Latin text is far beyond anything we have seen up to this point. Its outputs could be used as extra practice or to help inspire students with their Latin composition assignments, so long as proper credit is given. With all that said, all of the outputs from this AI are based on incomplete lexical training data, so all Latin outputs must be examined sceptically and in detail prior to its use. Unfortunately, it is a much different situation with Ancient Greek.

Ancient Greek Translation and Composition

The most popular Ancient Greek parsing program is currently the Perseus Word Study Tool (Perseus Digital Library Project, n.d.). This parsing application was developed by the Perseus Digital Library Project at Tufts University, and it is a searchable dictionary that provides almost all possible grammatical interpretations of the word form searched and links the search results to the equivalent Liddell-Scott-Jones (LSJ) and Lewis and Short dictionary entries. Its capabilities are limited to the Ancient Greek, Latin, Arabic, Old Norse in varying capacities. However, it is an excellent aid for translating Ancient Greek and Latin texts, especially since each Ancient Greek and Roman text available in the digital collection is encoded with links to the Word Study Tool search for each word.

Unfortunately, unlike Whitaker's Words, Perseus is not encoded to recognise all inflections of Ancient Greek words. Instead, it is based on words that appear in the texts in the digital collection. So, some forms, despite being grammatically correct, may not produce an output from Perseus, but it does usually produce the most common options. For example, if I inputted “ὦν” into the Word Study Tool, it outputs these forms (Figure 15).¹⁰

Figure 15. Perseus Word Study Tool parses an Ancient Greek participle.

When inputting the same form into ChatGPT, it has similar difficulty as previously seen with Latin (Figure 16).

Figure 16. ChatGPT parses an Ancient Greek participle.

It is unable to recognise the participle out of context and appears to class parsing questions as one word sentence translations, mistaking the Ancient Greek word as a finite verbal form. Even with some guiding phrases to train the AI, it cannot fully understand the word out of context. Unfortunately, ChatGPT's parsing ability cannot meet the applications of the Perseus Word Study Tool, and furthermore, it has some fundamental issues with its translation capabilities.

Unlike for Latin, Google Translate does not have an option to translate Ancient Greek. It only has the option to translate in Modern Greek, and any Ancient Greek input would output according to Modern Greek grammar rules. At this time, there is no effective Ancient Greek translation software available, so ChatGPT would be a welcome addition to Ancient Greek studies if it works. Unfortunately, there appears to be a foundational issue with its use of vocabulary.

After inputting a simple test phrase, οἱ πολέμιοι ὑπὸ τῶν συμμάχων ἐδιώκοντο (Taylor, 2016b, p. 20),¹¹ ChatGPT is able to output a fairly sound English translation with some problematic parsing of the Ancient Greek words (Figure 17).

Figure 17. ChatGPT translates an Ancient Greek sentence into English.

The wording of the grammatical information provided for each word is not exact and is sometimes incorrect. For example, it identifies articles as pronouns and misattributes the τῶν with the accusative case, yet it easily recognises the sentence as a passive construction, describing it in a roundabout way. Longer passages output very similar results. The passages are translated reasonably well, but the grammatical description of each word in context is inconsistent. Although this is problematic, students could use this function to check their Ancient Greek to English translation homework with significant apprehension. Despite this, the English to Ancient Greek function is essentially useless.

When prompting ChatGPT to translate a sentence from English to Ancient Greek, ‘The giant who eats men is not in the field now’ (Taylor, 2016b, p. 34),¹² it claims to output an Ancient Greek sentence (Figure 18).

Figure 18. ChatGPT translates an English sentence into “Ancient Greek.”

This is in fact an entirely Modern Greek sentence using Modern Greek grammar and vocabulary. Inputting the same sentence into Google Translate's Modern Greek function outputs a very similar sentence (Figure 19).

Figure 19. Google Translate translates an English sentence into Modern Greek.

Clearly there is a flawed connection in the training data that associates Modern Greek words and terms with Ancient Greek ones. This could likely be because there are no effective English to Ancient Greek translation tools available online or ChatGPT was not trained using a clearly-delineated Ancient Greek corpus. Unfortunately, because ChatGPT tends to claim incorrect outputs as fact without a disclaimer, as seen with Catullus 16 earlier, this AI should not be used for English to Ancient Greek translation whatsoever.

Requesting compositions in Ancient Greek further demonstrates the lexical issue with ChatGPT. Although it does have some access to Ancient Greek concepts and vocabulary, it tends to mix it together with Modern Greek words and forms. The results tend to be hybrid Ancient and Modern compositions that cannot be considered one or the other (Figure 20).

Figure 20. ChatGPT composes an “Ancient Greek” poem in the style of Homer.

This is a major issue with the software that makes it even less useful than the clunky translation programs available publicly. This can eventually be improved with new training data, but at this time, students should avoid the English to Ancient Greek functions of ChatGPT.

Compared to Latin, ChatGPT's effectiveness with Ancient Greek is exceedingly poor. This appears to be an issue caused by a partial lexicon in the training data and a tendency for ChatGPT to break unrecognised words into smaller parts and compare them to known lexica; in this case, Modern Greek. These inherent issues with the grammatical data are too significant to ignore. At best, an Ancient Greek language student could check their Ancient Greek to English translations with very specific guiding phrases inputted in advance, but generally Ancient Greek language students should avoid using ChatGPT at this stage. Despite this, ChatGPT tends to work quite a bit better with Classical Sanskrit.

Classical Sanskrit Translation and Composition

There are a variety of Classical Sanskrit search functions available online today, but my preferred application is SanskritDictionary.com (Sanskrit Research Institute, n.d.-b). This multi-layered dictionary, parsing, and vocabulary tool was created by the Sanskrit Research Institute in Auroville in 2011 (Sanskrit Research Institute, n.d.-a). It not only has an effective parsing search, but it also offers root search and sandhi calculator tools. For each word search, in Devanagari or the International Alphabet of Sanskrit Transliteration (IAST), it trawls the 20 major Sanskrit dictionaries for relevant forms. It is usually able to identify the most relevant forms and provides a link to the dictionary entries from each relevant dictionary dataset.¹³ This is exceedingly helpful for parsing work at the introductory level.

Just like with the Perseus Word Study Tool, SanskritDictionary.com takes an input of an inflected form and outputs the potential interpretations (Figure 21).

Figure 21. SanskritDictionary.com output for pujābhiḥ.

For this test, I used the Instrumental-Dative word pujābhiḥ, and the dictionary does output one of those relevant options. Since the grammar study aspect of the application is still in beta, it does occasionally miss possible outputs. Interestingly, ChatGPT is able to provide more information than the parsing software in very specific instances.

When using test words with ChatGPT, I wanted to see how well it could handle inflected words when they were unaffected by sandhi and samasa forms. It is able to identify Classical Sanskrit words equally when written in Devanagari or Romanised script, but, oddly, ChatGPT appears to always first consider a word as a possible compound before considering it as a stand-alone root. For example, when I request a grammatical description of pujābhiḥ, it outputs a somewhat complex description of a compound interpretation (Figure 22).

Figure 22. ChatGPT parses pujābhiḥ.

Although this does present a roundabout path to one correct answer, it does not provide all possibilities. This compound interpretation is consistent with almost any inflected word entered into ChatGPT, but if you train the AI before this by asking it to discuss the root form, in this case pujā, before asking the same parsing question, it provides a much clearer answer (Figure 23).

Figure 23. ChatGPT parses pujābhiḥ with guiding phrases.

In this output, it actually indicates both possible answers, which is a step above SanskritDictionary.com and the parsing abilities it presented with Latin and Ancient Greek. This appears to be a consistent output pattern. If it is trained with proper guiding phrases before asking questions, ChatGPT is a step above any parsing tools currently available for Classical Sanskrit study, but it only works if the person using it already knows the base root of the word they are parsing, which is counterproductive for students that need an identification aid. Until the AI can be properly trained without this information, it would be best for Classical Sanskrit students to stick to SanskritDictionary.com for parsing. Unfortunately, ChatGPT's capacity for translation is similarly strained.

Sanskrit, like Latin, is currently an option on Google Translate. It is unfortunately still quite baseline, but it is able to grasp the ideas and vocabulary of most words. For the sentence kūjantaṃ rāma rāmeti madhuraṃ madhurākṣaram (Goldman and Goldman, 2004, p. 268),¹⁴ it outputs a near translation (Figure 24).

Figure 24. Google Translate translates a Classical Sanskrit sentence into English.

There are some minor mistakes with the participle and the adverb here, but overall it catches the form of the sentence. This might be an issue with sandhi interpretation.

ChatGPT, on the other hand, has a bit more difficulty translating the sentence (Figure 25).

Figure 25. ChatGPT translates a Classical Sanskrit sentence into English.

It catches some major aspects of the sentence in its translation, but it misconstrues others. Furthermore, the parsing does not match the translation whatsoever. It is particularly odd that the AI can identify sandhi divisions in its translation, yet it is unable to describe it grammatically. For example, it identifies the sandhi between rāma and iti as a verb in its parsing yet properly identifies the direct speech in its translation. There are also several issues with the grammatical terms used to identify each word. For students without much grounding in Classical Sanskrit, this could cause great confusion, so ChatGPT should be avoided for Classical Sanskrit to English translation at this time. English to Classical Sanskrit translation does not seem to do much better.

When inputting the sentence ‘The demon with the face of a monkey is unable to kill the king’ (Goldman and Goldman, 2004, p. 236),¹⁵ into Google Translate, the output phrase is missing some semantics behind the words, but it presents a generally sound sentence (Figure 26).

Figure 26. Google Translate translates an English sentence into Classical Sanskrit.

The word vaktraḥ here is meant to stand in for face, but the word literally means the ‘speech organ area of the face’ (Monier-Williams, 2002, p. 912). Overall, however, it does present a decent translation with proper sandhi and logical word choices. ChatGPT goes in a different direction with its translation (Figure 27).

Figure 27. ChatGPT translates an English sentence into Classical Sanskrit.

The word order here is unusual, but since the case endings are generally okay, this can be glossed over. However, ChatGPT's translation uses a much more philosophical formation of the meaning rather than literal, using viśiṣṭaḥ to indicate that the ‘monkeyness’ was a distinguishing feature. This is quite a loose translation, and introductory Classical Sanskrit students need to work as closely to the grammar as possible to reinforce their understanding.

This issue appears to persist with longer passages, but ChatGPT is able to, consistently, spontaneously compose short works in Classical Sanskrit (Figure 28).

Figure 28. ChatGPT composes a short Classical Sanskrit verse.

Just like with Latin, it can produce a variety of forms that are reasonably grammatically sound and generally follow traditional sandhi paradigms. This could be a helpful tool for developing practice questions and inspiring prose compositions, but like all the other uses of ChatGPT discussed so far, the outputs should be scrutinised before they are used.

Overall, ChatGPT's Classical Sanskrit outputs are generally sound. Despite the general vocabulary choices and misidentifications of grammatical terms, with proper guiding phrases, the AI can capture the general meanings of Classical Sanskrit in its translations. It is also quite effective at composition and translating from English into Classical Sanskrit. There is great promise in ChatGPT's uses for Classical Sanskrit teaching and learning as it develops.

ChatGPT for Students

Looking at the results of ChatGPT's translation tests, the current model could be quite useful for ancient language students if it is used properly. Though its parsing abilities are not as good as the current parsing applications in each language, it can be used to check translation homework and explain grammatical concepts. Despite students needing to be exceedingly cautious when using this AI, it could also have one other purpose.

Because ChatGPT is a conversational model, it does respond to questions quickly and coherently. As mentioned before, ChatGPT does have the capacity to play games and compose bodies of work based on patterns in its training data and the queries posed by a user. If you ask ChatGPT to compose a short vocabulary test following specific parameters, it does actually produce a simple test (Figure 29).

Figure 29. ChatGPT runs a Classical Latin vocabulary test.

There are still some problems with naming conventions in the training data, as seen with the mix-up between first and fourth declension Latin nouns, but the actual Latin forms requested are accurate, although simplistic. After several tests in all three languages with intentional errors, it is clear that ChatGPT can identify if input answers do not match the requested answers. Furthermore, the AI appears quite supportive of the user and nestles the answer key in encouraging phrases. The AI can even compose a series of sentences according to given parameters, giving students more practice phrases when they run out in their textbooks (Figure 30).

Figure 30. ChatGPT runs a Classical Sanskrit sentence translation test.

This could also be useful for teachers if they need to quickly produce some new sentences for class.

Unfortunately, the internal language errors seen above with machine translation do appear here as well. The Classical Latin quizzes do have some terminological errors that can be confusing, and the Classical Sanskrit quizzes tend to use more arbitrary and nonsensical vocabulary, but the overall outputs are consistent and are accurately marked. The Ancient Greek quizzes, on the other hand, have the same foundational vocabulary issue as before. The responses are made with a mix of Ancient Greek and Modern Greek forms, and it misreads responses. So, the vocabulary quiz function can be quite useful for students and teachers of Classical Latin and Classical Sanskrit, but students and teachers of Ancient Greek should avoid it completely.

ChatGPT for Ancient Language Teachers

After seeing the vast number of pitfalls a person can encounter when using ChatGPT to work with Classical Latin, Ancient Greek, and Classical Sanskrit, it is hopefully clear how crucial it is for teachers to scaffold their students' use of ChatGPT and warn them about the problems they could encounter. Bearing in mind that ChatGPT has foundational issues with Ancient Greek, these suggestions should only be used by Classical Latin and Sanskrit language teachers. Ancient Greek language teachers should at least warn students about ChatGPT and inform them that it would just give them wrong answers at this stage.

If you intend on introducing ChatGPT to your students, do a demonstration in class to show them the general uses that AI can have for aiding ancient language study. This includes its conversational vocabulary quizzes, its somewhat decent translation ability, and its ability to spontaneously compose formulaic prose and poetry in Classical Latin and Sanskrit. You could even lead your students to correct the grammar of ChatGPT's outputs as a group activity. Vitally, these activities and suggestions need to be contextualised with ChatGPT's major pitfalls seen earlier, especially the fact that ChatGPT cannot parse or translate properly before it is fed specific guiding phrases to model its outputs (Figure 31).

Figure 31. An example Classical Latin guiding phrase for ChatGPT.

ChatGPT does have the capacity to restrict itself to parameters and produce outputs according to give parameters, so one useful technique for teachers would be to develop a training document with set blocks of guiding phrases to share with your students. That way, students could easily copy and paste guiding phrases and forms into ChatGPT before using the AI to practise vocabulary or to check their translations, making it less likely for errors to occur.

Conclusions

The arising of new and powerful AI technologies is quite scary, but it is going to become a part of our everyday lives. If we see the amazing abilities of these programs, so do our students. They will teach themselves how to use them to their benefit, but this may lead to significant misconceptions in their learning. Instead of ignoring the rising tide, we should teach ourselves how to effectively work with these AI models so that we can help our students learn how to properly use these powerful tools to streamline their learning.

Although ChatGPT 3.5 is quite flawed, it is an amazing advancement in machine learning tools with applications in the machine translation of ancient languages. It is able to recognise the grammar and vocabulary of ancient languages, translate passages, and compose texts at an alarmingly accurate and rapid rate. ChatGPT is quite good at translating and composing works in Classical Latin and Classical Sanskrit; however, it does make quite a few grammatical and semantic errors. For Ancient Greek, on the other hand, it is essentially useless due to foundational issues in its language corpora. If ancient language students and teachers take these problems into account, ChatGPT can be used as an excellent support tool for grammar and vocabulary study, translation checking, and composition inspiration.

The flaws seen in AI models now will slowly be corrected, and eventually they will be able to do all the things mentioned in this paper effectively and efficiently. In fact, the newest model from OpenAI, ChatGPT 4, was released to paid subscribers in March 2023, and this model has even broader general knowledge and problem-solving abilities (OpenAI, 2023). It can now work in 26 languages, accept visual inputs, and receive more complex guiding information to define its behaviour. Although this new model is not currently widely available, it is significantly more effective than any previous model, and this trend will continue as AI research develops.

We as Classics teachers need to recognise the applications and advancement of AI technology as it grows, so we can grow along with it (Watson and Eaton, 2023). This way, we can provide the best possible learning experiences for students by expertly making use of cutting edge technologies in our teaching programs. Furthermore, we can show students how to effectively use these programs as tools for their own future language study. If we ignore AI developments, we may be swept away like obsolete technologies as they surpass us.