Translating culture through AI: Evaluating the performance of ChatGPT and Gemini in rendering Arabic and English proverbs

Abstract

The use of artificial intelligence in various areas is a growing phenomenon. Translation is a field that has long been associated with machine assistance and automation. This study examines AI translation tools' abilities to adequately transfer meaning culturally and functionally beyond word replacement. For this purpose, a sample of 50 proverbs was selected due to their cultural non-literal sense. The prominent AI tools, namely Gemini and ChatGPT, were selected as translation programs due to their growing use, especially in translation. The analysis worked with Arabic and English, each serving as both source and target languages. The results showed that the translation program and translation direction influenced the outcomes. Additionally, the quantitative analysis showed that among literal translation, paraphrasing, and cultural substitution, GPT was most likely to employ the first strategy while Gemini showed a tendency for the last. Furthermore, GPT showed a more balanced distribution of strategies even across languages. Meanwhile, Gemini’s strategy use varied especially when comparing the use of paraphrase in each direction. The qualitative analysis indicated that Gemini outperformed ChatGPT. The analysis also showed that cultural substitution has the highest potential in recreating the function of the ST, while literal translation was most likely to cause translation loss. Moreover, the programs, especially ChatGPT, could still commit translation errors that modified sense even when employing strategies like paraphrase and cultural substitution as they lack cultural nuance and critical thinking skills. The study recommends the development of parallel corpora that contain culturally equivalent idioms based on usage, so people do not rely on programs that may fall short in transferring meaning.