Artificial Intelligence Applications in Machine Translation and Their Role in Bridging Semantic Gaps Across Languages: A Comparative Analytical Study of Chat GPT and Deep Seek
DOI:
https://doi.org/10.25130/jfa.conf.10.1.24Keywords:
Artificial Intelligence, Machine Translation, ChatGPT, DeepSeek, Semantic Fidelity, Arabic-English Translation, BLEU, TERAbstract
With the fast-growing of neural machine translation (NMT), there is still a lack of insight into the performance of these models on semantically and culturally rich texts, especially between linguistically distant languages like Arabic and English. In this paper, we investigate the performance of two state-of-the-art AI translation systems (ChatGPT, DeepSeek) when translating Arabic texts to English in three different genres: journalistic, literary, and technical. The study utilizes a mixed-method evaluation methodology based on a balanced corpus of 60 Arabic source texts from the three genres. Objective measures, including BLEU and TER, and subjective evaluations from human translators were employed to determine the semantic, contextual and cultural quality. Our results show that our model, ChatGPT, consistently achieves performance gains over DeepSeek, especially when applied to technical and journalistic text and with higher BLEU scores and lower TER values. But neither these models nor any of the state-of-the-art models perform well for the literary texts, the ones that can hint to the difficulties these models face to deal with idiomatic expressions, metaphor, narrative tone. The results illustrate genre sensitivity in AI translation quality and emphasize the ongoing importance of human supervision, particularly in cultural and stylistic contexts. This work aims to contribute to the growing corpus of AI translation literature by providing a genre-specific, empirically grounded comparison of two of the most high-profile models, and to draw attention to the necessity of greater context-sensitive and culturally sensitive translation algorithms.
Downloads
References
1. Bahar, P., Bisazza, A., & Monz, C. (2019). A comparative study on artificial intelligence-based machine translation. Computational Linguistics Journal, 45(3), 451–475. https://doi.org/10.1162/coli_a_00356
2. Bojar, O., et al. (2018). Findings of the WMT 2018 shared task on machine translation. Proceedings of the Third Conference on Machine Translation.
https://aclanthology.org/W18-6401/
3. Brown, T., Mann, B., Ryder, N., et al. (2020). Language models are few-shot learners. NeurIPS, 33, 1877–1901. https://doi.org/10.48550/arXiv.2005.14165
4. Castilho, S., Gaspari, F., Moorkens, J., & Way, A. (2017). Is neural machine translation the new state of the art? The Prague Bulletin of Mathematical Linguistics, 108(1), 109–120.
https://doi.org/10.1515/pralin-2017-0013
5. Dabre, R., Kunchukuttan, A., & Bhattacharyya, P. (2017). A survey of multilingual neural machine translation. ACM Transactions on Asian and Low-Resource Language Information Processing, 17(3), 1–21. https://doi.org/10.1145/3158661
6. Denkowski, M., & Lavie, A. (2014). Meteor universal: Language specific translation evaluation for any target language. Proceedings of the EACL Workshop.
https://www.cs.cmu.edu/~alavie/METEOR/
7. Freitag, M., Al-Onaizan, Y., & Sankaran, B. (2020). BLEU might be guilty of underestimating NMT: Revisiting metric sensitivity. ACL, 7740–7745.
https://www.aclweb.org/anthology/2020.acl-main.692/
8. Hassan, H., Aue, A., Chen, C., et al. (2018). Achieving human parity on automatic Chinese to English news translation. arXiv preprint. https://arxiv.org/abs/1803.05567
9. Koehn, P. (2020). Neural Machine Translation. Cambridge University Press.
https://doi.org/10.1017/978110869324
10. Lo, C. K. (2020). A critical review of automatic machine translation evaluation metrics. Machine Translation, 34(3), 167–210.
11. Mohammed, M. A., & Al-Azzawi, I. G. (2025). The Impact of Culture in Translating English Idioms into Arabic. Arab World English Journal for Translation & Literary Studies 9 (1):103-118. http://dx.doi.org/10.24093/awejtls/vol9no1.7
12. Popel, M., & Bojar, O. (2018). Training tips for the Transformer model. The Prague Bulletin of Mathematical Linguistics, 110(1), 43–70. https://doi.org/10.2478/pralin-2018-0002
13. Pustejovsky, J., & Stubbs, A. (2012). Natural Language Annotation for Machine Learning. O'Reilly Media.
https://learning.oreilly.com/library/view/natural-language-annotation/9781449306663/
14. Ribeiro, M. T., Goharian, N., & Singh, R. (2022). Beyond BLEU: A toolkit for evaluating machine translation from a user-centric perspective. Proceedings of the NAACL-HLT.
https://aclanthology.org/2022.naacl-main.124/
15. Sennrich, R., Haddow, B., & Birch, A. (2016). Improving neural machine translation models with monolingual data. ACL, 86–96. https://doi.org/10.18653/v1/p16-1009
16. Tiedemann, J., & Thottingal, S. (2020). OPUS-MT: Building open translation services for the world. Proceedings of EAMT, 479–480.
17. Toral, A., & Way, A. (2018). What level of quality can neural machine translation attain on literary text? Translation Quality Journal, 9(2), 138–164. https://doi.org/10.1017/S1470542718000089
18. Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://doi.org/10.48550/arXiv.1706.03762
19. Wu, Y., Schuster, M., et al. (2016). Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint. https://arxiv.org/abs/1609.08144
20. Zhang, H., Li, J., Liu, H. (2023). Comparative evaluation of DeepSeek and ChatGPT in multilingual NLP tasks. Journal of Artificial Intelligence Research, 71, 90–112.
21. Zhang, Y., Sun, S., Galley, M., et al. (2020). Dialogpt: Large-scale generative pretraining for conversational response generation. ACL, 270–278. https://doi.org/10.18653/v1/2020.acl-main.372
