Comparative Analysis of Transformer Model: mBART and mT5 on Question Answering System for Nepali Text

Abstract

Despite significant advances in English question answering using transformer models such as Text-To-Text Transfer Transformer (T5), Bidirectional Auto-Regressive Transformers (BART), and Generative Pre-trained Transformer (GPT) trained on datasets like Standford Question Answering (SQuAD), research on Nepali question answering remains limited due to the scarcity of annotated data and fine-tuned models. This study presents a comparative analysis of two multilingual transformer models mBART and mT5 for Nepali question answering using transfer learning. A translated Nepali SQuAD dataset was developed and fine-tuned with both models, incorporating data augmentation to address data scarcity. Evaluation using BLEU, ROUGE, BERTScore, Exact Match, and F1 Score shows that both models perform well, with mBART slightly outperforming mT5. This work provides a foundation for future research on Nepali question answering systems.