Does Dialect Tag Conditioning Help? a Systematic Benchmark of Neural Machine Translation for Indonesian-Bugis

Abstract

Machine translation for local languages remains challenging due to the scarcity of data. This challenge also occurs with languages that have dialects. In this study, we focus on the translation of Buginese dialects, an underexplored area in neural machine translation research. To support this investigation, we introduce the Indonesian-Bugis parallel corpus annotated with ten Bugis dialect labels, consisting of 10,500 sentence pairs. We evaluate three NMT families of sequence-to-sequence architectures. The first comprises bidirectional recurrent encoders (BiRNN, BiLSTM, BiGRU), in which the encoder processes the source sequence in both forward and backward directions. The second comprises their unidirectional counterparts (RNN, LSTM, GRU), which process the source sequence in a single forward direction. The third is the Transformer, which replaces recurrence with multi-head self- attention. All recurrent variants are equipped with an attention mechanism over the encoder states. We further investigate dialect conditioning through three input/output settings-untagged, source-side dialect-tagged, and both-side (source and target) dialect-tagged-yielding twenty-one experiments evaluated with BLEU, chrF++, and TER. As the result, the Transformer achieved the highest BLEU (1.86 with source-side tags and 1.84 with both-side tags), while the both-side-tagged Transformer obtained the best (lowest) TER (98.11). In contrast, the untagged GRU-Attention model attained the highest chrF++ (19.75). The impact of dialect tagging was inconsistent: source-side tags raised mean BLEU modestly and reduced TER but lowered mean chrF++, improving BLEU for only four of the seven architectures, and both-side tagging performed comparably to source-side tagging. For both side-tagged systems, the models reproduced the correct target dialect tag with near-perfect accuracy (~100%). Per-dialect analysis revealed significant disparities (e.g., Bulukumba, Sengkang, and Sidrap were most affected, and Sidrap exhibited the highest variance); these differences are due to the strong linguistic variation among Bugis dialects.