Author

Tiago Pimentel

1 paper indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#natural language process... Feb 2026

What Language is This? Ask Your Tokenizer

UniLID, a simple and efficient LID method based on the UnigramLM tokenization algorithm, is introduced, which supports incremental addition of new languages without retraining existing models, and can naturally be integrated into existing language model tokenization pipelines.

Clara Meister, Ahmetcan Yavuz, Pietro Lesci et al. · arXiv.org · 2 citations