BioBigBird: A Sparse Attention Model for Long-Range Dependency Processing in Biomedical Text
While domain-specific Large Language Models (LLMs) have encoded vast biomedical knowledge, their limited context windows often hinder a deep understanding of nuanced relationships within and across texts. To address this limitation, we introduce BioBigBird, a bidirectional language model pre-trained on extensive biomed...
Roshan Balaji, S. Pavan Kumar, Vasudev Gupta et al.
· 0 citations