Vasudev Gupta

BioBigBird: A Sparse Attention Model for Long-Range Dependency Processing in Biomedical Text

While domain-specific Large Language Models (LLMs) have encoded vast biomedical knowledge, their limited context windows often hinder a deep understanding of nuanced relationships within and across texts. To address this limitation, we introduce BioBigBird, a bidirectional language model pre-trained on extensive biomed...