Open access
Jul 2026
GENERALIZING TINYBERT WITH CROSS-ATTENTION, LORA AND BI-GRU FOR DEEP LEARNING OVER NON-NUMERIC DATA: BEYOND FAKE NEWS DETECTION
A universal hybrid architecture integrating the compact TinyBERT transformer, a cross-attention mechanism, low-rank adaptation (LoRA), and a bidirectional gated recurrent unit (Bi-GRU) is proposed, capable of delivering high accuracy, interpretability, and computational efficiency for heterogeneous textual data in multi-task and multilingual environments.
O. Khobor, V. Lytvyn
· Наука і техніка сьогодні · 0 citations