Sheyasree Ghosh

A Comprehensive Evaluation of Long-Context Reasoning in Transformer and State-Space LLM Architectures

A comparative study of Transformer-based and State-Space Model designs when it comes to long-context reasoning finds Transformer-based models have a pronounced tendency to forget distant information while StateSpace Model-based designs retain information more stably and yield better results for tasks that involve long...