Skip to content

SmoLSTM: A Compact Vision-Language-Action Model with Recurrent Memory that Persists

Sep 2026 · arXiv.org · Vol abs/2609.22854 · 0 citations · 26 references
Computer Science

TL;DR

SmoLSTM couples a frozen 256M-parameter SmolVLM backbone to a matrix-memory LSTM control layer in which observation tokens and action queries are unified in a single causal stream that is never reset throughout the entire episode, capturing the episode in a recurrent state.

Abstract

Vision-language-action models often predict actions from only the current observation, which can leave tasks involving object occlusion or visually identical objects ambiguous without episode history. The usual countermeasure, widening the observation window, turns the horizon into a hyperparameter and lets per-step cost grow with it. We instead capture the episode in a recurrent state. SmoLSTM couples a frozen 256M-parameter SmolVLM backbone to a matrix-memory LSTM control layer in which observation tokens and action queries are unified in a single causal stream that is never reset throughout the entire episode. Recurrent-state storage is therefore O(1) in episode length. A flow-matching action head predicts chunks of 10 end-effector pose deltas and gripper commands at each control step. Our single policy, trained jointly on 7,461 demonstrations across 140 tasks and evaluated on held-out initial states, performs best, reaching 85.1% subgoal coverage and 77.5% full-task success on LIBERO-Mem with 0.04B trainable parameters, surpassing both the benchmark's own object-centric baseline and recent memory-based approaches. Resetting the recurrent state at every control step reduces full-task success to 7.0%, showing that the trained policy relies on context carried between decisions. The same model achieves 79.6% average success on standard LIBERO.

View source

Similar papers

MemBodied: Recurrent Associative Memory for Vision-Language-Action Models

Findings support MemBodied as a practical alternative to expanding the policy context for history-dependent manipulation, a fixed-size episodic memory with two complementary components: an associative state that records interactions across policy calls and an episode anchor that preserves a compact representation of th...

Tej Deep Pala, N. Majumder, Bryce Goh et al. · 1 citation
#machine learning Preprint Sep 2026

SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Models

SimpleMemVLA, a VLA without a dedicated memory module that uses the backbone's native video context directly as memory, achieves state-of-the-art results on four memory benchmarks without loss on general-purpose control, and with the same backbone and training setup it outperforms retrieval, compression and recurrent-s...

C. Yin, Wang Xu, Jun-Peng Yang et al. · 4 citations
Preprint Aug 2026

G0.5: One Autoregressive Stream for Robot Reasoning and Action

G0.5 is introduced, a pretrained autoregressive VLA in which a single transformer decoder emits reasoning and action tokens under a single objective, which exceeds state-of-the-art models across 7 independent regimes.

Yi-Cheng Liu, Zibin Dong, Baijun Ye et al. · 17 citations · ⚡2
Preprint Aug 2026

PredVLA: Predictive Sensorimotor Modeling for Sub-Million-Parameter Robot Manipulation

A mechanism-by-mechanism transition to the recurrent behavior-cloning baseline shows that replacing the predictive pathway with direct observation input produces the largest single performance drop, accounting for approximately $70\% of the endpoint gap.

Hiroki Sawada, Shunichi Kasahara · 1 citation
Preprint Sep 2026

LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory

LookStep is proposed, a unified end-to-end framework that combines Language Centric Future State Modeling and Event Driven Rolling Memory that uses language labels to generate coarse-grained navigation progress and future states for each candidate action, while autonomously deciding whether to write each observation in...

Kun-Yang Yu, Ying-Zhe Li, Hongyu Xu et al. · 2 citations
#machine learning Preprint Aug 2026

LM-X: Explainable Vision--Language--Action Modeling via Progress, Event, and Uncertainty Prediction

LM-X is introduced, which organizes prediction across task, event, and motor scales without claiming anatomical correspondence and shows that explicit multi-timescale predictive state can strengthen control while exposing interpretable internal estimates.

Jin Lou, Zhi Jing, Xu-Peng Wang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.