Skip to content
Preprint

Momba: Network Modernization Improves Multi-Objective Reinforcement Learning

Aug 2026 · 0 citations · 43 references
Computer Science

TL;DR

Recent advances in neural network design are integrated: observation and feature normalization, weight normalization, and modeling of distributional returns with an entropy-regularized MORL algorithm, demonstrating that these changes substantially improve the quality of the produced solution sets without requiring major changes to the underlying algorithm.

Abstract

Recent advances in deep reinforcement learning (RL) have shown that improving neural network architectures can yield substantial gains in sample efficiency and asymptotic performance without altering the underlying algorithms. In contrast, work on multi-objective reinforcement learning (MORL), which aims to discover a set of policies that balance trade-offs among conflicting objectives, has predominantly focused on algorithmic innovations, leaving the area of architectures underexplored. While the optimal policies and value functions can differ significantly depending on the trade-offs, MORL algorithms commonly represent them with simple feedforward networks conditioned on the trade-off. This raises the question of whether the performance of the algorithms could be improved with more expressive function approximators. In this paper, we integrate recent advances in neural network design: (i) observation and feature normalization, (ii) weight normalization, and (iii) modeling of distributional returns with an entropy-regularized MORL algorithm. The empirical results across standard continuous control benchmarks demonstrate that these changes substantially improve the quality of the produced solution sets without requiring major changes to the underlying algorithm.

View source

Similar papers

Open access Oct 2026

Controllable Orthogonalization for Stabilizing Neural Network Training in Deep Reinforcement Learning

Deep reinforcement learning (DRL) relies on neural networks trained under temporally correlated experience, evolving replay distributions, and bootstrapped targets, conditions that can destabilize neural function approximation. This study investigates controllable orthogonalization as a network-level mechanism for impr...

Jia Yu, Dun Su, Hua Cui et al. · 0 citations

Towards Better Training Signal: Advantage Clipped Policy Optimization

ACPO is a practical and effective alternative to conventional IS-ratio clipping for RL post-training of LLMs, and establishes a connection between ACPO and gradient clipping in policy mirror descent (PMD), which is a standard technique to stabilize optimization process.

Rui-Chuan Huang, Jing-Han Liu, Cong-Liang Chen · 0 citations
#machine learning Preprint Aug 2026

Parameter Exploration for RLVR via Variational Learning

Evidence that parameter-space exploration can improve reinforcement learning for LLMs is presented, and a family of methods called Perturbed Parameter Policy Optimization (3PO) is introduced which use different sampling strategies and different rollout grouping for reward estimation.

Vatsal Venkatkrishna, Nico Daheim, Iryna Gurevych · 0 citations
#natural language process... Preprint Sep 2026

Expert-Space Exploration in MoE Reinforcement Learning

ESRL is introduced, an architecture-aware framework that explicitly explores the expert-routing space of MoE models, and preserves high-confidence experts as anchors, and restricts stochastic routing to a plausible candidate pool, thereby retaining reliable computation paths.

Hong-Yi He, Zheng-Wen Lin, Xiao Liu et al. · 0 citations
#machine learning Preprint Oct 2026

Understanding Enrichment in Reinforcement Learning

When rewards are sparse, reinforcement learning with verifiable rewards (RLVR) often uses hints or intermediate guidance to generate more successful rollouts. This enrichment biases policy-gradient updates unless corrected via importance weights, but existing methods omit correction or truncate importance weights in or...

Jinwoo Kim, Shraddha Barke · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.