End-to-End AI-Native Physical Layer for Robust 6G MIMO-OFDM: Adaptive Constellation Shaping and Attention-Based Detection

Abstract

Sixth-generation (6G) physical-layer designs require robustness against non-analytical channel distortions and hardware impairments that violate classical linear assumptions. This paper presents an AI-native framework for MIMO-OFDM systems that jointly optimizes adaptive constellation shaping and neural detection through end-to-end learning. The proposed model employs a differentiable channel layer incorporating Rayleigh fading, power amplifier nonlinearities, and phase noise, enabling gradient-based optimization of complex constellation coordinates under strict average power constraints. The receiver utilizes a Real-Valued Feedforward Neural Network with Spatial Attention (RFNN-SA) to dynamically weight fading streams and mitigate channel estimation errors. Extensive simulations demonstrate that the proposed model achieves a 3.2 dB SNR gain at BER=10⁻³ for 16-QAM, 3.8 dB for 64-QAM, and 4.1 dB at BER=10⁻² for 256-QAM over MMSE detection at equivalent operating points. Under realistic CSI uncertainty, performance degrades by only 22%, compared to 52% for classical baselines. With a 0.95 ms physical‑layer detection inference latency, the proposed architecture provides a computationally efficient and impairment-resilient foundation for practical 6G physical-layer deployments.