Direct Preference Optimization (DPO) has become an important method for aligning large language models (LLMs) with human preferences because it removes the need for explicit reward modeling and reinforcement learning. However, its performance depends heavily on the quality of preference data, and noisy preference data in real-world settings can weaken alignment performance. To address this issue, we propose a bilevel optimization framework and prove, under some idealized conditions, that this framework can recover the DPO optimum under clean data. We further derive a prior form for the learnable weighting function under label-flipping noise. Considering that high-quality metadata may be difficult to obtain, we propose a prompt augmentation consistency method that enables meta-learning even when metadata is completely unavailable. To reduce the high cost of higher-order gradients in LLM meta-learning, we combine central-difference approximation with LoRA fine-tuning and develop a scalable training scheme. Experiments on TL;DR summarization and Anthropic Helpful and Harmless dialogue show that the proposed method improves alignment performance over multiple DPO baselines under different noise rates.
Low Earth orbit (LEO) satellite networks exhibit rapidly changing topology and time-varying traffic hotspots, which makes hop-by-hop routing highly sensitive to local congestion and state staleness. Existing routing methods either rely on global path computation or use plain local observations, while graph-enhanced approaches often focus on generic neighborhood representation rather than direct comparison among candidate next hops. To address this issue, this paper proposes a Local Graph-Aware Routing method (LGAR) for dynamic LEO satellite networks. LGAR organizes the current node, reachable candidate neighbors, and candidate links into a local graph, and then constructs structured action representations through node encoding, relation message extraction, and attention-based context aggregation. The resulting representations are integrated into an off-policy actor-critic framework to support adaptive hop-by-hop routing decisions. Experiments under the hub-inversion setting show that LGAR achieves an average total delay of 47.64 ms and an average queueing delay of 5.81 ms while maintaining a delivery rate of 99.93%. Compared with MATMR, LGAR-NoGraph, and GRLR, LGAR reduces the average total delay by 12.38%, 12.85%, and 30.75%, respectively. Additional scenario, ablation, and scalability results further show that LGAR generalizes beyond the main setting and that its gain mainly comes from local graph modeling and relation-aware action encoding.
Wen-Xiang Zhang, Yiao Gao, Ke-Yan Bai et al.· 2026 8th International Confe...· 0 citations
