Large language model agents are emerging as an important new class of users of desktop applications. However, existing desktop interfaces were designed for human end users or application developers, leaving agents to interpret visual controls or discover and compose low-level software operations. This mismatch can hind...
Motion-Omni is presented, an end-to-end framework in which a spoken dialogue model natively outputs explicit facial expression together with hand, upper-body and lower-body motion, generated directly from the hidden states that produce the speech.
Chengqian Ma, Wei Tao, Hao-Yu Zhang et al.· 0 citations
This work constructs two human-verified benchmarks, VRQABench for controllable spatial lookahead and OpenWorldQA for open-domain physical prediction, and proposes Privileged-Future On-Policy Self-Distillation (PF-OPSD), a model that outperforms baseline by 10.6% and 10.9% on VRQABench and OpenWorldQA, respectively, whi...