OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing

Overview of OmniOPSD. A single Omni-MLLM plays both roles. The student (blue) rolls out from a label-free prompt; the teacher (orange), conditioned on training-only privileged information (label, reference answer, optional rationale), re-scores the same tokens with no gradient. A token-level JSD divergence distills the teacher into the student on-policy, with an optional task reward. At inference, only the label-free student path is used.

Overview

Abstract

Reinforcement learning for multimodal large language models (MLLMs) is often hindered by severe reward sparsity in complex reasoning tasks. This challenge is particularly pronounced in human-centered scenarios involving states, emotions, intentions, and behaviors, where heterogeneous multimodal signals and subjective human factors make high-quality chain-of-thought (CoT) annotations expensive and difficult to obtain. Although many multimodal datasets provide expert-annotated ground-truth labels, directly using these labels for supervised fine-tuning may encourage shortcut learning and provides limited transparency for safety-critical human–AI interaction. To address these limitations, we propose OmniOPSD, a Rationale-Privileged On-Policy Self-Distillation framework that uses frontier-generated rationales as teacher-side privileged evidence rather than student imitation targets. The student samples its own rollout from the original multimodal input, while the rationale-privileged teacher scores the same tokens and provides dense token-level supervision. Thus, the student learns on its own trajectory distribution without directly imitating frontier-model completions, and inference requires no labels, rationales, CoT annotations, or closed-source model access. Experiments on MER-UniBench show that OmniOPSD achieves state-of-the-art performance with an average score of 84.19, and ablations further support the value of rationale-privileged teacher guidance.

Key Idea

Decoupling evidence from imitation

Frontier MLLMs can describe rich multimodal evidence, but their generated rationales are unverified outputs, not gold reasoning. OmniOPSD keeps that evidence on the teacher side and keeps optimization on the student's own trajectories.

Rationale as privileged evidence

Frontier-generated evidence-aware rationales condition a local teacher during training only. They are never used as target sequences for the student, which separates evidence acquisition from policy learning.

On-policy self-distillation

The student samples its own completion; the rationale-privileged teacher re-scores the same tokens and supplies dense token-level guidance via a generalized JSD objective. One local model, an EMA teacher — no frontier logits.

Label-free at inference

Supervision is computed entirely within a local model — no cross-tokenizer distillation and no closed-source access. At test time the model sees only the original multimodal input.

“In multimodal reasoning, knowing the answer is not knowing the evidence.”

Main Results

State of the art on MER-UniBench

84.19

MER-UniBench mean
(audio–video–text) — new SOTA

+4.71

points over the strongest
tri-modal baseline

3 / 3

best mean across all
modality settings

Model	Sentiment				Basic Emotion				Fine-grained	Mean
Model	MOSI	MOSEI	SIMS	SIMS v2	MER23	MER24	MELD	IEMOCAP	Fine-grained	Mean
Input modality: Audio, Text
OneLLM	64.01	54.09	63.39	61.98	25.52	17.21	28.32	33.44	22.25	41.14
SECap	55.76	54.18	59.51	57.41	40.95	52.46	25.56	36.92	36.97	46.64
PandaGPT	66.06	61.33	62.93	58.88	33.57	39.04	31.91	36.55	31.33	46.84
Qwen-Audio	70.09	46.90	70.73	65.26	41.85	31.61	49.09	35.47	32.36	49.26
SALMONN	81.00	67.03	68.69	65.93	55.53	45.38	45.62	46.84	45.00	57.89
AffectGPT	83.46	80.74	82.99	83.75	72.94	73.41	56.63	55.68	59.98	72.18
AffectGPT-R1	80.13	80.01	84.49	82.31	81.69	93.49	63.74	63.85	65.49	77.24
OmniOPSD (Ours)	88.90	87.37	87.14	86.52	77.27	86.47	71.06	86.50	56.09	80.81
Input modality: Video, Text
Otter	52.89	50.44	57.56	53.12	16.41	14.65	22.57	29.08	16.63	34.82
Video-LLaVA	56.37	61.64	53.28	57.45	36.93	30.25	30.73	38.95	34.00	44.40
PandaGPT	58.50	64.25	62.07	65.25	39.13	47.16	38.33	47.21	35.07	50.77
Video-ChatGPT	54.42	63.12	64.82	65.80	44.86	46.80	37.33	56.83	39.80	52.64
VideoChat2	66.84	54.32	69.49	70.66	33.67	54.50	36.64	48.70	39.21	52.67
LLaMA-VID	61.78	63.89	69.35	67.48	50.72	57.60	42.75	46.02	45.01	56.07
VideoChat	65.13	63.61	69.52	72.14	48.73	57.30	41.11	48.38	44.52	56.71
Chat-UniVi	54.53	63.18	68.15	66.36	57.62	65.67	45.61	52.37	48.00	57.94
mPLUG-Owl	72.40	72.91	72.13	75.00	56.86	59.89	49.11	55.54	48.18	62.45
AffectGPT	82.39	81.57	87.20	86.29	74.58	75.29	57.63	62.19	61.65	74.31
AffectGPT-R1	78.78	79.07	85.91	85.85	77.72	85.29	61.09	67.42	62.42	75.95
OmniOPSD (Ours)	88.29	88.12	87.53	87.94	76.32	85.27	68.16	79.84	56.24	79.74
Input modality: Audio, Video, Text
PandaGPT	61.92	67.61	68.38	67.23	40.21	51.89	37.88	44.04	37.12	52.92
R1-Omni	58.02	56.48	71.82	68.58	64.17	67.43	43.20	51.58	55.24	59.61
Emotion-LLaMA	66.13	67.66	78.32	77.23	59.38	73.62	46.76	55.47	52.97	64.17
Emotion-LLaMAv2	86.28	87.69	87.72	86.02	77.28	86.90	51.32	84.05	62.90	78.91
AffectGPT	81.30	80.90	88.49	86.18	78.54	78.80	55.65	60.54	62.52	74.77
AffectGPT-R1	79.39	79.24	88.25	84.97	84.32	93.75	63.12	74.26	68.05	79.48
OmniOPSD (Ours)	90.06	89.03	90.76	89.14	85.79	92.43	71.89	89.09	59.52	84.19

Main results on MER-UniBench. WAF for MOSI / MOSEI / SIMS / SIMS v2; HIT for MER23 / MER24 / MELD / IEMOCAP; EW-F1 for OV-MERD+; Mean is the unweighted average. Best per column in orange. Baselines reproduced from published MER-UniBench tables.

Analysis

Training dynamics & ablations

Training dynamics comparing OmniOPSD and GRPO on Qwen2.5-Omni-3B across answer reward, format reward, and completion length.

Training dynamics on Qwen2.5-Omni-3B. Against outcome-only GRPO, OmniOPSD drives both the answer-accuracy reward and the format reward up faster and higher. Its completion length grows then stabilizes rather than collapsing — the model preserves reasoning behavior instead of degenerating into short, shallow answers.

CoT-style privileged context helps OPSD, not GRPO

In-domain macro-F1 with Qwen2.5-Omni-3B. The same generated CoT context is used only as teacher-side privileged evidence.

Method	MELD	MIntRec 2.0	IEMOCAP
GRPO w/o CoT	0.6088	0.2982	0.2272
GRPO w/ CoT	0.6093 ↑0.08%	0.2921 ↓2.05%	0.2208 ↓2.82%
OmniOPSD w/o CoT	0.6289	0.3192	0.2254
OmniOPSD w/ CoT	0.6427 ↑2.19%	0.3248 ↑1.75%	0.2388 ↑5.94%

Generated rationales are never a target the student imitates. They help most as the privileged context that conditions the local teacher scoring the student-generated tokens.

Cite

BibTeX

@article{cheng2026omniopsd,
  title   = {OmniOPSD: Rationale-Privileged On-Policy Self-Distillation
             for Affective Computing},
  author  = {Cheng, Zebang and Chen, Shuimu and Yang, Boxue and
             Guan, Yuanshen and Chen, Jingyi and Lian, Zheng and
             Peng, Xiaojiang and Ma, Fei and Cui, LaiZhong and Tian, Qi},
  journal = {arXiv preprint arXiv:2606.15920},
  year    = {2026},
  url     = {https://omniopsd.github.io/}
}