Publication
arXiv
Stage
Preprint
What we read
Summary of the paper
Authors
Lei Yang, Mengyin Liu, Jia Wang, Hangyu Guo, Liang Zhao, Zheng Ge, Kang An, Binxing Jiao, Qi Han, Daxin Jiang, Siqi Shen, Xiangyu Zhang
Universities and research institutions
Not yet supplied in verified metadata; the Brief does not guess.

What they did and found

onPanda uses a token-level correction loop: the annotator locates the first inappropriate token, substitutes from top candidates or edits the token, then truncates the rest and continues generation from the corrected prefix. A small controlled study found median annotation time fell by 52% versus manual post-editing, with data remaining close to the model’s sampling distribution.

Why it matters

This approach can speed up creating on-policy alignment data, but results depend on model, task, and API capabilities.

onPanda’s core idea is to let humans fix only a few errors and then let the model continue. This preserves most of the model’s original behavior while speeding up the labeling process. The method also records token-level fixes, which can support more precise training signals.

In practice, speed and fidelity depend on having access to models that expose token probabilities and allow token-level editing. When these conditions hold, token-level correction can make alignment data generation cheaper while staying aligned with the rollout model’s distribution.

Read the paper PDF ↗

Original sources · 1
  1. onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction ↗arXiv · 2026-09-21

Check the original paper for its authors, methods, version and access terms.

What remains uncertain

Controlled, small-scale study; generalizability to broader models/tasks uncertain.