Publication
arXiv
Stage
Preprint
What we read
Summary of the abstract
Authors
Lei Yang, Mengyin Liu, Jia Wang, Hangyu Guo, Liang Zhao, Zheng Ge, Kang An, Binxing Jiao, Qi Han, Daxin Jiang, Siqi Shen, Xiangyu Zhang
Universities and research institutions
Not yet supplied in verified metadata; the Brief does not guess.

What the paper reports

The paper presents onPanda, an interactive tool where annotators locate the first inappropriate token, choose a substitute from the model’s candidates or edit the text, then generation resumes from the corrected prefix. A small controlled study suggests this reduces median annotation time by about half compared with manual post-editing, while preserving the model’s sampling distribution for on-policy data.

Why it matters

Token-level correction supports precise, low-cost supervision that can yield better-aligned data for LLMs and agents and enables practical, fine-grained trajectory annotation for realistic environments.

onPanda introduces a correction-first workflow: annotators identify the first token that deviates from a desired response, select a replacement from the model’s candidate tokens or edit the text, and then push the corrected prefix forward to continue generation. This loop repeats until the response fits the annotator’s goals, with the majority of final tokens generated by the model. The approach aims to retain the model’s sampling distribution while providing precise, position-aware supervision that yields natural positive–negative samples for on-policy data.

the authors also release Panda-CVL, a dataset annotated with onPanda, and a benchmark for token-level correction, highlighting the method’s potential to support on-policy supervised fine-tuning and preference data without overhauling existing generation pipelines. The work emphasizes efficiency and practicality, suggesting that token-level, in-context edits can dramatically cut annotation time while preserving useful distributions for subsequent learning and evaluation.

What this does not tell us

Abstract-only scope; preprint; findings come from a small controlled study and may not generalize to broader populations or all tasks.

Original sources · 1
  1. onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction ↗arXiv · 2026-09-21

Check the original paper for its authors, methods, version and access terms.