Paraphrase changes can flip a robot reward score
A benchmark shows that different wordings for the same task can yield different progress scores.
- Publication
- arXiv
- Stage
- Preprint
- What we read
- Summary of the paper
- Authors
- Wonje Jeung, Sangyeon Yoon, Hyesoo Hong, Yoonjun Cho, Dongjae Jeon, Bumjun Kim, Jean Oh, Youngjae Yu, Albert No
- Universities and research institutions
- Not yet supplied in verified metadata; the Brief does not guess.
Institution metadata: OpenAlex record ↗
What they did and found
Researchers tested whether describing the same goal in different but equivalent words changes the robot's reward signal. Across many models, small wording changes often flip the score from success to failure, even when the robot's actions stay the same.
Why it matters
Word choice can mislead planners. Relying on a single wording may steer the robot toward the wrong goal; checking phrasings or averaging scores can cut bias.
In experiments, researchers saw that rewriting the same goal in different words changed the reward signal used to guide learning, even when robot acted same.
A practical takeaway is to test rewards with different wordings and scores across phrases to avoid teaching the robot to chase wording rather than goals.
Original sources · 1
- Same Trajectory, Contradictory Rewards (ROBORMBENCH): Paraphrase Fragility in Vision Language Reward Models ↗arXiv · 2026-09-04
Check the original paper for its authors, methods, version and access terms.
What remains uncertain
This test uses English phrases and fixed tests; results may differ in other languages or real-world tasks.
What is your take?
Ask a question, add useful context or share a different perspective. Keep the conversation respectful and grounded.