Trained visual-interface agents generalize to new tasks on real sites
A new set of web environments tests how a visual-interface agent trained in WebHarbor handles tasks on live sites, and what the results mean for real-world use.
- Publication
- arXiv
- Stage
- Preprint
- What we read
- Summary of the paper
- Authors
- Zhaoyang Wang, Qianhui Wu, Boyu Gou, Demi Wang, Jiacheng Zhang, Peichun Hua, Qize Li, Shi Qiu, Yifan Jiang, Yifan Zhang, Zexu Jin, Baolin Peng, Bolin Shen, Chetan Bansal, Dehan Kong, Deren Lei, Dongshen Peng, Fengqing Jiang, Hao Cheng, Haoqing Wang, Jiaming Shan, Jicheng Wang, Jingguang Zhou, Jixuan He, Kaide Zeng, Kangyu Zhu, Lama Moukheiber, Mufan Qiu, Peng Xia, Qihan Sun, Qingwei Lin, Qiushi Sun, Ruichen Zhang, Shuqing Luo, Si Qin, Sicheng Fan, Tao Xiong, Wenlin Yao, Xinyu Zhang, Xinyue Zeng, Xuanrui Li, Xuchao Zhang, Yian Wang, Yichao Liu, Yiming Liang, Yiyang Zhou, Yizhu Jiao, Yuetai Li, Yurun Chen, Yuxuan Zhang, Zheyuan Yang, Zhiyuan Yao, Zhongyang Li, Zihao Li, Zili Wang, Jianfeng Gao, Huaxiu Yao
- Universities and research institutions
- Not yet supplied in verified metadata; the Brief does not guess.
What they did and found
A visual-interface agent trained with rewards in WebHarbor was tested on unseen live tasks and sites. It performed best among open-source options and stayed competitive across four benchmarks, though it dropped when login was disabled and it imitates sites rather than reproducing them.
Why it matters
Offline training in realistic settings can still yield solid results on real sites, but limits come from how closely the training mirrors real pages and authenticated tasks.
WebHarbor creates many realistic web tasks and allows learning offline. The agent scored well on live-site tests when access to live pages was allowed.
The main caveat is that the environment imitates a site, not copies it exactly. Real sites change, and login or complex actions can pose challenges.
What remains uncertain
WebHarbor imitates target websites rather than reproducing them exactly; evaluation may miss some real-world edge cases and some authenticated workflows were tested with limited coverage.
Original sources · 1
- WebHarbor: Docking Real Websites for Evolving GUI Agent Environments ↗arXiv · 2026-11-02
Check the original paper for its authors, methods, version and access terms.
What is your take?
Ask a question, add useful context or share a different perspective. Keep the conversation respectful and grounded.