History-Dependent Logging Makes Off-Policy Evaluation Exponentially Hard, Researchers Show
A preprint finds constant state coverage can still yield exponentially uninformative data when loggers depend on history.
- Publication
- arXiv
- Stage
- Preprint
- What we read
- Summary of the paper
- Authors
- Pranaya Jajoo
- Universities and research institutions
- Not yet supplied in verified metadata; the Brief does not guess.
What they did and found
We present two small-state POMDPs with a common logger that carries memory. Even with bounded action and belief coverage, any estimator of a known target policy needs exponentially many logged episodes as horizon grows. A reset mechanism erases critical transitions, making information decay despite rich data and known models.
Why it matters
This warns that simple coverage guarantees may not ensure reliable offline policy evaluation when logs hinge on past histories. Stronger structure tying logger continuation to history is needed before trusting target-policy values.
The paper builds two environments where past actions shape future data. Even with constant chances of actions and spread across latent states, distinguishing the true policy value requires many samples, especially as the horizon grows.
The result implies organizations should not assume offline estimates are cheap or quick when logging depends on history; stronger assumptions or data may be needed for credible evaluation.
Original sources · 1
Check the original paper for its authors, methods, version and access terms.
What remains uncertain
The findings come from a specific worst-case construction with history-dependent logging; applicability to all real-world settings needs further validation.
What is your take?
Ask a question, add useful context or share a different perspective. Keep the conversation respectful and grounded.