- In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning (RL). Reproducing existing work and accurately judging the improvements offered by novel methods is vital to maintaining this rapid progress. Unfortunately, reproducing results for state-of-the-art deep RL methods is seldom straightforward. In particular, non-determinism in standard benchmark environments, combined with variance intrinsic to the methods, can make reported results difficult to interpret. Without significance metrics and tighter standardization of experimental reporting, it is difficult to determine whether improvements over the prior state-of-the-art are meaningful. In this paper, we investigate challenges posed by reproducibility, proper experimental techniques, and reporting procedures. We illustrate the variability in reported metrics and results when comparing against common baselines, and suggest guidelines to make future results in deep RL more reproducible. We aim to spur discussion about how to ensure continued progress in the field, by minimizing wasted effort stemming from results that are non-reproducible and easily misinterpreted.
[hide][/hide]
- 近年来，深度强化学习（RL）被用于解决很多领域中的难题，并取得了令人瞩目的成绩。为了保持快速发展的局面，复现（Reproducing）已有的研究并准确评估新方法所带来的进步是很重要的。可惜，顶尖的深度强化学习方法很少能被简单的复现。尤其是，标准基准环境中的不确定性和不同方法之间的内在差异导致研究中的结果难以理解。如果实验过程缺乏显著性的度量和严格的标准化，则我们很难确定先前顶尖技术取得的进展是否有意义。在这篇论文中，我们研究了复现实验所面临的挑战、合适的实验技巧和报告流程。通过与常见基准进行对比，我们阐释了报告中度量方法和结果的可变性，同时提出了使深度强化学习未来的研究成果更易复现的指南。我们希望减小研究人员在不可复现和易误解的结果上花费精力，并引起大家对如何使该领域持续发展进行讨论。
[hide][/hide] |
熟悉论坛请点击新手指南
京公网安备 11010802022788号
论坛法律顾问：王进律师
知识产权保护声明
免责及隐私声明
