OffCon$^3$: What is state of the art anyway?

Author name not available (Why is that?)

Publication date: 27 January 2021

Abstract: Two popular approaches to model-free continuous control tasks are SAC and TD3. At first glance these approaches seem rather different; SAC aims to solve the entropy-augmented MDP by minimising the KL-divergence between a stochastic proposal policy and a hypotheical energy-basd soft Q-function policy, whereas TD3 is derived from DPG, which uses a deterministic policy to perform policy gradient ascent along the value function. In reality, both approaches are remarkably similar, and belong to a family of approaches we call `Off-Policy Continuous Generalized Policy Iteration'. This illuminates their similar performance in most continuous control benchmarks, and indeed when hyperparameters are matched, their performance can be statistically indistinguishable. To further remove any difference due to implementation, we provide OffCon

^{3}

(Off-Policy Continuous Control: Consolidated), a code base featuring state-of-the-art versions of both algorithms.

Has companion code repository: https://github.com/fiorenza2/OffCon3

This page was built for publication: OffCon$^3$: What is state of the art anyway?

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6359164)