One red flag for me is that they're simultaneously claiming that there's no training during the OpenAI Gym while also claiming that the optimization approach is relevant. In that case, what is being optimized? It seems like they might be optimizing over previous simulations - there's frequent reference to having access to a "simulator". In that case, that should effectively count as training, right? I was under the impression that the OpenAI Gym was supposed to benchmark untrained approaches so they could be compared by learning time. Hence the gradually increasing training curves in the other approaches.