14 karma · joined November 9, 2020
I was always wondering about a question that is touched in the paper: despite the analytic gradient computation being intuitively more efficient and mathematically correct, it is much harder to learn a policy with it than with the “brute force trial-and-error” black box methods.
This paper brings many new perspectives on why.
Instead, using stochastic gradient approximations (policy gradient method such as proximal policy optimization) might be better suited to solving these kinds of problems. Effectively, they do not compute the exact gradient locally, but rather kind of a global approximation by trying out random sequences of actions and determining which of them are closest to the desired outcome.
Hence, stochastic gradient approximations might be considered some kind of hybrid between greedy local optimization (such as following the exact gradient) and global optimization.
However, realizing the profit is a different thing. That will work only for a few who invested early and deinvest early. Basically, btc is a pyramid scheme, the only difference to other pyramid schemes is that this fact is not obfuscated. Every investor knows that he is buying a fundamentally worthless good, but is hoping that someone else will eventually pay him a higher price.
That works well until the market is saturated and no one is willing to buy btc any more. Then, the price collapses.
Thus: even if most investors today have made profit, that simply means that the bottom of the pyramid has not been reached. At that point, most investors will have made loss.
The ergonomics of a split keyboard is very appealing. For the last months, I’ve simply used two thinkpad trackpoint usb keyboards as a cheap but working substitute :-D
import ipdb; ipdb.pm()
for quick post mortem debugging, which also allows to see the variables at the moment where the exception was raised (Command locals()).
Is there a scenario where this adds value?