pjrule

I’ve done something like this in the past with a spreadsheet, but I recently fell out of the habit because of the overhead of taking a few minutes at the end of a long day and opening the spreadsheet up on my laptop. I’d like some kind of cross-platform life logging app that is equally usable for longform entries when I have full access to a good keyboard and quick notes when I just have a phone and a few seconds of time. Some kind of highly customizable rich schema support (long text entries + basic biometrics like self-reported sleep) would be ideal. Any suggestions?

Yeah, Hanson phrases this idea poorly. It might be better to say that creating large backroom software has something of a high barrier to entry. For instance, popular packages like DynamoDB (internal to Amazon.com before public release) and TensorFlow (internal to Google before public release) make certain types of work easier and more efficient; they’re also large enough that they couldn’t easily be reproduced by smaller competitors. Those companies had to pay the said “large fixed cost,” as well as a continuous cost for maintenance, to develop those tools. Economies of scale play a related role here—the more engineers you can afford to hire, the less you have to outsource and the more you can create in-house solutions that can be reused companywide and ultimately give you a competitive advantage long-term.

In fairness to Hanson, a natural monopoly is distinct from a “monopoly” in the popular sense. Public utilities are often natural monopolies—for instance, a city might have only one power company because it doesn’t make sense to build the redundant infrastructure for two power companies. Doing so would incur a high, unnecessary fixed cost. [1] I think Hanson’s trying to draw an analogy to such firms, though I find it a bit loose.

[1] https://en.m.wikipedia.org/wiki/Natural_monopoly

Ben Recht also has an excellent series of blog posts (very related to this survey on arXiv, but broader) on the intersection between reinforcement learning and optimal control. An index is available here: http://www.argmin.net/2018/06/25/outsider-rl/

That’s excellent! Thanks!

Thanks so much! I read this (and a few related papers) today. Besides the novel algorithm discussed in the new Atari paper, do you have a reference implementation of online vector quantization you might be able to recommend? I think I could probably figure it out from the paper alone, but sometimes it's nice to see code other people have already optimized. :)

As someone working on a reinforcement learning/neuroevolution problem right now, I find this to be extremely exciting. Fewer parameters, ceteris paribus, is always better—the fact that the experiments in this paper were run on one workstation, rather than on a massive farm of TPUs à la AlphaGo, implies quicker development iteration time and more accessibility to the average researcher.

The staging of components in this paper (compressor/controller), where neuroevolution is only applied to a low-dimensional controller, reminds me of Ha and Schmidhuber's recent paper on world models (which is briefly cited) [1]. They employ a variational autoencoder with ~4.4M parameters, an RNN with ~1.7M parameters, and a final controller with just 1,088 parameters! Though it's recently been shown that neuroevolution can scale to millions of parameters [2], the technique of applying evolution to as few parameters as possible and supplementing with either autoencoders or vector quantization seems to be gaining traction. I hope to apply some of the ideas in this paper to multiple co-evolving agents...

[1]. https://worldmodels.github.io

[2]. https://arxiv.org/abs/1712.06567

Indeed, I had a similar experience. Perception is strange.

This is a fantastic idea, and I think I’ll definitely be applying! Have you considered offering two tiers of grants? Most AI side projects don’t necessarily require ~$30k in resources, but I’m sure many people, including myself, could benefit from ~$3k in GCE credit. Opening up the same total resources to more people would allow for a greater number of moonshots, which are what this grant seems to be about. :)

Natural selection at work? ;)