Well, universities can’t compete in things like car production or rocket manufacturing but find ways to contribute nonetheless. Researchers have and always will struggle to get resources relative to BigCorp - AI is just joining the party. Daimler and Lockheed are no more open than Facebook is, AFAIK.
There is still plenty to do and to analyze. Verifiable AI, more efficient models, knowledge transfer, 1000 brains, human interpretability, etc.
This is a great list - thank you. I learned a long time ago to not use “you” or similar in code reviews, as it is too easy to associate a critique of code with a critique of the coder. Changing “You should change this” to “This should be changed” or even “We should change this” can make a world of difference.
The author of this post laments how deep learning works, but the suggestion is to solve the problem somehow by only reasoning about how other neurons are firing locally. Isn’t that exactly what deep neural networks already do? A basic feedforward layer has individual neurons that fire if their inputs exceed a threshold, which is based on the timing of the firing of the other neurons.
The article presents a vague non-solution.
Of course, deep learning has not (yet) cracked full self-driving or general cognition, but without providing more examples or differentiators, this post is somewhat lacking.
Research is not a zero-sum game, it’s about bringing useful knowledge to everyone. If releasing your code aids in understanding what you’re contributing, then by all means please contribute!
You don’t even need to spend a ton of time cleaning it up. Releasing it “as is” is fine.
Hoarding your code as a defense against criticism goes against the entire purpose of open academic work.
The Tufte package also allows for typesetting a book with executable Julia code, only in LaTeX: https://github.com/sisl/tufte_algorithms_book
It was used to make Algorithms for Optimization and Algorithms for Decision Making.
Love this.
Doesn’t Microsoft terminal have to be backwards compatible for a ton of historic features though? I would imagine that those sorts of requirements would essentially hobble it. Breaking backwards compatibility isn’t too intriguing for their bottom-line.
Algorithms for Decision Making was recently released as open access. The authors were able to get open access rights for their previous book, Algorithms for Optimization. It is a general reference book for optimization algorithms, with concrete implementations in the Julia programming language.
Having a partner for your side project helps a lot, in the same way having a gym buddy helps with working out. I generally need my projects to be rewarding unto themselves, that is, the act of working on them should be fulfilling. If the end goal is the only motivation, it is really easy to lose interest during the long slog and do something else.
The other big thing is separating work time from the rest of my life. I have a set routine. I start and end at basically the same times every day.
Finally, I keep track of todo items for my project. When I am not feeling motivated, it is a lot easier to say I’ll do this one small todo task here or that one task there, which makes things manageable.
The cup's context in the talk is that J Blow is discussing how technology is not guaranteed to increase over time, but that we regress as well. The Lycurgus Cup was created in Roman times and showed extremely high levels of technology and skill, to a degree that could only have been achieved by repeated refinement. This technology was then lost.
I work on self-driving flying cars, and I can assure you that the R&D phase includes work that _is_ as glamorous as it sounds, but there are also a ton of i's to dot and t's to cross. The very nature of building a usable product requires reliability, and reliability requires extensively covering all cases. Systematically going through everything is a lot of work.
The best advice that I can give is to seek joy in solving the "industry" version of your project (the joy of producing a truly reliable product - reliability is really hard), and/or to pursue your passions in side projects.
This appendix chapter of Algorithms for Decision covers deep learning techniques in a concise but encompassing 17 pages. Algorithms for Decision Making provides a broad introduction to algorithms for decision making under uncertainty and covers a wide variety of topics related to decision making, introducing the underlying mathematical problem formulations and the algorithms for solving them. It is available online for free, but the authors seek to also publish with MIT Press.
It would be very interesting to see proper MCTS applied to 2048. Monte Carlo Tree Search performs well on problems with linear reward. 2048 has exponential reward, with subsequent states having much higher tile scores. This nonlinearity tends to cause MCTS to fall into traps - one rollout will find a state evaluation with a much higher score, and will thereafter bias all subsequent rollouts to that same (potentially suboptimal) path. If there are clever ways around this, such as board evaluation heuristics that scale linearly, then I'd love to see them. In practice, a minimax approach that enumerates all successor states up to a particular depth has worked best for me.
Nice and concise paper. All that changed was outputting a vector of predicted game values, as we go from having a zero-sum game with two players (which can be summarized with a single value) to a game with n players. Makes sense.
The comparison against MCTS shows strong performance from AlphaZero. Would be curious to see the performance of AlphaZero vs. number of its own rollouts - ie is the probability head output alone already encoding enough information to play well, and how deep does it have to look ahead to determine good play. Finally, for tic-tac-mo and connect 3x3, it should be possible to determine the optimal move. How much training / lookahead is required to achieve that? Does AlphaZero achieve perfect play for these games?
The paper's first listed contribution is "an independent reimplementation of DeepMind's AlphaZero algorithm". Maybe I missed it, but I don't see a link to a repo with the implementation.
Such an amazing video. Really interesting how the primes and the spirals are actually (basically) unrelated, but how it nevertheless leads to an interesting concept on the distribution of primes. Love the visualizations.
I am not a lawyer, but I for one would be thrilled to hear of any of this being used for good by anyone, whether for personal or commercial uses. I don't see anything other than the standard MIT Press protection clause behind the cover at the moment but will definitely look into it. Thanks!
Many of the algorithms would be very laborious to implement and explain in a language like C or C++. Julia has the advantage that the actual code looks very much like the pseudocode you mention. In fact, the guideline implementation for the ACAS X collision avoidance software that my prof developed, that is to become the de-facto issued by the FAA, has been produced in Julia rather than in pseudocode because it actually runs. Real code has no ambiguity. (assuming you avoid undefined behavior).
Julia is a fantastic language. Really hope it catches on.
Julia turns out to be a very nice language for problems like this. Many of the algorithms end up looking very much like the pseudocode from the original paper. We can use unicode as well, which makes a lot of the math (like gradient symbols) easier to read. We drop the type annotations to make it more legible. "Real" implementations could definitely improve what we have in terms of error checking and efficiency improvements, but we optimized for understanding and clarity.
Very often getting the figures to generate properly helped us catch mistakes in the code. I usually did this in a Jupyter notebook, because I can iterate there a lot faster than making a small in the book source and running the compilation process there.
I guess we didn't really have to get around the problem - the algorithm was of principle importance and then we had to figure out how to best produce figures for it.
The branching factor for games like Go is too high. Alpha Go Zero uses a stochastic policy to focus its attention on useful branches. This policy is optimized alongside the predicted value for each node.