Hacker News Reader

mannigfaltigon A New Twist on Neural Networks

In their second publication they actually use the probabilistic unit, but computing those requires running an EM algorithm for each layer of capsules: https://openreview.net/forum?id=HJWLfGWRb

mannigfaltigon 'We can't compete': why universities are losing their best AI scientists

I doubt this will be a real problem because I've seen brilliant people excel in provincial universities with very little means and talent around. Internet education has gotten pretty good and probably reflects most of the experience of studying at a top university with the best researchers in the field. People might be 5% worse prepared. Nothing to be concerned about.

mannigfaltigon RCN is much more data efficient than traditional Deep Neural Networks

I still find it incredibly hard to tell whether this is overblown hype or legit scientific progress. There is no indication whatsoever that this approach scales to deep feature hierarchies and that is likely what you need to compete on hard tasks like classification on ImageNet. Given the amount of money at play (several hundred millions of dollars), writing 70 pages, making code publishable is certainly an obvious way to get the most out of the hype.

mannigfaltigon The quest to evolve neural networks through evolutionary algorithms

My guess would be that symmetry is a simple heuristic measure of physical fitness. Visual attraction is basically a strong regularizer that restricts the search space to phenotypes with particular traits. Asymmetry means that the joints wear out more quickly and muscles might not coordinate optimally leading to less strength and a reduced ability to hunt and to fight predators. AFAIK it is also a quite robust predictor of all kinds of diseases because it often means that the growth signalling is out of tune throughout the system. Visual selection basically performs environmental selection more immediately and more effectively: an asymmetric person might still survive, but its offspring has a lower overall chance to survive. The teaching signals of that are much weaker.

mannigfaltigon Record surge in atmospheric CO2 seen in 2016

The problem is that anxiety is warranted in this case. You can’t do anything about solving the problem for everybody, but you can potentially save your own life:

- Work like a madman to get into the 1%.

- Move to a region that is self-sustaining and is little impacted by global warming.

- Buy guns, tools and a farm.

mannigfaltigon The quest to evolve neural networks through evolutionary algorithms

> evolution of evolvability and then a seemingly unrelated subject: the evolution of robustness.

Isn't evolvability only about robustness? What other criteria would improve evolution? Mutations just happen, so the question is how well the phenotype can deal with these mutations. If it can incorporate mutations well, it can tunnel to different useful phenotypes and therefore is robust.

mannigfaltigon The quest to evolve neural networks through evolutionary algorithms

I think, the main effect of sexual reproduction is that, much like GANs and competitive self-play, it creates species-internal competition: Both sexes need to impress, which makes cheating an obvious strategy (makeup, steroids, Shakespeare quotes, LISP etc., but many such examples can be found in the animal world), and hence both sexes also need to be able to detect cheating. Some species are rather asymmetric in that regard. For example, in humans it is mainly women who attract (they masquerade as fruits [make up is likely a cross-cultural phenomenon; and, well, breasts] tapping into male food gathering circuitry); men compete in hierarchies trying to impress and women select men from the top of the hierarchy. Complex dynamics emerging from this likely lead to the immense growth of the human cortex.

Sexual reproduction basically outsources some of the selection effort to the cognitive apparatus of the species itself, thereby introducing a massive amount of additional selection signals (mainly by the much increased necessity to model other minds, namely minds of the opposite sex). Many of these signals promote traits that are useful for survival (mainly intelligence and health).

mannigfaltigon One pixel attack for deceiving deep neural networks

That seems like a strange/unnecessary way to put it because DO is noise.

mannigfaltigon One pixel attack for deceiving deep neural networks

You can also call it just subgraph; not all of them, but exponentially many.

mannigfaltigon One pixel attack for deceiving deep neural networks

Wait, that can’t be wrong because that is literally what DO does. It is a convex hull regularizer around the network activations using noise. That is also why dropout does not solve susceptibility to adversarial examples: It merely extends the regions that the NN generalizes to outward; but that is limited because high-dimensional spaces are counter-intuitively large and the noise required to cover a descent fraction of the “unmapped” space would completely prevent learning. AFAIK, Yarin Gal merely provides a Bayesian interpretation of the noise.

mannigfaltigon Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

Actually, GANs reach state of the art in anomaly/outlier detection and drug/molecule prediction, so there is certainly more to it than just artistic applications:

https://openreview.net/forum?id=S1EfylZ0Z

https://www.ncbi.nlm.nih.gov/pubmed/28703000

http://pubs.acs.org/doi/abs/10.1021/acs.molpharmaceut.7b0034...

mannigfaltigon China temporarily shuts down thousands of factories in pollution crackdown

Totalitarian systems are quicker to come up with the rules (in this case for self-preservation). And when time matters, then it has obvious advantages.

mannigfaltigon Sci-Hub: Public access to research papers

How is Assange Putin’s mouthpiece?

mannigfaltigon A Theory of Sequence Memory in Neocortex

Small correction: Learning algorithms that determine the contribution of a unit to the result over time can also be local in time, namely by tracking the contributions online (for example by eligibility traces or RTRL).

mannigfaltigon A Theory of Sequence Memory in Neocortex

> By firing earlier it inhibits neighboring cells, creating highly sparse patterns of activity for correctly predicted inputs.

This part is so vague. It seems to lack an explanation of how interneurons inhibit other neurons nearby. Also, wouldn‘t sparsity even occur without the early firing enabled by distal pattern matching?

> When relatively few neurons are active relative to the population, then such pattern recognition is robust.

Why?

mannigfaltigon Iceland's attempts to replant its forests

The same actually happened in Spain 400 years ago. There used to be a lot of forests in Spain: http://m.dw.com/en/spain-replants-after-centuries-of-defores...

mannigfaltigon AlphaGo Zero: Learning from scratch

I guess deep world models are still severely riddled by all sorts of problems: vanishing gradients, BPTT being O(T), poor generalization ability of NNs (which likely is due to the lack of attractor state associative recall, as well as concept composability), lack of probabilistic message passing to deal with uncertainty, and perhaps some priors about the world are necessary to make learning tractable (such as spatial maps and fine-tuning for time scales that contain interesting information).

mannigfaltigon AlphaGo Zero: Learning from scratch

It also presumes that one can simulate the world at low cost. In AlphaGo Zero it takes 0.4 s for 1.600 node extensions, but in this case the cost of the world is negligible. Anyway, assuming you need that many node extensions to get decent quality updates, that puts a rather a tight limit on the cost of simulating the world.

mannigfaltigon AlphaGo Zero: Learning from scratch

> It's an uninteresting reduction since linear algebra can describe almost everything.

The question is whether it can do so efficiently. As far as I know, alternating applications of affine transforms and non-linearities are not so useful for some computations that are known to occur in the brain such as routing, spatio-temporal clustering, frequency filtering, high-dimensional temporal states per neuron etc.

mannigfaltigon Facebook acquires anonymous teen compliment app tbh, will let it run

Eww.

mannigfaltigon How to Set Up a Company in Germany for English-Speaking People

[deleted]

mannigfaltigon WaveNet launches in the Google Assistant

I am wondering what their baseline is. They call it "Current Best Non-WaveNet". Quite frankly, Apple's most recent deep learning-based speech synthesis sounds superior, but there aren't enough samples to for a proper comparison: https://machinelearning.apple.com/2017/08/06/siri-voices.htm...

mannigfaltigon Interactive Vim tutorial

Exactly, often changes happen so quickly that you cannot really tell what happened. You need to try it a few times to get used to the concept.

mannigfaltigon The hippocampus as a 'predictive map'

What are your thoughts on this paper?

mannigfaltigon The hippocampus as a 'predictive map'

> Despite the name 'neuronal networks' that we have history to thank for, it's better think of them as just long compositions of functions with a lot of parameters one can tweak.

Actually, the name is not so misleading because `ReLU(Wx + b)` is an approximation of the response of a large ensemble of integrate-and-fire neurons in terms of the firing rate [1] (it models populations of pyramidal neurons in the cortex). The ReLU activation function basically approximates a logarithmic activation function [2]. Obviously many kinds of computations are left out in this model, for example in pyramidal neurons, the integration of the incoming spikes is often sublinear rather than linear; the branches of the neurons that receive signals (dendrites) can themselves perform complex computations such as spatial clustering and logical operators basically sending small (dendritic) spikes forward and backward within an neuron [3]; short-term plasticity is left out entirely and so are countless types of interneurons which inhibit other neurons nearby and measure mean activity across populations of neurons etc.

[1] https://web.stanford.edu/group/brainsinsilicon/documents/Won...

[2] https://www.utc.fr/~bordesan/dokuwiki/_media/en/glorot10nips...

[3] https://neurophysics.ucsd.edu/courses/physics_171/annurev.ne...

mannigfaltigon The decline of Turkish schools

I'm wondering what role brain-drain plays in this catastrophe. Who wants to live at the border of chaos? Smart and open-minded people emigrate to Central/Northern Europe, US/Canada or Australia to live a much safer and fulfilling life. What remains is the conservative personality type (high dutifulness, low openness to experience, tight integration in communities), who is much more likely to reject democratic ideals and to accept theocratic dictatorships.

One might conclude that the only way to reliably stabilize such regions is to massively incentivize them for the personality types that have emigrated over the course of the past decades or centuries, even.

mannigfaltigon Tree Style Tabs Extension Ported to Firefox 57

I've added a modification that removes any margin at the top:

    #TabsToolbar, #sidebar-header {
      visibility: collapse !important;
    }
    
    #TabsToolbar {
      margin-bottom: -21px !important;
    }

Then go to the hamburger menu > Customization and check "Title Bar" at the bottom left. Then it pretty much looks like FF 56.

I've also changed the style of TSTs in the add-on settings (all the way at the bottom; not in userChrome.css) such that the tabs don't take up as much vertical space and to hide the close box button (and only show it on the mouse-over event):

    .closebox {
      display: none
    }
    
    .tab:hover .closebox {
      display: block;
    }
    
    .tab {
      padding-top: 0px;
      padding-bottom: 0px;
    }

You can make `.tab` even tighter:

    .tab .icon {
      padding-top: 2px; /* reducing it by 2px */
      padding-bottom: 2px;
    }
    
    :root {
      --tab-height: 22px; /* reducing it by 4px */
    }

mannigfaltigon Firefox Quantum Lands in Beta, Developer Edition

A better modification is this:

    #TabsToolbar > * {
      visibility: collapse;
    }
    
    #TabsToolbar {
      margin-bottom: -21px !important;
    }

You can also re-enable the title bar under hamburger menu > Customize > "Title Bar" at the bottom left.

mannigfaltigon Firefox Quantum Lands in Beta, Developer Edition

It works now on Firefox Developer Edition 57.0b3 (64-bit), macOS High Sierra. It took me a while to figure out that one needs to activate it by clicking one of the icons at the top.

Edit: This was a really bad idea because upgrading to 57 destroyed my sessionstore.js. None of the backups were up-to-date for some reason. Be careful and make backups!

mannigfaltigon Firefox Quantum Lands in Beta, Developer Edition

> set `xpinstall.signatures.required` to false

This seems to only work in the Nightly and Developer channel, not with the Beta channel. Is that correct?