blennon

It's hard to tell since there's no context but one way I interpreted it is that he used to think it was mostly only the established synapses were changing. Instead, with this new result, perhaps he sees many more synapses being formed, not just existing synapses changing their strength.

From the article and their website, it doesn't seem like KnuEdge's compute units are spiking neurons. Spinnaker and IBM 's TruNorth using spiking neurons to compute.

The company changed names somewhat recently from Intellisis (mentioned in the article). Try searching for that.

Is Julius really state-of-the-art? Looks like they use n-gram and HMMs. Those were the methods that achieve SotA 5+ years ago. My understanding is that Google and Microsoft are using end-to-end (or nearly) neural network models; these outperformed the older methods a few years ago. Not sure how CMUSphinx works under the hood.

This is HUGE in my opinion. Prior to this, in order to get near state-of-the-art speech recognition in your system/application you either had to have/hire expertise to build your own or pay Nuance a significant amount of money to use theirs. Nuance has always been a "big bad" company in my mind. If I recall correctly, they've sued many of their smaller competitors out of existence and only do expensive enterprise deals. I'm glad their near monopoly is coming to an end.

I think Google's API will usher in a lot of new innovative applications.

The author is wrong in stating, "yet neuroscientists have not scrutinized their brains for one simple reason: They don’t have a neocortex."

Birds have an analogous structure to the mammalian neocortex [1].

[1] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2906560/

I'm not sure why this comment is being down voted. This comment is spot on. As a neuroscientist reading this article I asked the same question.

It is so easy in clinical research to be fooled by randomness and the placebo effect.

I believe you are mistaken.

Adult neurogenesis has only been shown to occur in three brain regions as far as I know: in the hippocampus (declarative memory and learning)[1], the olfactory bulb (smell)[1] and the striatum (procedural memory and learning) [2].

Adult hippocampal neurogenesis (AHN) has been linked to exercise. The conceptual leap which requires empirical validation is that exercise induced neurogenesis in the hippocampus indeed has a positive effect on memory. Maybe those experiments exist.

[1] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3106107/ [2] http://www.ncbi.nlm.nih.gov/pubmed/24561062

Can someone explain the rationale for leaking this? I have been a strong critic of NSA domestic surveillance, but what is described in the article seems to be the appropriate use of the U.S. intelligence apparatus.

Really?

"U.S. Transportation Secretary Anthony Foxx said the president's 2017 budget proposal includes $4 billion over 10 years for pilot projects, including a program to test self-driving cars by investing in technologically connected roads. He provided few other details on how the money would be spent if it comes to fruition."

That sounds to me like they are try to pick technologies to invest in. If that's the use of funds, I think it's a waste. Let the market choose the best tech.

No! Humanity has not fundamentally changed. We are still made of flesh and blood, we still have the same cognitive biases, we are still fighting religious wars, we still feel jealously, greed, fear and yearn for power. In that sense, nothing has changed. The only shot we have is learning from history.

When I talk to friends about the existential threats to our country this is the other one I think of. Terrorism doesn't even make the list.

I'm totally with you on our country becoming more of an oligarchy. I think we're talking shades of grey here. Today, things aren't good. The trend also isn't in our favor. To paraphrase Robert Reich, money and power go hand in hand, more money buys more power and more power alters the rules of the game in favor of those in power. Sadly, elected officials aren't bought and the moneyed interests have undue influence over the legislative process. This is one of the most important issues of today and I wish more people realized this. The only popular presidential candidate who is talking about this issue is Bernie Sanders.

I agree with the author wholeheartedly, except for the mathematical analysis at the end.

I believe one of the existential threats to our country is becoming a dictatorship. I'm always shocked when I share this point of view with people around me and their reaction is basically, "that's so unlikely it's not worth worrying about". There are many examples of this throughout history, e.g. the Roman Republic, the Weimar Republic, those that the author cites, etc. We should not be so arrogant to think that we are somehow exceptional and immune to this.

My personal belief is that there are certain constants in the dynamics of human nature, and one of them is seek power. Broad, unchecked domestic surveillance is a dangerous tool to give to a potential dictator. We should not set up a framework where a power-seeking individual can circumvent our democracy through coercion.

An interesting and early (1992) demonstration of deep nets learning latent structure in the visual domain was presented by Usui et al. They presented a deep autoencoder with the spectral values of color chips and show that the hidden layer representation corresponds to the Munsell color space where each dimension corresponds to a psychological/intuitive color attribute. Pretty amazing. Even cooler that they showed this in 1992.

10.1364/JOSAA.9.000516 Shiro Usui, Shigeki Nakauchi, and Masae Nakano, "Reconstruction of Munsell color space by a five-layer neural network," J. Opt. Soc. Am. A 9, 516-520 (1992)

In one sense we did know about the capabilities of neurons, or at least made similar predictions. How all of the ingredients come together to achieve this is not understood.

The computational properties of neurons are massively complex, e.g. local filtering in the dendrites, attenuation of depolarizations down the dendrite, the effect of local excitatory and inhibitory inputs, just to name a few. The authors of the manuscript present their interpretation of some of the these properties to achieve the goal of spatio-temporal pattern recognition by neurons (I haven't read past the manuscript yet). Others have done the same.

The figures are at the very end of the document. This is a common way of preparing and submitting manuscripts to peer reviewed journals.

It is surprising and comments like this always pop-up in response to comments like mine. The question is how much information do you really need to identify individuals? I can't answer that since I haven't seen the data set they are actually working with. In terms of the information they report, I doubt you could identify anyone.

There are good arguments to be made about your point, but I don't think this is necessarily one of them. As the article quotes, "The data is anonymous, but it allows us to see where those folks have gotten care."

What Amino has done can easily be done with anonymized data. They only need to know patient age, patient gender, physician, location of physician, what the physician billed for (I assume the ICD-9 codes). I doubt there's enough information there to identify individuals.

I agree and disagree. I would definitely start with a simpler algorithm like logistic regression or even trees and use temporal features like power at different frequencies by taking an FFT. I would even add time lagged features. After that I'd graduate to a single hidden layer MLP.

If I had tons of data (a lot of different people using it), I'd experiment with LSTM neural networks because I think the temporal information is crucial for determining a movement.

I think a killer app would be aimed at weight lifters. If you go to the gym, the serious weight lifters record their reps and weight for each exercise. The app would utilize the iWatch or some other wearable and detect the exercise and count the reps. Then it would prompt the user for how much weight they used.

I agree with you in terms of approach that AI will emerge first from brain emulation. I disagree with you on the timeline. I know you say 'starting around 2030', but I think that's a little ambitious.

While I'm an AI/machine learning practitioner now, my recent Ph.D. work was on computational modeling of the nervous system; namely the cerebellum. The reason I say 2030 is ambitious, is because there are still a lot of unknowns to perform whole brain simulation. To start, we need whole brain connectivity or wiring diagrams at an extremely detailed level. There are some efforts that are part of the BRAIN initiative that are taking a stab at this, but I don't think they're be ready by 2030. Second, you have to understand the physiology of these neurons in order to simulate them. This is incredibly complex and poorly understand. While we understand neuronal physiology in general, there are a great many details that vary by cell type. Additionally, you have to capture neuron morphology, synaptic plasticity, the effect of neuromodulators, ... the list goes on. By capture, I mean understand them well enough to describe them mathematically so that they can be simulated computationally.

Until then, traditional machine learning and artificial neural networks will be increasingly useful and interesting.

I think a similar problem needs to be solved in the language generation neural networks. While generated sentences often are formed syntactically correct, they don't make a lot of sense -- there's no over arching thought that it is trying to express. This is even more apparent when you read the subsequent sentences generated. It's clear there's no conceptual goal it's trying to convey.

How is that not a neural network? This system is basically exploiting the property of action potential propagation down the axon to capture relative timing differences.

As an aside, there are a number of useful properties seen in biological neural networks that aren't yet incorporated into multi-layer perceptrons. E.g. short-term plasticity, axonal delays, spiking neurons, etc. I expect that some of these will find their way into the MLP formulation when we can figure out an effective mathematical way of doing so.

Good point about dropout. Do you know of any quantitative studies that address this specifically?

Indeed, another weakness is that multi-layer perceptrons and their deep counterparts are susceptible to "catastrophic forgetting" -- training away old memories when new training data is presented. This is one reason why it's necessary to repeatedly run through the training data. This is a result of the way learning works in these neural networks. Like you say, humans have the ability to learn associatively in one-shot, a very different learning method.

There are neural networks like Grossberg's Adaptive Resonance Theory (ART) which do not suffer from this flaw.

Here is a link to the original research behind this PR: http://advances.sciencemag.org/content/1/1/e1400121

> From what I can see, CFS definitely can be cured by "some sort of emotional technique". It's just that a lot of CFS patients simply don't want to believe it, so they essentially end up doing the exact opposite of what is required to recover, resulting in further illness rather than recovery.

How much of this is due to your personal experience vs survey of the data? "Cure" is a very powerful word. There are several examples of CBT providing beneficial results, but nowhere near complete cures as far as I know. It's important to separate our personal experience from the data about a disease with possibly heterogeneous patient population.

For the record, I have no doubt psychological state has an effect on ME/CFS symptom severity.

Can you elaborate on your personal protocol?

There is some evidence for viral markers of CFS [1-3].

[1] http://journals.plos.org/plosone/article?id=10.1371/journal.... [2] http://www.ncbi.nlm.nih.gov/pubmed/?term=15113035 [3] http://journals.plos.org/plosone/article?id=10.1371/journal....

There are some opinions expressed in the comments here that are not consistent with a consensus report recently published by the Institute of Medicine [1].

The committee argues that, "ME/CFS is a serious, chronic, complex, multisystem disease that frequently and dramatically limits the activities of affected patients. In its most severe form, this disease can consume the lives of those whom it afflicts. It is 'real.' It is not appropriate to dismiss these patients by saying, 'I am chronically fatigued, too'".

Theodore Ganiats, one of the committee members wrote, "The literature review found sufficient evidence that ME/CFS is a disease with a physiologic basis. It is not, as many clinicians believe, a psychological problem that should not be taken seriously" [2].

The committee proposed new diagnostic criteria and a new name "systemic exertion intolerance disease".

In my opinion, getting past the psychological and psychosomatic theories of CFS is a huge step towards meaningful research funding to address this disease. The IOM publication [1] and committee members efforts to disseminate the findings of this report [2,3] to the medical community is a big step forward.

[1] http://www.iom.edu/Reports/2015/ME-CFS.aspx [2] http://annals.org/article.aspx?articleid=2118972 [3] http://jama.jamanetwork.com/article.aspx?articleid=2118591

Martin Pall hypothesizes that sauna therapy increases the availability of tetrahydrobiopterin (BH4) through multiple mechanisms [1]. A small study with people with chronic fatigue syndrome provides evidence to confirm this [2] (I don't think this publication is peer reviewed). I believe there is an association between BH4 and cardiovascular health, but this is not my field.

[1] http://www.ncbi.nlm.nih.gov/pubmed/19581054 [2] http://www.townsendletter.com/Nov2013/sauna1113.html