Extreme Learning Machine | Hacker News Reader

Extreme Learning Machine

en.wikipedia.org

50 pointssytelus7 comments

"ELM" = "Extreme Learning Machine" is apparently heavily based upon previous work, if not outright plagiarism. The Wikipedia article on ELM discusses this only briefly:

https://en.wikipedia.org/wiki/Extreme_learning_machine#Contr...

Yann LeCun' opinion on ELM: 'What's so great about "Extreme Learning Machines"?':

https://www.reddit.com/r/MachineLearning/comments/34u0go/yan...

A sort-of "How-to plagiarize successfully" guide including "Easy but Proven 5 Steps to Academic Fame" with Huang's work as a gleaming example:

https://elmorigin.wixsite.com/originofelm

Two very detailed plagiarism analyses of two of Huang's papers. They are quite sarcastic, funny and enlightening:

https://docs.wixstatic.com/ugd/5256f1_ac3a0f6b08524a3ca6101d...

And finally, as one poster says, the provenance of "Deep Learning" itself has some controversy:

https://www.researchgate.net/post/Why_Extreme_Learning_machi...

"But, just recently I also found that there is also a problem with the inventorship of Deep Learning. http://people.idsia.ch/~juergen/deep-learning-conspiracy.htm... For this, I post a question for discussing these issues. https://www.researchgate.net/post/Any_enlightment_on_the_ori... https://docs.wixstatic.com/ugd/5256f1_556b42c00d0d4d5bb199fb...

As much as people dislike Schumidhuber for being bit of a curmudgeon, his contributions have been undermined in the Deep Learning community at large. He did not deserve to be shafted for the Turing award.
His contributions were as impressive as the other 3, and a lot of it was done in isolation away from the support of the NA AI/ML community.

ELMs are just neural nets with random hidden weights. This idea has been around since the 60's before being rebranded to get unoriginal papers published.
This is equivalent to a low-rank Fourier approximation of Gaussian process regression (also known as random features) for some kernel. Random kitchen sink approximations are similar, but better performing since the weight value distribution actually has theoretical justification. RKS is a really great way to use approximate Gaussian process regression on larger datasets.

I really enjoyed the discussion here [0] on Yann LeCun’s comments.
In some ways, it’s like random Fourier features/kitchen sinks (Cf. Ben Recht). They’re more insightful than they are useful.
[0] https://www.reddit.com/r/MachineLearning/comments/34u0go/yan...

Is this an example of reservoir computing?

The two first sentences of the respective Wikipedia article answer that:
https://en.wikipedia.org/wiki/Extreme_learning_machine "Extreme learning machines are feedforward neural networks"
https://en.wikipedia.org/wiki/Reservoir_computing "The reservoir consists of a collection of recurrently connected units"
So, no.

The ideas seem quite relate. A common reservoir computing setup involves learning a linear map from a back box dynamical system (which can be a feedforward network if we really want it to) to some output. The only significant distinction I see from a short observation is that the input and output size in reservoir models are the same from what I’ve seen

Reply on news.ycombinator.com