808 karma · joined June 22, 2014
For example:
Deep Residual Learning for Image Recognition
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
We are doing vector arithmetic in this input space, and the outputs seem to be semantically changing according to the vector arithmetic.
I've been running convnet-benchmarks forever now, and I've been running them independently on separate personal hardware. I do this as a hobby.
I've done an apples-to-apples comparison, and my benchmark review only puts the facts forward, I dont attack them.
If you read my other social media comments, I've been pretty positive about TensorFlow, and I've even put in some groundwork to write Torch bindings for it.
Please stop spewing nonsense interpolated from like 2 super-weak data points.
Source code: https://github.com/larspars/word-rnn
Unlike what the public perception is, NVIDIA, Facebook and other companies collaborate very very closely. NVIDIA's latest R3 release has FFT-based convolutions inspired by Facebook's work, and Facebook integrated some of the optimization suggestions for fbfft that NVIDIA gave. Further optimizations on the FFT approach (like tiled-FFTs) have been suggested and prototyped at FB and carried over by NVIDIA. The relationship is more of a deep collaborative effort.
At the end of the day, we want cuDNN to win, as it is a baseline system for all software.
However, if you want to do research, you want to have open-source code, because you often want to do funky changes to the base algorithm. NVIDIA falls short on this, and we at Facebook pledged to always keep our code open source (bitten by the fact that we had to start with closed-source blackboxes for our initial FFT work).
AFAIK, Lush is not used inside Facebook labs.
You also cant clip them and get away with it, you have to either renormalize the layers to do half-precision (and live with the extra cost) or stick to full-precision. I was doing fun stuff early this year that did fixed-precision nets (8-bit/16-bit). Things get very interesting :)
http://resume.linkedinlabs.com/
and I love it. Would Purzue have a plugin to import your LinkedIn profile or Github profile for example?
http://hacksoflife.blogspot.com/2012/12/integrating-luajit-w...
Edit: to clarify, this does not mean torch cant use more than 2GB, but native lua allocations cant be more than 2GB, so ffi-based or C based allocations (which is how torch tensors are allocated) have no such limits.
I'm not saying it all boils down to just this quote, but I thought it was interesting and wanted to replay it here.