Hacker News Reader

Comment by ot | Hacker News Reader

[Note: this is a repost from https://news.ycombinator.com/item?id=7927077. The original set off the voting ring detector and was demoted]

I came across about Torch in this FB comment [1] by Yann LeCun:

    I have moved to Torch7. My NYU lab uses Torch7, Facebook AI
    Research uses Torch7, DeepMind and other folks at Google use
    Torch7.

Apparently it is used in many deep learning labs. I think that in Toronto they mostly used Matlab and Python, does anybody know if this is still true?

[1] https://www.facebook.com/yann.lecun/posts/10152077631217143?...

View on HN

I am one of the maintainers. From information first-hand, Torch is used by:

- Facebook

- Google DeepMind and slowly Google Brain is moving as well.

- Certain people at IBM

- NYU

- IDIAP

- LISA LAB (not exclusively, but some students started using it)

- Purdue e-lab

- Several smaller companies (10-100 companies)

There will definitely be several commonly asked questions, and this is my personal perspective on them.

Why torch/lua, why not python+?*

No reason. Just because. Mostly because LuaJIT is awesome (with it's quirks) and LuaJIT is extremely portable. (we embed torch routinely in tiny devices, afaik not practically possible with python).

Is Torch better than Theano/etc.etc.?

Better and worse. Every framework has it’s oddities. I like the super-simple design and the compactness of traversing from high-level easy-to-use API to bare-metal C/assembly.

Also, torch’s ecosystem was grown not with exclusively lab experiments in mind, with Yann’s strong robotics research, packages were developed with practicality in mind all the time. Custom chips are being developed for convnets (TeraDeep) and they use Torch.

Where’s the doxx???

If there’s documentation, I’ve tried to make people aware of it, mostly by consolidating everything torch to this one page: https://github.com/torch/torch7/wiki/Cheatsheet

What about Julia?

I like Julia a lot, and it’s definitely cool, but the packages for NN and GPUs aren’t very strong, so advantages over Julia are simply code that’s already written.

If there are any more questions, feel free to ask them here or just open an issue on the github package.

Thanks for reading.

Edit: Apologize for the formatting, not very good at hackernews

I'd love to try out Torch7, but I've grown accustomed to Theano's automatic differentation, which is the killer feature for me. That's the one thing keeping me from exploring the alternatives. Not having to calculate any gradients makes trying different architectures and activation functions a lot less daunting.

Are there any plans for Torch7 to support something similar? I suppose it would be nontrivial since to my knowledge Torch7 does not use symbolic representations of the computations internally, which is what enables Theano to do automatic differentation in the first place.

I'm writing a calculus package that's inspired by Julia's, which would have an automatic differentiation engine, but it is nowhere close to being finished, nor is it a priority at this point. So sadly, we wont see Sander using torch anytime soon :)

Torch is an amazing project and has been a great inspiration for my leap in to deep learning. I wish the maintainers the best of luck. It's ambitious to write a deep learning lib. That being said, you guys are doing a great job.

The comparison helps quite a bit. Mind if I lift this for our site? I'd link back to this thread as well.

Thanks!

go for it.

*Mostly because LuaJIT is awesome (with it's quirks)

Could you give some example of its quirks?

my biggest annoyance with LuaJIT is the lack of 64-bit addressing, and there's no easy fix for this. Mike Pall said he'd fix it in LuaJIT 3.0 (with an overhaul of the GC), whenever that'll be.

http://hacksoflife.blogspot.com/2012/12/integrating-luajit-w...

Edit: to clarify, this does not mean torch cant use more than 2GB, but native lua allocations cant be more than 2GB, so ffi-based or C based allocations (which is how torch tensors are allocated) have no such limits.

This is a big annoyance for me as well. I use jemalloc to make sure that my FFI allocations don't 'steal' this lower memory from my pure Lua allocations. It works great on Linux, but only so-so on OSX.

http://www.canonware.com/pipermail/jemalloc-discuss/2014-Feb...

https://gist.github.com/neomantra/9122165

Hi,

I am the lead engineer at ArrayFire[1]. Is there anyway I can get in touch with you ?

[1]http://www.arrayfire.com/docs/index.htm

Sure, the best way to get attention is to open an issue on github.com/torch/torch7 (that way you get the attention of all the core developers). If you'd like to ask something more discrete, I'd suggest you email one of the four here: https://github.com/orgs/torch/members

It may seem obvious, but I feel like the page you linked could use an occurrence of the word 'C++'. The code snippet was obvious to me, but just a thought.

Thanks for the feedback! While the code snippet is from C++, we have language wrappers for Java, R and Fortran. We'll try to be more clear about this in our documentation.

ArrayFire looks very interesting, as a novice a drop in enhancement of the nn (or nnx for dropout) package for torch would be very useful.

We are getting there. In the mean time we have an example on how you can go about building one yourself. You can see it here: https://github.com/arrayfire/arrayfire_examples/blob/master/...

Thanks for the response, currently I'm doing R&D with easier to use high level representations and if it proves fruitful will be looking at getting the most cost effective speedups in about 6 months.

Whats the license for the free edition? I would be sticking to open source code until we secure capital, but would be very interested in your commercial services down the line.

That comment from LeCun is very interesting, because this means he moved from his own language, Lush, which he maintained for many years, to Torch7. And LeCun has very high standards for expressiveness and efficiency.

Oh, dont get me started about Lush :), this is my personal opinion, but I tried to use lush for a bit, and I ran away as fast as I can. However, in retrospect, that's probably because it was the first functional programming language that I was introduced to.

Also, torch has a great set of people building the ecosystem, lots of communication and collaboration online (github, forums, across labs) which lush lacked.

Toronto and Montreal are main Python oriented labs.

Reply on news.ycombinator.com