HNHacker News
TopNewBestAskShowJobs

smhx

808 karma · joined June 22, 2014

submissionscomments
smhx··on PyTorch 1.0 is out
we are fixing this shortly. it was an oversight.

Edit: fixed links should be going live in a few mins, via: https://github.com/pytorch/pytorch.github.io/pull/141

smhx··on PyTorch 1.0 is out
Yes, it is called Chainer and it's quite competitive.
smhx··on PyTorch 1.0 is out
training and inference.
smhx··on PyTorch 1.0 is out
in the release notes, there's a section called "Breaking Changes". There are no major breaking changes from 0.4.1, and all your code should largely work as-is.
smhx··on Group Normalization
available in PyTorch master via: https://github.com/pytorch/pytorch/pull/5968
smhx··on Tensor Comprehensions
Tensor Comprehensions are not a deep learning framework, but a way to write new optimized operators for deep learning. Hence, Tensor Comprehensions can be integrated with TensorFlow.
smhx··on The cpu_features library
an alternative library [cpuinfo](https://github.com/Maratyszcza/cpuinfo) is a similar offering but also additionally exposes more information such as cache sizes, topology information (number of sockets, etc.) and information on attached integrated GPU.
smhx··on Making Pillow-SIMD, optimizing image processing in Python
I am the lead devel of pytorch. No you dont need to install CUDA to get pytorch to work on GPUs, we ship the relevant bits with our binaries. All you need is a working NVIDIA driver.
smhx··on Making Pillow-SIMD, optimizing image processing in Python
you dont need to install CUDA or CuDNN to install pytorch. We package and ship all required dependencies with our binaries.
smhx··on A New Twist on Neural Networks
The article does not mention the first and second authors of the research work, which is an atrocious thing to do.

The paper was authored by Sara Sabour, Nicholas Frosst, Geoffrey E Hinton in that order.

smhx··on Caffe2: Open Source Cross-Platform Machine Learning Tools
http://caffe2.ai/docs/caffe-migration.html#null__how-is-caff...

> Caffe2 is built to excel at mobile and at large scale deployments. While it is new in Caffe2 to support multi-GPU, bringing Torch and Caffe2 together with the same level of GPU support, Caffe2 is built to excel at utilizing both multiple GPUs on a single-host and multiple hosts with GPUs. PyTorch is great for research, experimentation and trying out exotic neural networks, while Caffe2 is headed towards supporting more industrial-strength applications with a heavy focus on mobile. This is not to say that PyTorch doesn’t do mobile or doesn’t scale or that you can’t use Caffe2 with some awesome new paradigm of neural network, we’re just highlighting some of the current characteristics and directions for these two projects. We plan to have plenty of interoperability and methods of converting back and forth so you can experience the best of both worlds.

smhx··on Why Slack is inappropriate for open source communications
thanks for the pointer. I started using this today for pytorch
smhx··on Introducing Keras 2
as core-devs, we dont plan to build-in something like Edward. However, folks in the community are brewing something:

https://discuss.pytorch.org/t/bayesian-computation-in-pytorc... https://discuss.pytorch.org/t/distribution-implementations/4...

smhx··on Introducing Keras 2
Currently not many frameworks have actual fusion of kernels (to avoid launching many GPU kernels). If you look underneath a theano.scan or TF.scan, GPU kernels are still being launched individually (but are likely stream-overlapped where appropriate).

With TF's XLA compiler, they are slowly getting towards kernel fusion, which will then reduce launch overheads.

We have similar things in the works for pytorch: to quickly JIT at runtime the dynamic graph that is getting executed. More news on this will come when time-appropriate.

smhx··on PyTorch – Tensors and Dynamic neural networks in Python
the biggest hindrance to do this are language constructs that cannot be or are inconveniently expressed in the symbolic graph, such as python's if vs tf.if and for vs theano.scan, or conditioning on some python-code (not tensor operations). So to build an eagerly evaluating symbolic graph framework that is allowed to do arbitrary things would mean that you would (to an extent) reimplement the language you are working with.
smhx··on PyTorch – Tensors and Dynamic neural networks in Python
we actually share the same git-subtree between Lua and Python variants. TH, THNN, THC, THCUNN are shared.
smhx··on PyTorch – Tensors and Dynamic neural networks in Python
it's 0-indexed just like everything else in python
smhx··on PyTorch – Tensors and Dynamic neural networks in Python
the C libraries are compatible with Windows, they are used in Torch windows ports. We just dont have any Windows devs on the project to help and maintain it :( .
smhx··on PyTorch – Tensors and Dynamic neural networks in Python
it's the same license file as https://github.com/torch/torch7 and http://torch.ch

The C libraries are shared among the Lua and Python variants

smhx··on PyTorch – Tensors and Dynamic neural networks in Python
It's a community-driven project, a Python take of Torch http://torch.ch/. Several folks involved in development and use so far (a non-exhaustive list):

* Facebook * Twitter * NVIDIA * SalesForce * ParisTech * CMU * Digital Reasoning * INRIA * ENS

The maintainers work at Facebook AI Research

smhx··on Optimized-Torch – Torch installation in a self-contained folder
I'm enthusiastic that Intel is working on optimizing Torch's Tensor and nn libraries, and optimizing Neural Nets in general.

This repo is most definitely not ready for primetime.

Two comments:

1. I haven't seen an Intel team that does non-trivial DNN optimizations, almost all the (multiple) teams that have worked on optimizing neural networks have worked on trivial stuff like adding OpenMP optimizations to existing code, and integrating MKL into existing frameworks. Usually adding OpenMP optimizations isn't as relevant, because one uses CPUs for neuralnet inference, on a server where multiple processes are running. Per-core efficiency is much more important than cute threadpool optimizations like OpenMP which look great in micro-benchmarks.

The meaty and practically relevant optimizations have come from folks in the community like Andrew Lavin and Marat Dukhan ( https://github.com/Maratyszcza/NNPACK ) which are focused more on optimizing both single-core efficiency as well as multi-core.

2. The way this fork integrates MKLDNN into Torch is not great, where they add MKLDNN fields into the core Tensor / Storage struct, which is not great from a broader perspective. I will try to reach out to the authors and make this more proper.

p.s.: torch installs in a self-contained folder already as pointed by @gcr

smhx··on FastText – Library for fast text representation and classification
Link to the paper: https://arxiv.org/abs/1607.01759

Quotes from the paper:

Both char-CNN and VDCNN are trained on a NVIDIA Tesla K40 GPU, while our models are trained on a CPU using 20 threads.

Table2 shows that methods using convolutions are several orders of magnitude slower than fastText.

Our speed-up compared to CNN based methods increases with the size of the dataset, going up to atleast a 15, 000× speed-up.

Table 2 shows the speedups of:

ConvNets: 2 to 5 days on GPUs

FastText: 52 seconds on CPU

smhx··on Why is Python slow
and that's when you dont do the optimization.
smhx··on AI, Deep Learning, and Machine Learning: A Primer [video]
The only objective response I'll give to this comment is the number of citations (as pointed out by another user) https://news.ycombinator.com/item?id=11885161 .

It's not perfect, but I cant give a reasonable answer to the other extreme of an opinion. fwiw, as a researcher I spent quite some time on this paper, but that subjective point doesn't mean anything to you.

smhx··on AI, Deep Learning, and Machine Learning: A Primer [video]
The video is almost cringe-worthy in it's factual inaccuracy around 2012.

The reigniting of deep learning around 2012 was because of Krizhevsky, Sutskever & Hinton winning the Imagenet challenge (1000 object classes)

Contrary to how much Google tried to sell Andrew Ng's "breakthrough" 2012 experiment with tons of PR, the paper is very weak, and cant be reproduced unless you do a healthy amount of hand-waving. For example, to get an unsupervised cat, you have to initialize your image close to a cat and do gradient descent wrt the input. Or else, you dont get a cat... It is not even considered a good paper, forget being breakthrough. Also, those 16000 CPU cores etc. can be reproduced with a few 2012-class GPUs and much smaller time-span than their training time.

The next slide after the 2012 breakthrough that shows the Javascript neural network -- contrary to what it looks like -- is not TensorFlow either. It has cleverly and conveniently been given TensorFlow branding, so most people just confuse it, but it's just a separate Javascript library akin to convnet.js

Since your page gets a ton of hits, it's at least worth it to publish a comment about these GLARING inaccuracies.

smhx··on Arxiv Sanity Preserver
Being in the deep learning community, the number of papers appearing has been getting out of control. Most of the papers appear on arxiv and are of low quality. This is particularly problematic right before conference deadlines.

Karpathy's Arxiv-sanity helps a lot to keep in touch with the latest and greatest deep learning without having to spend all my time reading papers.

smhx··on Deep Learning Is Going to Teach Us All the Lesson of Our Lives
there's a subtle transition between "deep learning masters very specialized skills" to "let's generalize this thought to everything possible".

I work in deep learning and I think the article is more fear than food for thought.

smhx··on Deep Neural Networks for Acoustic Modelling and Speech Recognition [pdf] (2012)
This tracks the current state of speech recognition. https://github.com/syhw/wer_are_we
smhx··on Learning Physical Intuition of Block Towers by Example
The corresponding UnrealEngine Torch plugin is released here: https://github.com/facebook/UETorch
smhx··on I Trained a Deep Q Network Built in TensorFlow to Play Atari Pong
I didn't mean it in a condescending way. I was stating that this is not algorithmically new, and is a reproduction of the paper. From the README of the repository, it is not obvious that this is a reproduction. The information is fairly hidden, and the DQN paper is given as a reference. A casual glance might suggest that there is research innovation there.

Here are other reproductions just FYI:

Original Torch version from DeepMind: [github mirror] https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner

Caffe: https://github.com/muupan/dqn-in-the-caffe

Theano: https://github.com/spragunr/deep_q_rl

Neon: https://github.com/DoctorTeeth/dqn

← PreviousPage 2 of 4Next →