Teodolfo

If these values really meant anything, then Anthropic should stop working with Palantir entirely given their work with ICE, domestic surveilance, and other objectionable activities.

PyTorch was partly inspired by the python Autograd library (circa 2015 [1]) to the point where they called their autodiff [2] system "autograd" [3]. Jax is the direct successor of the Autograd library and several of the Autograd developers work on Jax to this day. Of course, for that matter, PyTorch author Adam Paszke is currently on the JAX team and seems to work on JAX and Dex these days.

[1] https://pypi.org/project/autograd/#history

[2] https://www.cs.toronto.edu/~rgrosse/courses/csc421_2019/read...

[2] https://web.archive.org/web/20170422051747/http://pytorch.or...

Direct indexing is WAY more valuable to US citizens living in the EU than US citizens living in the USA because of the painful intersection of MiFID II rules and US tax law (PFIC tax cancer makes buying EU domiciled funds a non-starter). Brokers will not sell US domiciled ETFs to US citizens living in the EU unless they can opt out of the consumer disclosure rules (e.g. by becoming an elective professional client of their broker under MiFID II rules). So these 2 million US expats have no choice but to manage a portfolio of individual stocks or pay exorbitant AUM fees. The first half-way decent direct indexing product that accepts US expats residing in the EU will make a killing!

Igor Markov, along with Sat Chatterjee, seem to be pursuing a bizarre vendetta (after Sat failed to take over their project) against the lead authors of the chip placement work, not some sort of intellectually honest critique.

This was covered previously in the press and on social media, with statements from a variety of prominent researchers (e.g. [1][2][3]).

The code is even available for the Nature paper's method, along with an FAQ: https://github.com/google-research/circuit_training#FAQ

[1] https://twitter.com/ZoubinGhahrama1/status/15122035096467415...

[2] https://twitter.com/JacobSteinhardt/status/15215993404137881...

[3] https://twitter.com/sguada/status/1521587406385807361

"The AlexNet paper is generally recognized as the paper that sparked the field of Deep Learning"

Uh not really. That would have been "A fast learning algorithm for deep belief nets" in 2006.

Also weird how this list completely ignores speech recognition. Deep learning's success in speech recognition predates AlexNet and motivated Google to create TPUs [1], and, more generally, invest in deep learning.

[1] https://www.wired.com/video/watch/the-story-behind-google-s-...

Going from 25% top-one error to 22% top-one error is a massive jump on ImageNet and very meaningful in a lot of applications. That said, there is no reason to believe attention-based models are the only way forward on image classification. The humble ResNet-50 can get near 22% top-1 error when trained properly.

Whether we need attention or not is a more interesting question on seq2seq models on text data.

Julia provides no meaningful advantages. PyTorch and JAX are too good. For typical deep learning workloads, Julia will not easily have a speed advantage. Everything goes down to the same cuDNN kernels anyway.

Julia seems like an attempt at a better matlab, but the machine learning world moved to python first.

(Also 1-based indexing is almost as obnoxious as the 24/7 Julia shill brigade.)

In some unit systems (https://en.wikipedia.org/wiki/Geometrized_unit_system) they are both units of length.

Do you happen to have links to the papers?

Edit: Looks like https://www.sciencedirect.com/science/article/abs/pii/S17505... and https://www.sciencedirect.com/science/article/abs/pii/S17505...

There are plenty of free, open access journals that are reputable.

JMLR http://www.jmlr.org/ is quite successful. There are some fields, such as machine learning, that are not dominated by for-profit journals. Why is this possible in some fields and not others? My answer would be that it is possible in all fields, but incumbency advantages can be very strong and coordination across academic volunteers can be more difficult when they are individually less secure.

Execution quality matters and given enough order flow a they can still match client orders to make a profit without giving a worse price than NBBO.

The guide suggests gmail and two factor, but why not also https://landing.google.com/advancedprotection/ ?

In 2009 NVIDIA wouldn't give Geoff Hinton's group the time of day, let alone a single free GPU to try out to see if they wanted to buy more. Geoff Hinton spoke at a NIPS keynote in 2009 and told 1000 people in the audience to all go out and buy a specific NVIDIA card if they wanted to do work in this area and still NVIDIA didn't bat an eye.

I heard there were layoffs of engineers there. But obviously unrelated. OpenAI will continue unchanged.

I'm sure many AI experts at these places view it negatively. These are not monolithic organizations, but massive companies filled with real people with varying opinions.

The singularity cult is real. The geek rapture will occur when they blog about someone else making God in a Box hard enough to drown out the normal charlatans. Please fund their blogging, it is the most impactful way you can spend your money.

Not if you use Lyft.

Also perhaps machine learning experts are paid well enough in industry research labs in tech companies that the difference isn't perceived as that large given the diminishing marginal utility of money.

How much do they pay? And how is the pay structured?

I would love to read an approach to effective political giving. Does anyone know of such a document? If the effective altruism movement had focussed on political action and donations instead of private charity to supply bed nets we might have a much better world (including money for bed nets).

Jürgen maintains a lot of sock-puppet accounts (even on wikipedia and amazon reviews). Let's see if any of them show up in this thread. He is a perfectly good researcher and his students have done a lot of excellent work, but I don't approve of his choices to use an army of sock-puppets or his (I will try to put this charitably) somewhat non-standard ideas on how academic credit should be apportioned.

Who promised the renaissance?! We should put them in the stocks and throw overripe fruit at them!

It is probably hard for companies like skymind with no recognized experts. Getting a big name on board isn't always possible however. And investors are right to be leery given the number of people branding themselves as deep learning experts without any credible track record of results.

Amusing trivia: Peter Brown, co-CEO of Renaissance Technologies, got his PhD under the supervision of Geoffrey Hinton.

More like 2006, although there were convnets and other neural nets working well long before then of course. It just took several years for the techniques from the papers in the scientific rivial to make their way into products. The vision community was quite resistant to it, even more than the speech community.

You seem very angry about my comment. I'm not sure why. Roughly 14% of NIPS attendees are women and OpenAI seems to be doing worse than that very low threshold for its research staff. If they want the best candidates, they need to work hard to erode systemic biases and patriarchal power structures that turn away good candidates. The longer this goes on, the harder it will be to fix since the senior people in the organization will mostly be early hires and candidates will start to find the problem more and more visible.

I wish OpenAI would work harder to hire more women on their research staff specifically.

You can read the terms of service here: https://cloud.google.com/terms/

They include a deprecation policy.

Probably a fair bit more than 350k.

Kaldi + voxforge http://kaldi.sourceforge.net/about.html http://www.voxforge.org/