Fede_V

That list is correct, but it's only half of the problem. The other other half of the problem is supply.

In fields like mathematics, physics, computer science, grad students have excellent non-academic options. There is an oversupply of PhD students with respect to the number of available positions, but the excess PhD students have fantastic career avenues.

In fields like the humanities, and, to a lesser extent, the molecular biology/ecology, there is a huge excess of PhD students, and there is a very limited number of career options outside of research (for bio, there is pharma, for everything else, eh). This is compounded by the fact that for professors, postdocs are an absolute steal (http://www.sciencemag.org/careers/2017/01/price-doing-postdo...). Professors get to pay 50-55k (and that's on the high very high end) a year for a PhD trained researcher who will often work themselves to the bone in the hope of getting the publications that will allow them to become a professor. Thus - the very people who have the power to change the system instead greatly benefit from the status quo.

The real reform that has to take place is that PhD programs need to become way more selective. Not every PhD should have a guaranteed spot as a professor (that would be insane) but when less than 8% of postdocs become professors, the system is in crisis.

With an RBF Kernel, you are stuck solving the dual problem which has O(n^2 * m) complexity. With the linear kernel, you can solve the primal scales with O(n*m) complexity.

Right, especially if you are selling it as something to maintain privacy, I'd like to see the source code.

This is incredibly awful, but not particularly surprising.

One of Paul Graham's most read essays is about the qualities that YC looks for in founders (http://paulgraham.com/founders.html). In that essay, he specifically discusses 'Naughtiness' as a positive quality, and how it's one of the most important features of potential founders.

The problem is that being clever and hacking around a system for curiosity is governed by a very different set of incentives than being clever and hacking for profit. Feynmann's safecracking at Los Alomos was done to complain about an absurd system and to play pranks on his colleagues, not to enrich himself.

Being willing to break dumb rules to show how pointless they are is an excellent public service. Breaking rules that stand in the way of your profit is completely different, and it's important to call it out.

This is a great post. However, it didn't touch on the two main problems of AD:

- Using Dual Numbers requires that all functions that you call into accept templated parameters. If you want to use GSL, BLAS, or any other mature math library, you are probably out of luck.

- Even if you are willing to port the code and modify the functions to accept templated parameters, very highly optimized math libraries make assumption not just about the behaviour of numbers (their API, defined by how they behave under addition/subtraction, etc) but also about their ABI. For example, a well tuned LAPACK like OpenBlas or MKL has very well tuned loop sizes to optimize cache behaviour assuming that floats are of a particular size.

If you want more advanced materials, both Kyunghyun Cho and Yoav Goldberg posted excellent notes: https://arxiv.org/abs/1511.07916 and https://arxiv.org/abs/1510.00726

Correct me if I am wrong, but you are trying to determine the effect that many different tweaks have on Google's ranking. You are not explicitly trying to reverse engineer their search algorithm, but, implicitly, you are trying to build a model of their model, and to then use your implicit model to drive up rankings.

Of course, the final determination is entirely up to Google, but this seems kind of a risky game to play.

On top of being a genius, reading Knuth is such a pleasant experience. Without having ever interacted with him, he comes across as such a wonderful soul.

I don't think I should have said 'that's exactly right' - I should have said 'that's partially right'. The funding climate right now is very competitive, and the ability to obtain funds is more important than it's ever been.

For another example, here is Peter Higgs (the guy who developed the theory that predicted the boson) discussing modern science: https://www.theguardian.com/science/2013/dec/06/peter-higgs-.... As far as quantitative data about trends in scientific funding - we have lots of data showing that the funding climate is the worse it's ever been, but I wouldn't even know how to quantitatively prove that being 'strategic' is more important now than it was in the past.

This is something other commenters picked up on - but the tragedy of this is that in the current scientific climate, doing good, unglamorous solid science has nowhere near the payoff than coming up with feel-good theories and then marketing the shit out of them.

Peter Thiel said that: "the political people who are nimble in the art of writing government grants have gradually displaced the eccentric and idiosyncratic people who typically make the best scientists. The eccentric university professor is a species that is going extinct fast" - and that's exactly right. Cuddy is a terrible scientist, but she's a fantastic communicator, an engaging speaker, and a great 'seller'.

How do we fix this? Brian Nosek is doing some absolutely fantastic work on this, and I hope it picks up more traction, but I'd like to see the ecosystem around scientific communication seriously revamped. There are a few publications (the Economist and the Atlantic come to mind) that have excellent scientific writers, but things like TED or Malcom Gladwell's books are a blight upon science.

I might be wrong but I thought the STL design docs also had restrictions on algorithm speed? As in - you are free to choose any design you want, as long as it has a given API and has O(1) access.

Lambdas are an incredibly useful construct, and, once you master them, they do make a lot of programming tasks much easier.

However, C++ lambdas picked the most horrifically ugly syntax possible, and they switch between three subtly different semantics (copy, reference, move) depending on a single glyph. I feel bad for the people working on modern C++ - maintaining backwards compatibility is a huge constraint upon design space.

I'm not surprised that lambdas slow down people that aren't already experienced with them. Since I rarely program in C++, whenever I go back to it, I always have to spend a bit of time bashing my head against the horrific syntax.

Thanks for the explanation, that makes a lot of sense.

This was exceptionally well written, and it closely mirrors my experience as a python programmer slowly moving into Rust.

Rust is awesome, but there are a few rough corners where you end up having to write really un-ergonomic code.

Depending on how you define 'soon' - it might take a little longer. A bunch of really good people at Facebook AI just worked on Starcraft using reinforcement learning, and even getting the micro down is really hard.

That doesn't really affect university research that much. It's a much bigger deal in private companies, obviously.

I am surprised they only used ~2000 samples. That's a very tiny amount of data to train a decently sized CNN, even with data augmentation.

I presume using a pretrained net probably won't help you since the features found in pathology slides are so unlike those of natural images.

numba works on python code, not on dags generated by tensorflow.

In practice, what numba does is turn the python code into llvm types, and then compile those with LLVM. What the OP is doing is turning tensorflow dags into llvm types, and then compiling those with LLVM. You can look at numba and the OP's project both as front ends to LLVM.

I have a feeling that implementation speed is an instance of Goodhart's Law (https://en.wikipedia.org/wiki/Goodhart%27s_law). If you keep everything constant (code quality, amount of tests, documentation, etc) then a faster engineer is obviously better. However, if you start using speed as a criteria to judge engineers, then the easiest way to increase speed is to sacrifice things which make code maintainable and modular.

Finding metrics which work well even when people try to game them is incredibly difficult (if not impossible).

The cops who have to look at these awful images to track down those monsters are heroes. I cannot imagine having to do that every day without ending up completely broken up inside.

I'm usually a huge fan of pg, but in his essay about founders, there was one section which I wasn't very comfortable with: http://www.paulgraham.com/founders.html

Specifically, when he talks about naughtiness, he says:

Morally, they care about getting the big questions right, but not about observing proprieties. That's why I'd use the word naughty rather than evil. They delight in breaking rules, but not rules that matter.

I have absolutely no interest in trying to play language games - and maybe I'm misunderstanding what pg is saying, but it has always seemed to me that the judgement of whether the rule that was broken was consequential or not is post-hoc. For example: had airbnb gotten in serious trouble and floundered in the aftermath of when they started scraping craigslist for listings, they wouldn't be clever and naughty, but reckless and foolish. Had zenefits managed to grow even more or hire some key lobbyists and get the law changed in time, their CEO would be hailed as a visionary genius that cut through pointless red tape.

Anyway: the reason I brought that up is that a lot of the ethically dubious things that Elizabeth Homes did are very similar to things which a lot of tech companies did at one time or another. Trying to push ambitious young men and women to look at rules and regulations as something they should take pride in hacking and bypassing is a dangerous game - even more so in fields that are highly regulated.

Addendum: part of the original hacker ethic was to ignore stupid rules. For example, we take delight in Feynmann cracking safes at Los Alamos, or finding some clever hack to bypass a pointless procedure. However: I think it's one thing to hack a system to make a point about how stupid it is, but it's completely different if you add a monetary incentive, and suddenly the rules that get broken are those that stand in the way of you making money. The two sets have a fairly small overlap.

Addendum two: in a complex society like the one we live in, we have a lot of dumb rules. I'm not trying to defend them - we should obviously get rid of them, even in healthcare. A lot of economists have written intelligently about how to make the approval process of the FDA more agile.

Focusing on your last point: this is true in a lot of fields.

ML is amazingly powerful, but if you don't have sufficient domain knowledge, or you aren't collaborating very closely with actual experts, you can make very dangerous mistakes. Domain knowledge helps a lot - not just in malware, but in biology, image analysis, etc..

No. Physicists just take a lot more math classes (including statistics and data analysis) during their undergrad/graduate studies than biologists or psychologists do.

Yeah research gate has some of the most awful dark patterns/invitation spam of any social networking site. Co-authors constantly inviting you to join, convincing you people that are not on the site are on the site, etc...

Finite difference approximations to the derivative are not practical at all when doing deep learning. Derivative using backpropagations can be calculated at the same speed as the forward pass - while derivative approximated using finite differences require 2 passes PER parameter, and run into numerous numerical issues with accuracy.

It's an absolute non-starter.

Oh, much simpler than that - simply that you can pose the problem you are trying to solve in a way that the derivative can be calculated. If you can, you can bring about the whole deep learning machinery to bear on it.

I think most people are very bad at predicting the future, and I know that I'm very bad at it, so I won't try to make detailed predictions about the long term effect of AI since I would get it all spectacularly wrong.

However, in the short term, I think the areas where AI has the potential to make the biggest impact are where we have huge amounts of data (and gathering data is cheap/getting cheaper) and we have an obvious objective. I am really excited to see what Deep Mind will do with their collaboration with the NHS, and I think some incredibly exciting thing will happen when people with a deep understanding of biology collaborate with people who really understand deep learning.

The difficulty is that in industry the people that often have an easy time raising funds are impressive bullshit artists that are great at selling stuff (Theranos, Verily) while in academia people often spend too much time in their own comfortable silos, and straddling multiple fields is a very difficult act to pull off if you want to ever get tenure.

While LeCunn and Ng are real world experts on AI and Deep Learning, the other two people in the article have very little technical understanding of deep learning research.

The huge triumph of DL has been figuring out that as long as you can pose a problem in a differentiable way and you can obtain a sufficient amount of data, you can efficiently tackle it with a function approximator that can be optimized with first order methods - from that, flows everything.

We have very little idea how to make really complicated problems differentiable. Maybe we will - but right now the toughest problems that we can put in a differentiable framework are those tackled by reinforcement learning, and the current approaches are incredibly inefficient.

Excellent. I look forward to playing around with the datasets as soon as they are released to the public.

Have you considered partnering with some differential privacy people and releasing the datasets to kaggle?

This is a real problem, and it sucks. By offering money with strings attached (about bureaucracy, overhead, etc) you have an opportunity to shape how things are done though.