I don't understand this comment. How does averaging noisy signal, even systematically noisy signal, result in something that is noisier than any individual signal? I would have assumed the average would converge on (real signal + systematic error).
109 karma · joined October 8, 2016
I don't understand this comment. How does averaging noisy signal, even systematically noisy signal, result in something that is noisier than any individual signal? I would have assumed the average would converge on (real signal + systematic error).
If you don't had that scale, uncommon tracks will be full of interviewers who are out of practice for your question, and poorly calibrated to evaluate the candidate.
Also, some companies might not want to allow any of these methods, since they each have their own tradeoffs/blind spots.
It might be worth it if I were willing to pay a huge amount to get back results from an experiment faster, by using lots of TPUs- distributed learning on GPUs doesn't seem easy yet.
Do you have a recommendation for a specific data processing experiment of theirs I should check out? I really feel like I just missed a paper where they proved some real advantage over other hardware, and once I found that I'd understand. I respect the Azure teams generally and assume they know what they're doing- but can't escape the hunch that what they are doing is network acceleration, and are just releasing to the public cloud because they have these sitting around anyway.
The first paper is actually one I've spent a significant amount of time trying to use, to the point of collaborating with one of the authors. His conclusion was that FPGAs used to be competitive with GPUs for approximated nets, but the Tesla GPUs were such a jump forward in practical network performance that it wasn't worth trying to compete outside specialized realms like binary nets.
The second paper was interesting- I can imagine why the problem they are trying to solve would be a good fit for FPGAs. However, I'm suspicious that they implemented an entirely different algorithm on the FPGA, and didn't measure the performance of that algorithm on GPUs. I'm all for using the best algorithm for the hardware, but I worry they just used an overall better algorithm on FPGAs and conflated the results.
Are there any really good papers, projects, or products that show where FPGAs provide a major commercial benefit over a GPU?