[1]: https://www.cnbc.com/2018/03/01/new-fed-chairman-says-amazon...
709 karma · joined December 5, 2014
[1]: https://www.cnbc.com/2018/03/01/new-fed-chairman-says-amazon...
I'd love to hear stories on running a online store, consulting in the space or building software around the platforms (like Shopify).
Machine learning (ML) and Deep Learning (DL) in particular benefit from fast computer chips. The most impressive gains in AI in the 2010s (Computer Vision & Natural Language Processing) were made thanks to DL. There's a famous post by AI researcher Rich Sutton, which summarises this fairly neatly [1].
Now, this connection between DL and fast chips would tie the progress of AI pretty tightly to the progress of Moore's Law [2]. There's some compelling evidence that Moore's Law is at least slowing down [3]. On the other hand, there are industry experts like Jim Keller who pretty strongly disagree with these assessments [4] and even TSMC seems to be bullish on being able to keep up with Moore's Law [5].
Some estimates of GPT-3s training cost put it in the range of ~5-10 million dollars [6]. It's hard to say how big of an impact on the economy GPT-3 will have. It's probably safe to assume though, that OpenAI is already working on GPT-4. The jump in parameter sizes from GPT-2 to 3 was roughly 100x (1.5 billion vs. 175 billion) and I'm assuming the price of training the model increased in roughly the same proportion (I might be wrong here and if anyone can point me to evidence on this, it would be much appreciated). With these assumptions, and provided that GPT-4 won't be affected by diminishing returns of adding more parameters (big if), the price for it would be somewhere between 500 million and a billion dollars. That's still not an insane amount of money to put into R&D, but you'd probably want it to be at least be somehow economically viable to be able to justify putting a hundred billion dollars into GPT-5.
All this is to say, that I find making predictions of the progress of AI really hard due to the large amount of uncertainty related to the field and the underlying technologies (mainly the hardware).
[1]: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
[2]: https://arxiv.org/pdf/2007.05558.pdf
[3]: https://p4.org/assets/P4WS_2019/Speaker_Slides/9_2.05pm_John...
[4]: https://www.youtube.com/watch?v=oIG9ztQw2Gc
[5]: https://www.nextplatform.com/2019/09/13/tsmc-thinks-it-can-u...
[6]: https://venturebeat.com/2020/06/11/openai-launches-an-api-to...
Anyway, my point is that having faster and more energy efficient chips would make the training of stuff like GPT-3 cheaper.
How off base is that impression?
I knew that Django, the Python web dev framework, had been named after some obscure dutch musician. But after reading the book I had a wholly new appreciation for Django Reinhardt [1], the artist. Man was truly a great jazz improviser, whose legacy now lives on in the framework.
The Logrocket blog is an excellent example of this. Lots of useful blog posts. Another one I've been relying on is the blog of Robin Wieruch [1].
I guess stating this is a little obvious, but one of the upsides of using popular tools is the productivity boost you get from the surrounding ecosystem.
[1]: https://www.cnbc.com/2020/07/30/apple-aapl-earnings-q3-2020....
[1]: https://seekingalpha.com/article/4363542-big-5-twilight-zone
The app also seems to be kind of a wonky fit to Amazons, Microsofts or Apples portfolio (although Amazon does own twitch, so who knows).
I can't make a solid argument for it, but I wouldn't rule out either Netflix or Disney making a bid for TikTok. Both companies are great with video, and I think the current CEO of TikTok is the former COO of Disney. IDK.
Half of the drafts I write do not end up as published questions, just because the writing process itself is often helpful in figuring out what ever problem I have with my code.
What I'm wondering is how worried are the people of Taiwan about China encumbering on their sovereignty in the near to mid future? Or is this something that people over there do not give too much thought to in their day to day lives?
One can already use stuff like FastAPI [1] and Django is supposed to be getting async/await support later this year [2].
So that should make Python more competitive when compared to node.
Mediocristan and Extremistan are Talebs analogies for domains with different underlying mechanics (for lack of a better word).
I'm going cut some corners here, but to keep it short, the phenomena in Mediocristan can be modeled by using gaussian probability distributions (weight, height etc.), while the phenomena in Extremistan are better modeled using power law distributions, that have so called fat-tails. (wealth, land ownership)
One of the central arguments of the book is that people have been using tools from Mediocristan in Extremistan and that's what leads to these Black Swan phenomen, which in turn might have potentially ruinous effects.
One example he uses is a now defunct hedge fund called Long Term Capital Management (LTCM) [1]. My (poor) understanding of what led to LCTMs ruin (and required the Federal Reserve to intervene) is that they used some fundamentally Gaussian techniques to model derivatives.
The models predicted certain events to have a lot lower probability to occur than what was actually the case (fat-tails). Underestimating these outliers combined with the fact that the firm was highly leveraged, was ultimately what went wrong and had potentially disastrous effects on the financial markets.
[0]: https://people.wou.edu/~shawd/mediocristan--extremistan.html
[1]: https://en.wikipedia.org/wiki/Long-Term_Capital_Management
It's also a great aggregator book. It introduced me to the works of Kahneman, Tetlock, Poincare and Mandelbrot.
I wonder if they'll support plugins and importing other VS Code settings to the online editor.
But did not know about the military use of the song. Very interesting.
If you're into pop culture (sports, music, TV and Movies) I'd also recommend the Bill Simmons podcast. The interviews are usually great (especially the ones with actors) and the sports takes are entertaining as well.