"You're totally right in catching that mistake! The temperature is indeed one order of magnitude off. Let's walk through step by step what it should be:"
I think it's in large part just having to do with us developing our frontal cortex and like impulse control. I would have probably gotten dopamine addicted to it 15 years ago, as well as wouldn't have some nagging back-of-mind thoughts about having to use my time to be converted into money to survive at that age.
It's probably good if some portion of the engineering culture is irrationally against AI and like refuses to adopt it sort of amish style. There's probably a ton still good work that can only be done if every aspect of a product/thing is given focused human attention to it, some that might out-compete AI aided ones.
Crazy how we went from google feeling like they were a dinasour who could never catch up to openai, to almost feeling like the opposite in terms of being able to catch up. All within just 1-2 years.
It's definitely written by an AI. The end description of hitchhikers guide is "[...]the meaning of life. Which turns out to be an integer." No one would bother writing that.
Makes no sense since they should have checkpoints earlier in the run that they could restart from and they should have regular checks that keep track if a model has exploded etc.
Untwittered:
A Markov model and a transformer can both achieve the same loss on the training set. But only the transformer is smart enough to be useful for other tasks. This invalidates the claim that "all transformers are doing is memorizing their training data".
Best case is hardly a bubble. I definitely think this is a new paradigm that'll lead to something, even if the current iteration won't be the final version and we've probably overinvested a slight bit.
Think of it this way: Given any company in the world to invest that money, do you think it's best invested in your company or some other? Because if there's another one (eg nvidia, apple etc) then you should take the money out and move it into stocks in that one
The attention of humans got ruined with later generations. The generation before us were a different level of skilled, and it's hard as a millenial (me) and gen z to get close to them.
Just imagine some other people will carry the burden and mentally distance yourself from it to relax from it wearing you out. You can take up the burden again later once you've recovered and others are worn out
My hunch is that openai used ghibli as the example in their earlier dall-e blog posts strategically because anime was earlier said by the PM not to be protected by copyright in training. OpenAI is always sneakier than most people give them credit for.
I know the comments here are gonna be negative but I just find this so sick and awesome. Feels like it's finally close to the potential we knew was possible a few years ago. Feels like a pixar moment when CG tech showed a new realm of what was possible with toy story