50 karma · joined August 15, 2024
This also relates to my interp of the article. There's grinding/training and then there's "having that dog in ya". He's trying to remind people to do the latter.
As for Kokotajlo, he's a good SF author, but I don't think it's nearly as good a prediction as it appears at first glance.
Like any SF, he's correct about some things, and way, way off base about others. Specifically, he was mostly correct about short-term (1-2 year horizon) technical aspects, but anyone familiar with how transformers work would have guessed those. I'm not even in AI/rationalia land, but believe I was having these exact conversations by 2021. Specifically about multimodality and steam-of-consciousness ['chain of thought' reasoning]. He is spot-on about chip shortages (to my eyes, his most well-done prediction).
Other thoughts are a storytelling device, like a well-told horoscope. He makes non-categorical statements that were already happening as if they categorically will become a new thing. But really he's just going off the current trend that everyone knows, and thus will be nearly-impossible to falsify. For example, 'People who still remember 2021 think of it as the golden days, when conformism and censorship and polarization were noticeably less than they are now. Just as it is normal for newspapers to have a bias/slant, it is normal for internet spaces of all kinds—forums, social networks, streams, podcasts, news aggregators, email clients—to have some degree of censorship (some set of ideas that are prohibited or at least down-weighted in the recommendation algorithms) and some degree of propaganda.' Sounds scary! But it is literally just saying 'polarization and marketing specialization will continue to deteriorate'.... which, duh. In essence, you could make this statement for 2018-> 2021, or for 2008->2018 and 'feel' just as correct!
Other key thoughts were somewhat off. Specifically, 'It now costs an order of magnitude less compute (compared to 2020) to pre-train a giant model' is kind-of-true, in that it costs less now, but the models have gotten so much bigger that the prices haven't actually gone down much. Might be changing soon, but definitely several years off.
The scariest central prediction that the political stuff rests on is 'After years of tinkering and incremental progress, AIs can now play Diplomacy as well as human experts'. This part, as far as I can tell, is just not true and hasn't changed.
And, well, 2026 predictions are just not correct at all. Some examples: 'Mormons create a “Christian Coalition” internet stack, complete with its own email client, social network, payment processor, news aggregator, etc.' -- basically untrue. I know a few ex-mormons, such a thing is not prolific.
'The US and many other Western governments are gears-locked' The US is far from gears-locked, in fact, it's full-steam ahead to er, well, something.
'“Should I wear shorts today?” “Order some more toothpaste, please. Oh, and also an air purifier.” “Is this cover letter professional-sounding?”' -- kind of correct, but in practice most people just do the last one. I'm not aware of many that give chatgpt agents credit card access (refer to the famous vending machine experiment). But predicting this with GPT3 is not that crazy, cause people were already doing that.
Personally, I am much more biased towards the METR timelines/way of thinking: https://www.lesswrong.com/posts/deesrjitvXM4xYGZd/metr-measu...
I found Kokotajilo's response comment to this kind of, idk, immature, and not especially mathematical.
Not really familiar with translation machinery or luciferase, but it seems pretty compelling to me.
There are some central concepts, and which ones matter to you depends on what you're planning to do with your future. The key points are:
* Matrices/vectors as grids of numbers/computational tools.
* Matrices/vectors as positions and transformations of those positions.
* Matrices/vectors as more abstract geometric objects
* Matrices/vectors as algebraic objects.
There is no 'done right' imo. My sense is LADW is probably the best option for a motivated honors math student, because unlike Axler the author doesn't hate determinants for whatever reason. Strang would work well for engineers. Axler is mostly concerned with the last two, but IMO this makes him kinda niche.
(A: there be dragons)
if you lived in a society that valued, i dunno, tracking and hunting down giraffes in small groups, would you have the same struggles? what if just participating in society required ~20 hours of athletic activity a week? i'm not entirely convinced you would have this problem, based on the anthropology i've read.
the signal of a maladaptive culture is not 'i feel like the people around me have a moral failing'. It is 'i, and many others, feel like we've all got basically the same moral failing.'
personally, this has been a very helpful reframing. If I simply can't bring myself to do something, that means not that I am bad and my willpower is bad, it just means that something is materially wrong and I should consider addressing it by doing things that my body will let me do.
if you think CRISPR or P1 transduction were discovered outside academia, you are wrong. and this isn't even discussing stuff with no immediate clinical applications that is nonetheless important (jumbo phages, asgard archaea are hot rn)
Personally I need 10-20 hours of real sport a week to function really well, which seems to put me in some very high percentiles on HN. I need so much sport that I barely have time to do anything besides work and exercise, which means I don't do many non-athletic hobbies anymore.
And even in more computer-adjacent fields, this is still ridiculously reductive. Geoff Hinton is an academic through-and-through, and he changed the world even for computer scientists. What about someone like Don Knuth? I mean, even google's pagerank started as an academic project.
Engineering firms do great research too, but this is not the only way.
Major bummer that others have had differing experiences from me, here.
I've run the exact lab you're describing, and I think we gave full credit for anything between 5m/s^2 and 20 m/s^2 provided there was some acknowledgement that this was at odds with what was expected. We very often would check in halfway through class and either tell the kids what they were doing wrong, or even tell them to write something 'this is at odds with literally all known science and I think I don't trust this'. For this particular lab, I've never seen errors as large as the ones you've described, so your lab was likely very poorly set up.
In other cases, I've made extra time (and allow students to come in) in case their numbers were so weird as to be problematic; just depends on the lab. Any teacher worth their salt will do this. It's a shame the teachers you had were terrible and incentivized bad stuff.
If being in a lab has taught me anything, it's that doing good science is often morally difficult. Sticking by your guns is hard.
But you are right in some sense: there are definitely incentives to... misreport. The best we can do as teachers is to reduce those as much as possible and reward kids/students for being honest.
I'm not sure that invalidates my broader point in any case, but good to know!
I've been a lot more attentive to this kind of stuff lately due to some chronic health stuff that came up in the family. I think there's probably some framework in modern medicine that makes it less prone to adopting these kinds of methods. Maybe it's just historical baggage, who knows...
Do you have a citation on the fact that 'most' pathogenic strains have a plasmid making them so? Some guys in our lab have been playing around with plasmid copy number lately (in a largely 'basic science' kind of way) -- this could give some nice context for that work.