370 karma · joined October 7, 2022
The account name is hn.
The domain is fastmail.fm.
There isn't really a figure for how much compute time it takes to train this thing, but 8x H100s have 32PF of AI compute among them. This job had 2,100 (half precision[1]) PF-in-fugaku / 158,956 nodes-in-fugaku * 13,824 nodes-in-job = 182 PF-in-job, implying it can get the job done 5.6x faster, or a little over ten days at the most optimistic.
Electricity costs for these nodes for ten days looks fairly similar to the rental costs of 8xH100s for 60 days according to my research. Lambda labs seems to have very cheap instances for 8xH100, but AWS and its ilk are much higher. However, the comparison is a little weird, as Fugaku is also a few years old now, and the contemporary GPU at the time of its release was the A100 (1/13th of an H100). The next Fujitsu chip may well narrow the power/performance gap between itself and (say) Blackwell or whatever is current at the time.
[1] https://www.fujitsu.com/global/about/innovation/fugaku/speci...
Tell that to Gap, who sometimes sends up to five spams per day.
Therefore, the title should be “gnuplot for numpy.”
Installing your proposed $4B of solar will get you 4GW at peak, which will produce about 5840 GWh per year due to variations in sunlight over the days and seasons. Diablo Canyon produces 18,000 GWh per year. Replacing Diablo Canyon with solar will cost $12B plus batteries, maintenance, and operations staff.
Figures/Sources:
- $1m buys 1MW of solar panels that provides 1,460 MWh per year. https://coldwellsolar.com/commercial-solar-blog/how-much-inv...
- Diablo Canyon produces 18,000 GWh per year. https://diablocanyonpanel.org/history-of-dcpp/
“Volunteer computing” is probably a better way to refer to this phenomenon.
If you’re ever on funky hardware and want to ensure continuing Linux compatibility, I guess it pays to subscribe.
If I’m running M3 images on my Ryzen, performance is gonna be horrible, but why would I?
I've suffered through many books that had a very good message, but should have been 20% as long as they were. Much repetition in every possible spot, just to end up with a book of a respectable length that people don't feel bad spending $15 on. Comprehensive summaries are a great way to get around that.
On the other hand, these summaries are (by definition) derivative works that serve almost entirely as just distillations of the book. There's some opinion rolled in, but the description of the source material goes far beyond a book review to the point that it could be seen as a time- and money-saving replacement. If I were an author, I wouldn't be happy that my book was interpreted by a random third party, let alone that many folks will assume they got the value from the work and won't buy it.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1743237/pdf/v08...