Compute challenges are more real, but we are seeing for the first time huge amounts of global capital being allocated to solve specifically these problems, so I am curious what fruit that will bear in a few years.
I mean already the stuff that some of these low level people are doing is absolutely nuts. Tim Dettmers work training with only 4 bits means only 16 possible values per weight and still getting great results.