415 karma · joined June 20, 2023
The question is not whether you should have datacenters, or the most advanced chips, or the ability to build the most capacity. But do we need all this now? Will there be enough demand? People want to make profits form this thing, and what's being pointed out is that maybe there won't be enough demand to generate profits for all this investment.
I don't agree with everything he thinks, but I think I agree with him more than I do Sam Altman or Jensen.
1. Revenue if anthropic and openai is unlikely to grow to the high levels they need to pay for their commitments. Many of their heavy users (coding) will eventually offset a lot of usage to more efficient and cheaper open weight models. I know of people in a company I was at that spend thousands of dollars a month on tokens. I am sure that what they're using it for can be substituted in large part by way cheaper models.
2. A lot of corporate AI usage is being pushed by management that doesn't really understand the extent of its useful, and just wants to call themselves an AI-first company.
3. If this datacenter build-out proves to be beyond the actual demand, there might be a big economic crisis as to how much of the financial system is getting tied up with it (insurance money, private credit).
Whether openai and anthropic actually die, I'm not sure. But I don't think they'll be the next big tech companies. I think eventually they'll be absorbed by others.
The whole thing I think, can be summarized as: LLMs will be commodity like. And it's price will go down and eventually will run locally, it's not at all clear that this will bring AGI and that it's worth infinite amount (or trillions) of investment ahead of the actual demand or the AGI level do-it-all-for-you AI being reached.
Like were executive at Google and AWS and Microsoft expecting this kind of performance from models smaller than what openai/anthropic have been doing? Are we really in a "compute desert"?
But the normal speed one seems to be free or with very generous limits.
I'm not sure exactly how you did this, but I think you asked an LLM to come up with the wrong options. Two things to consider:
1. While the LLM can go r good options, they won't be always hard to guess. I wonder if instead you can have the LLM generate very close words (or skip using an LLM entirely) and put those as the options. 2. If you will generate options with an LLM, make sure you are mindful of its inability to shuffle things around. The correct answer was overwhelmingly the first or second option in the list. You should ask the model to give the options in a uniform order (say from true meaning then decreasing amount of replayability), then manually shuffle them so that the probability of which option (A, B, C or D) is always 25%.
I just cancelled my cursor, which was the best for my workflow. Will need to find something else.
It's a different kind of hype than Nvidia has, which is showing very high and fast growing revenues (which may not continue, but they're real now). Jensen I think is not as critical to the AI hype as Elon is to his companies.
All these major tech companies eventually get leadership changes. Apple, Google, Amazon, have all done well because they're real companies and go beyond their original leadership. Tesla and SpaceX I think would surely go down the moment Musk is no longer in leadership.
You don't have to say I wont be a rockstar, therefore let me use some AI to make a song, and in doing so give up on the joys of touching and making sounds with an instrument, a very old human thing we've been doing all over the world, having someone show you a song, or look up a youtube video and learning it from some random stranger.
Even better, being in love with a song and finally being able to play it yourself!
Maybe AI could've sufficed for Paul McCartney's interest in music, and provided a creative outlet. But we wouldn't have had something as great and as human as the legacy of the Beatles.
As I write this, I think when a melody sounds good it's likely related to the implied harmony in the notes being used, and obviously the expectations the listener gets and how they're handled. But I don't think there is a system of constructing good melodies in Western classical music theory.
AI code is much easier to read than AI text (or book). It's kind of like what people think of the AI generated book cover. That's how I feel about the generic AI writing.
I think physical activity, even just going on walks makes one feel change is possible. If something sucks and I sit home all day on YouTube, then it continues to suck. If I can change my environment, do things outside, see new people and find myself if different situations, then the thing that sucks starts feeling like maybe it also could change.
For example I doubt exercising in a basement just by yourself on 1 machine is likely to materially help with depression. At least not as much as going out doing a variety of things, or playing a game of basketball at the local gym/community center.
I think it may be the case that many of these people that commit suicide or do other dangerous things after motivation from AI, are actually using weaker models that are available on the free versions. Whatever ability there is in AI to protect the user, it must be lower for the cheaper models that are freely available.
I am calling for some care to go in your product to try to reduce the occurrence of these bad outcomes. I just don't think it would be hard for them to detect that a conversation has reached a point that its becoming very likely the user is becoming delusional or may engage in dangerous behavior.
How will we handle AGI if we ever create it, if we can't protect our society from these basic LLM problems?
I don't know how this doesn't give pause to the ChatGPT team. Especially with their supposed mission to be helpful to the world etc.
Venezuela has lots of oil and drugs. If different factions fight between themselves there's no reason you couldn't end up with a divided and dangerous country that in some ways could be worse for the people than Maduro.
The best way for "oppressed" people to be liberated is through some joint effort by parties that really want to help out and assume responsibility, or by supporting a revolution that naturally takes over. I don't think there's been any cases of success from this process of forcibly removing the dictator, and crossing your fingers that things will go well.
Usually antitrust is for consumers, but here I think companies like Microsoft and AWS would be the biggest beneficiaries of having more AI chip competition.
MIT - Do whatever you want with it (except deploy to production )