LittleTimothy

Reading the original bug report and the response by the maintainers.... yeeash. They are way more tolerant than they should be. At this point I feel the structure and verboseness of the original report are just massive flashing red flags that this is AI slop. The verbosity and detail are immediately and obviously out of line with the complexity - a better report would've been "Yo dawg I think there's a buffer overflow in this strcpy". The first thing I would've replied is "Are you sure" because I think the AI would've immediately done the classic "You're absolutely right that..." without anyone bothering to look at the code.

I think the natural response will just be lower responsiveness from the maintainers to anonymous reports.

Dumb question, would this be captured in NK's GDP figures? I imagine that the whole point of this sort of activity is so that they can skirt sanctions so there may well be a tonne of this activity going on that just never shows up in GDP estimates because.... well it's designed not to.

If the route we want to go down is looking at lobbying my guess is that the more complexity you add to the SNAP rules, the more lobbying you will get. If this is on the table every food company and their dog will be lobbying to X,Y,Z product on the list. The more prescriptive you are the more you're opening the door to lobbyists putting in quirky rules that benefit themselves over the competitors. It also just seems like a bad idea to let one unelected weirdo personally choose the rules surrounding millions of people's groceries. Plenty of people are saying "oh, well I agree with him on this one" - you haven't seen the list yet, it's RFK, by the end of this we're probably going to have anabolic steriods and bear meat subsidized, I'd read every line of small print before endorsing this.

Oh it's a totally different issue on the quantum side that leads to the same issue with difficulty verifying. There, the algorithms that Google for example is using today, aren't like prime factorization, they're not easy to directly verify with traditional computers, so as far as I'm aware they kind of check the result for a suitably small run, and then do the performance metrics on a large run that they hope gave a correct answer but aren't able to directly verify.

Depends on your understanding of human knowledge I guess? People talk about the frontier of human knowledge and if your view of knowledge is like that of a unique human genius pushing forward the frontier then yes - it'd be stuck. But if you think of knowledge as more complex than that you could have areas that are kind of within our frontier of knowledge (that we could reasonably know, but don't actually know) - taking concepts that we already know in one field and applying them to some other field. Today the reason that doesn't happen is because genius A in physics doesn't know about the existence of genius B in mathematics (let alone understand their research), but if it's all imbibed by "The Model" then it's trivial to make that discovery.

It's fascinating that this has run into the exact same problem as the Quantum research. Ie, in the quantum research to demonstrate any valuable forward progress you must compute something that is impossible to do with a traditional computer. If you can't do it with a traditional computer, it suddenly becomes difficult to verify correctness (ie, you can't just check it was matching the traditional computer's answer.

In the same way ChatGPT scores 25% on this and the question is "How close were those 25% to questions in the training set". Or to put it another way we want to answer the question "Is ChatGPT getting better at applying it's reasoning to out-of-set problems or is it pulling more data into it's training set". Or "Is the test leaking into the training".

Maybe the whole question is academic and it doesn't matter, we solve the entire problem by pulling all human knowledge into the training set and that's a massive benefit. But maybe it implies a limit to how far it can push human knowledge forward.

This is... interesting. From my understanding - and people can correct me if I'm wrong, but didn't Microsoft spend an extremely large amount of effort essentially trying to screw people who made things like this in the 2000s? Interoperability and the Open Office movement were prety hard fought. It's kind of crazy to see MSFT do this today. Did I just misunderstand and the underlying formats (docx etc) were actually pretty friendly, or have the formats evolved a lot since then? Or is it more a case of "It doesn't matter if it looks terrible because we're feeding it to the AI beast anyway"

A cynic might say it became suddenly easy when MSFT had a reason to allow you to genereate markdown to feed into it's AI?

The question I'd like to know the answer to is "What was the total cost of training Gemini 2.0 and how does it compare to the total cost to train equivalent capability models on Nvidia GPUs?". I'd be fascinated to know, and there must be someone at Google who has the data to actually answer that question. I suspect it's politically savvy for everyone at Google to pretend that question doesn't exist or can't be answered (because it would be an existential threat to the huge TPU project), but it would be absolutely fascinating. In the same way that Amazon eventually had to answer the "Soo.... how much money is this Alexa division actually making" question.

I don't know why you'd be confused by the valuations, they're just 2 different businesses. Google is an ad sales business with an extremely strong revenue driver (youtube and Search) so it churns out money but it's pretty well understood revenue stream and is facing some possible disruption. Nvidia is a chip sales business and whether through luck or through skill (or both) they've essentially ended up as the monopoly provider of the worlds best chips. It's not unreasonable to expect Nvidia can squeeze a tonne of cash out of their customers for the next few years in an undisruptable way. So it's easy to understand the valuations.

On the GPU side - One way to look at it is lets just ignore whether you're selling chips. Let's just look at where compute is done - because Google let's you rent time on the TPUs so if they're really great people will just choose to rent time on them. How much time is being rented on TPUs in the cloud vs. Nvidia GPUs? Google's TPUs are not a market leader, they're probably not even significant market share. Pretend Google decided to sell TPUs tomorrow - are many people who weren't choosing to rent TPUs going to decide to buy them? No. There's an element of interoperability and supply chain etc. etc. but I think the core of it is a full ML product the TPU isn't anywhere close to the Nvidia GPUs.

No.

This is progress towards quantum computing but no where near progress towards a real practical quantum computer that could break Bitcoin's algorithms. If progress continues it could be an issue in the future, check back in next time Google publishes a paper.

You mean some capricious billionaire could get so mentally ill he buys up your favourite ~~website~~ copyrighted work and then just spend all his time inviting Nazi's and bigots to use it despite absolutely destroying it's value? That seems incredibly unlikely to me.

It's interesting to see such a long discussion of the value of a peice of art without any real attempt to wrestle with what the actual value of the art is. The copyright of Tolkien's work isn't just an amorphous blob of future cash flows. His estate is as much interested in protecting the artistic integrity of his work as it is managing the finances. To put another way, should you be forced to sell your home if your neighbour offers the government to pay more property taxes on it than you would?

I think fundamentally this article seems to be asking "There are these guys right, and they have created this fantastic stuff and it's protected by copyright, but they're not choosing the most economically productive way to use their stuff, so we should find a way to take it off them". And I'm not sure that premise is correct. If you want to create economic value from a copyrighted material please... go create some copyrighted material with economic value.

I think the difference between cancel culture and just freedom of association is the difference between "I don't want to work with you because of X" and "I'm going to influence others not to work with you because of X". Whether that be anons on twitter calling up your employer to get you fired, or PG having a quiet work with his friends over at some VC firm, they're both the same strain of toxic.

As far as I understand it, only kind of? It's open source, but in their paper they did a tonne of pre-training and whilst they've released a small pre-training checkpoint they haven't released the results of the pre-training they've done for their paper. So anyone reproducing this will innevitably be accused of failing to pretrain the model correctly?

"You didn't use enough compute in your reproduction of our methods" is kind of a funny criticism today. Well yeah, you're Google. Sorry guys, no reviewing of our methodology unless you own a cloud. It wouldn't surprise me if it's true that more compute, more pre-training etc. provides a lot of utility, but that does make it difficult to verify the work.

One interesting aspect of this though is vice-versa, whilst Google has oodles of compute, Synopsys has oodles of data to train on (if, and this is a massive if, they can get away with training on customer IP).

They've already been headed that way for a long time, the problem is that they need huge amounts of capital to complete their foundry road map and that is meant to be cross-subsidized by the other part of their business, but now it's looking shakey that they'll even have the capital to execute the plan - setting aside whether it would even work if they could.

Difficult to see how this is anything other than a failure. I had high hopes when Gelsinger returned, but it seems that BK had done too much damage and Gelsinger didn't really get a grip on things. One of the things I heard that really highlighted the failure of this recovery was that BK had let Intel balloon in all sorts of ways that needed to be pared back and refocused, but head count under Gelsinger didn't just stay static but continued to significantly grow. It's no good giving the same politically poisonous, under-delivering middle management more and more resources to fail at the same job. They really need to clear house in a spectacular way but I'm not sure who could even do that at this point.

You need to look at how this market really operates. You have massive massive up front capital investment, you spent tens of billions of dollars designing a chip plant, and then billions more actually building one, and at the end of that 10 year long multi-billion dollar process you get 1 chip out of it. If that chip is better than your competitors the entire fabless semi-conductor industry is going to trip over itself bidding up your capacity and turning your investment massively profitable. If your chip is worse than your competitors you have to sell capacity at practically any price to recoup any possible return. And whoever did a good job? Well they've got billions to plough into the next process node. The company that did a bad job? Well now they need to go out into the market and try and get funding to make that entire bet again - but this time with a track record of failure. What's the likelihood a company coming from 0 can make all that work, with no existing assets to secure debt financing for the investment?

It's also important to understand what the US is paying for here. They're not paying Intel to make chips, they're paying Intel to make chips in the US as a geopolitical strategy. Intel will make these chips either way, but this action from the US means that Intel is continuing with their US fab plans whilst for example, dumping their EU plans.

I think you're massively over-estimating what the average consumer wants from their computer. I need a computer but you're saying I shouldn't buy a computer I should buy a laptop and then a separate monitor? Why!? What a weird thing to do. Call me crazy, but people will just buy a computer when they need a computer.

I know, I know, we could get them a laptop and a fancy display and a mechanical keyboard and we could load Ubuntu onto it for them too while we're at it so that they aren't locked in to Apple's walled garden.

That's just not what an average consumer is going to do, they're going to need a computer and buy one.

Such a silly framing for a review. The iMac is for someone who just wants a good simple desktop computer. Ah but what if you want customize? Don't. That's not what it's for. Sure, if you're a tech reviewer and you're all jazzed about HDR and mechanical keyboards then this product isn't for you. But then... your job as a tech reviewer is kind of to understand what different customers will want and "I just want a computer" I would say is literally the most obvious market.

Who is this for? Someone who wants a computer. Ah, but take away the people who want multi-monitor, take away the people who want mechanical keyboards, a bigger fancy monitor, and what are you left with? Still basically everyone!

And it's not like this is the only computer Apple makes. They have 1 basic hardware platform based on the M4, they give you the laptops for portability, the iMac as the basic desktop, the mini for if you want to do all the things that the iMac isn't configured for, and they have the Mac Pro to taunt professionals. It's an incredibly coherent product line up. No one is confused about which one they want.

Oh and by the way, it works really well for Apple too, because they are literally just taking the guts from their other products and sticking it in the same design desktop they've had for years. Do you really think that'll be improved by redesigning it and therefore charging 50% extra due to all the bespoke costs?

>Instead of trying to force genAI tools to tackle thorny issues in legacy codebases, human experts should do the work of refactoring legacy code until genAI can operate on it smoothly

Instead of genAI doing the rubbish, boring, low status part of the job, you should do the bits of the job no one will reward you for, and then watch as your boss waxes lyrical about how genAI is amazing once you've done all the hard work for it?

It just feels like if you're re-directing your efforts to help the AI, because the AI isn't very good at actual complex coding tasks then... what's the benefit of AI in the first place? It's nice that it helps you with the easy bit, but the easy bit shouldn't be that much of your actual work and at the end of the day... it's easy?

This gives very similar vibes to: "I wanted machines to do all the soul crushing monotonous jobs so we would be free to go and paint and write books and fulfill our creative passions but instead we've created a machine to trivially create any art work but can't work a till"

I don't think it is fundamentally the same question. If someone was about to throw a brick through a window I would say "Hey! Don't do that". But I wouldn't walk past a smashed window and think to myself "Better go fix that window".

Separate to that I also think that the question of bringing back a species like the Quagga is very different to bringing back the Mammoths. The Quagga went extinct recently and directly because of our hunting them. If we breed new ones, they'll likely fit into the ecosystem as it is today and may even thrive. Mammoths went extinct an extremely long time ago, the environment is totally different to when they were alive, we have no idea how they would impact the ecosystem today. Maybe they'd be totally unfit and would only exist as pets, or maybe they'd fit in great and drive some other animal or plant to extinction as their population explodes as an invasive species.

The question of "Why bring back extinct species" is the same as "Why not bring some new different species into Australia"

Didn't you hear, every silicon valley company personally invented everything they ever did. Chamath invented Data Science at Facebook!