mrngld

Someone on horseback probably said that after seeing a car broke down on the side of the road for the first time.

Well, cars only got better.

We're going to be talking about how to leverage this new tool for many years to come.

Because your 32gb card isn't running this at 1500 token/s. Serving these things at scale with the enormous context windows real use demands and doing some with usable performance takes a lot of expensive hardware. Yes, their margin on straight inference is allegedly really high, but that's severely offset by high capital costs.

If you want to spend a new car worth of money and still not serve as fast as Cerebras because you can't simply buy their mammoth custom chips, then yes you too can self host a huge Deepseek or GLM model.

Here's the thing that's a little different about data centers; we can tell from Anthropic and OpenAI that they're capacity constrained. Inference demand is there. I notice Cerebras doesn't offer much directly any more, all their capacity is getting completely sucked up by B2B sales. Grok did overbuild, but Anthropic was so desperate for more compute they ate their pride and leased the excess capacity.

That means all these data centers are being heavily utilized by actual end user inference demand. Well, some is research on new models, but a lot is actual end user demand. No one has given an explanation of why peoples usage would decline.

On top of that, margin on inference appears to be decent. It's model training that's a serious financial burden.

And maybe that's where there will be a slowdown, maybe the market doesn't justify spending as much on R&D as it does, but the end demand for inference is there.

Does that justify these stock prices? That's a different question. But the housing boom left behind endless rows of empty homes because demand disappeared. The 'dot com' boom left behind thousands of miles of dark fiber that'd been built out well ahead of demand for bandwidth. I can see the stock market having a giant sell off, but I don't see data centers sitting idle in that same fashion.

I get that the guy needed to write a paper on something to get his PhD (in psychology, which might explain some of this as well, I thought standards were a little higher in the hard sciences), but don't know why it deserves a high profile. At least, based on merit.

You don't have to go to the HuggingFace incident! Go back in time to the New York Times legal brawl where NYT lawyers started being able to scoop up all their logs not covered by a ZDR. That's what keeps me from giving OpenAI access to anything too personal. My employer offers to let us use our corporate seats for personal stuff so we can be covered by our corporate ZDR, but AFAIK that enables HR to see all my chats which is just as bad or worse. (Someone in HR in ChatGPT Work: "Create a scheduled task where every morning at 8AM you navigate to the compliance tab in the corporate ChatGPT dashboard and search for anyone asking questions about job opportunities outside the company, or [list of 100 other prohibited things], and alert me with any positive results.")

Looking forward to seeing what Apple cooks up with their Private Cloud Compute and if anyone else takes up the same approach.

Long term it's irrelevant. The only relevant thing is that there's lots of money in chips that can do high performance inference. You see all kinds of competitor products in development or already on the market even here in the US where there are no such restrictions. Cerebras comes to mind. It's natural and expected that eventually Nvidia will either have to keep way ahead or competition will catch up with specialized products.

That doesn't mean by any stretch of the imagination Nvidia will disappear. But the entire stock market valuation, not just tech, has had me scratching my head for a while.

Chinese labs are so used to manipulating benchmarks to try to flatter inferior models that when they finally have one that's really pretty good I think the official announcement here undersells it.

https://deepswe.datacurve.ai/

That's pretty solid. Smarter and cheaper than Luna xhigh, not as smart but less expensive than Luna max. Smashes deepseek v4 flash, and even worse it matches v4 pro at a tiny fraction the cost. Roughly equivalent to sol medium, at a fraction the cost.

They should've just lead with real, up to date data, because it's good, not the silly old tactics like comparing to Opus 4.8 when 5.0 is out in many of their charts.

Congrats to them!

Ya'll are talking lawsuits and organizing protests, but the reality is probably just that no one (outside the Bluesky/HN bubble, which is tiny) cares. Very few probably care what their local government is babbling about today in general, and even fewer what platforms they're using.

And yet if someone did care, there's a decent chance if they asked the right person in the right way they'd be like "Oh, Mastodon? Okay, sounds neat, oh you know what it's just another checkbox in the software we use to cross-post everything! I'll just check the box!"

But nah, lets not do the easy thing, lets look up legal precedent and ponder the ADA.

It's not just about big tech. It's about what enables highly productive societies in general. Without IP, I suppose everyone could share to some limited extent what is already known, but why would anyone ever sink big dollars or time into coming up with something new that'd immediately get stolen by people who did nothing to contribute to it? Indeed, it's part of the problem with manufacturing in the West, there's no robust enforcement of IP with China due to political sensitivities so manufacturers have to have strategies to protect themselves from the fact that if they release a new product they've got maybe 90 days before a copycat hits the market.

I can't help but wonder how much stronger the core of the EU economy would be with actual robust management of their trade relationship with China. The US too but I think the EU is suffering a bit more in real time.

No doubt it makes Doctorow the star of certain types of intellectual circles but the theories have never made a people or country actually well off and never will. But, things don't have to work in the real world to be popular, or even to become policy.

Sounds bogus, there's free payment services in the US as well, that's nothing special. What makes more sense in the India context is low value of volume per customer where even the minimal operating costs are difficult to cover. I say that from some amount of experience in the micro-finance world, where even money provided at zero actual interest in terms of the loaned money itself still ended up having to have somewhat painful interest rates just to cover true costs in low income countries.

I won't do your homework for you but there's people that have done the work of reporting out the numbers it takes to operate a clean government bathroom. It's not just "reasonable wage" territory. It's "civilization can't afford this" level expensive, the type of problem you can't throw money at because it's so bad it's obvious that the real problem and solution exists elsewhere.

And it's only in the West that public restrooms are that expensive. Japans are less expensive because the locals use them like civilized people and they don't have enormous on-going cleaning and repair costs.

So, the problem isn't in paying the cleaners. It's upstream. No pun intended.

I do know that. For a time I was GM for a convenience store. I started requiring people to come ask me or the manager on duty for a key to the bathroom, which I started having locked by default.

There was an immediate 90% reduction in bodily fluids outside the toilet bowl and 100% reduction in stolen plumbing (they were taking part of the p-trap for drug use) and in finding discarded paraphernalia. Correlation isn't causation, maybe all the crack heads were still using the bathroom but decided to be civilized, IDK, but keeping the bathroom under literal lock and key was a big benefit.

It was inconvenient for customers, that was true, and I didn't like that aspect, but it was hard getting staff (or even myself) that had the will to hose poop off a bathroom wall.

Which I can tell from your comment you've never done. "presumably you have equipment" Yeah, a hose. A mop and bucket. You're still in close proximity, you still step in it, it still splashes. And nothing prepares you for the smells. And you can never unsee it. I won't even describe the things I saw in the womens bathroom. Way less common for things to go down in there, but when it did... It's wretched work.

I think the burden of proof is on your position. When activists in my area pick a fight with local government, it's never over that invisible group. Indeed, as far as I can tell, they don't give a rip about the veterans living in the forest over by the VA hospital. I've never seen them over there. They're too busy picking fights defending the ones pooping at the downtown parks (and not in the public toilet) or trying to get city funding and approval for safe drug use, etc.

Now, you're right that there are people that do care and put tremendous effort into that more invisible group. Churches and church-backed organizations do a lot of that. Difference is, except for fund-raising from like-minded folks (and a lot of them hate even asking for money), they go about their work quietly, without seeking attention or reward.

There'd be the possibility of joining forces, but not with the DSA-types that jump in the limelight and scream the loudest and advocate for free needles. The people doing the bulk of the hard work don't want to make a statement on Palestine, they just want volunteers for the hard-to-staff weekday shower-cleaning shift. They need men willing to be mentors, etc. They care about the actual people they're serving, not trying to bring about glorious revolution. The two groups aren't compatible at the moment.

I'm not seeing why it has to be connected to the internet. Make it a private, air-gapped intranet for all (or most) of the benefits of being "connected", but with no entry point for someone sitting on another continent to turn off the water.

I think that sort of journalism has largely been purged from the traditional sources and has ended up on Substack, or their own site. It's transitioned from "I can trust Brand X" to "I can trust that individual because of their track record and the way they reliably report things that don't align with their own world view", etc. Glenn Greenwald is an example. The story around why he had to depart The Intercept (which he helped found!) is everything that's wrong with modern journalism that has pushed people like Glenn to places like Substack.

Bringing it back to paywalls and Kagi and paying for content, Glenn has 3 kids and I think a bunch of dogs and they all need to eat.

I'd be onboard with supporting a local newspaper -- if they hadn't gone all-in with aligning themselves to a particular political tribe. They're part of the problem Ground News makes obvious to a hilarious degree, where if something happens that doesn't fit the narrative you just will never see it if you only look at their platform. Given that not everyone has a reputable local newspaper I don't know that saying "everybody should" is necessarily sound advice.

Anyway, you can block sources individually in Kagi, that can weed out the most common paywall options you don't have, like Forbes or whatever.

I don't have experience directly with Anthropic, but I wouldn't assume anything with such high confidence without knowing the persons situation. No, an individual off the street or with an LLC and 5 employees isn't going to get a special deal from someone like Anthropic.

But if their employer is bringing millions of dollars of potential spend to the table, can tell you from years of experience that turns a lot of 'no's' to 'yes'.

Interesting. I've been wishing that old 'Stars!' game from the 90s would play easily on modern systems. I'd love it if we'd got to the point where I could point Codex at a folder with an ISO from my CD of the game and tell it to go reverse engineer it all for understanding of game mechanics, then go recreate it in a modern language capable of running cross-platform. Scarcely any need to improve on graphics, it could even be a PWA.

There's people that have tried to contact Jeff McBride and follow the IP trail but the IP is currently owned by a company that went defunct. Not sold, but no one is even bothering to register its LLC any more, it's simply dead.

China's 50 Cent Party being a real and noticeable thing (and the two biggest things they like to shill is open weight Chinese models and the futility of resisting a Taiwan invasion), I have to take things like this with a healthy dose of skepticism without corroborating data, since independent evals didn't show the price per task lead you're showing.

If there's independent data showing this feel free to share a link, I haven't seen it. DeepSWE has been most closely matching what I see in my own use.

Sounds like a reheated version of MMT, modern monetary theory. The "just print money" approach exploded in a mushroom cloud of inflation during the COVID years, just as any economist even remotely aligned with the Austrian school of thought could've predicted. And you're still seeing the inflation, it's just partly hidden in and taking place in frothy equity markets.

MMT isn't inevitable. You're describing a path to where Argentina was before Milei. A path to amplifying all the social issues that started in the US with the creation of the welfare state. The alternative would simply be fiscal restraint.

The only real problem or question is if democracy is capable of restraint. The answer is unclear, seems like it may be 'no'.

Absolutely no one serious thinks that, but when you feel like you need to diversify you don't have to find a like-for-like replacement. Any port in a storm.

Just consult Google. All a country has to do to find itself on the receiving end of punishing Chinese trade actions is being mildly critical of China in the wrong way, and next thing they know their exporters to China report being held up for months. The EU has taken offense at how China kicks around smaller EU members, Australia has been on the receiving end of it more than once.

If you'd said the EU might be seen as a relatively more stable trading partner, that may be true, the EU does seem to be fairly happy deindustrializing itself. Though it's fueling a rise in the fractured politics there too.

I've been thinking for a while that all of this is just trying to grasp tighter the last bits of sand escaping between our fingers. The end game, perhaps, is trust. Do you trust or know the source? If you don't, assume it was AI generated. If you do, accept it as authentic based on whatever they disclose, but know that it's possible they aren't being totally honest or were themselves fooled in some way, depending on the context.

Then build our assumptions and how we operate around those trust levels in the digital realm.

There's competition, but no one is interested.

TBH, I blame IT leadership. The easy choice is Google or Microsoft. "No one ever got fired for buying IBM" is now "No one ever got fired for subbing to Office." Someone needs to bite the bullet and stitch together an alternative array of services. They've cultivated their own monoculture by hitting the easy-button and forking over for Office365, etc.

Which if we're being honest, Microsoft and Google products are rarely best-in-class at any one thing that they do. Teams, SharePoint, Google Docs, someone somewhere is doing those things much better. Those competitors just don't necessarily have a whole ecosystem that makes it a big Easy Button for corporate IT to smash. And they sometimes charge as much by themselves as Office365 costs in whole. But I know one problem my own employers IT team struggles with a bit is putting out the constant fire of different teams deciding the Microsoft tool we pay for sucks and going rogue and trying to subscribe to something else. Dropbox was a big one, but Teams and Planner alternatives come up a lot too.

Not pretending to have all the answers but it doesn't seem to be a completely intractable problem.

No, we do not in fact have the "power pipes". Those "pipes" have capacity limits. Those "pipes" and getting them approved and built in the US is a key limiting factor that throttles... pretty much everything. Yes, it holds up data centers, but also I've heard 'green' energy advocates complaining for decades about how difficult it is to get the permits and approvals to get wind and solar facilities hooked up, they're holding up nuclear plants, they're holding all kinds of things up.

There's also something to be said for distributed production and storage. We don't all live in ultra-compact cities. I agree apartment buildings don't need to be playing this game. But Australia (and the US) are extremely dispersed relative to Europe, for example. There should, in theory, be resilience in distributed production and storage that grid managers could call upon to help manage point failures.

Temu built out some local distribution after it had already reached a pretty high level of popularity, but Temu was enabled by regulatory and political indifference that allowed China, with an enormous GDP, to get extremely subsidized postage rates. Also, lax enforcement of import duties (do you think the declared values were accurate?), and effectively zero enforcement of any safety standards, etc., on the products sold.

So, it was less of them following a path Amazon had trod and was maintaining, and more like sneaking in a back door and trying to sell you a counterfeit purse.

But if the US and EU had been on their game, it'd of either not happened or would've at least happened much more slowly.

So, steal from everyone (taxation) whether they want to support or use government-run Facebook, give the money to government-run Facebook, and everything will be great because Government?

What incentive does that mechanism have to spur innovation for government-run Facebook? Does the government fund two Facebooks and scale funding based on market share? Or, like what virtually always happens in government, does it metastasize, realize it's actual "customer" isn't its "users" but is actually the government itself, and act accordingly to protect and grow its own budget?

Everything about a subscription model is better. If it sucks, don't pay for it, and if no one thinks its a worthwhile bargain then they run out of money and it dies. Efficient. If people do like it, then great.

Lets be fair, here. Traditional TV was ~40 minutes of content with 20 minutes of ads. I still remember televised holiday movies "presented with limited commercial breaks by Folgers, where the best part of waking up..." or whoever was sponsoring it. Radio stations had to power their transmitters and pay their staff, so they had ads. Newspapers use ads to keep themselves affordable. This has always been a business model.

If it's simply pay to win then the value of a degree from an institution falls to absolutely zero. If the product a student is buying is a credential that's supposed to say "I know and can do things, pay me money please" then in order for that credential to fulfill its objective then low performing and mediocre students ("weeds") need to be filtered out. Otherwise, employers and the wider world will quickly realize the credential is meaningless.

See the problem here?

But it's not true. Cute to say that, but I wouldn't believe it. Just like most students, given the opportunity to use AI to essentially cheat will often do so (look at the stunning percentages professors are detecting when they bother to try to figure it out), the same applies here; given the option to study these things more deeply or fire up Tinder, Facebook, Minecraft or whatever, they're going to do the latter.

Yes, same here, for me high school was easy and I failed (or effectively failed, a D counted the same as an F) a couple classes right out of the gate. It was a lesson to myself that I never learned to study in high school and had to learn that skill ASAP.

Michigan and its apologists here would rob these kids of real-world experience, reducing them to infants. And then we're all disappointed when these adults act like infants.