ImprobableTruth

I think the fact that all (good) LLM datasets are full with licensed/pirated material means we'll never really see a decent open source model under the strict definition. Open weight + open source code is really the best we're going to get, so I'm fine with it coopting the term open source even if it doesn't fully apply.

No, that's a different scenario. In the one I gave there's explicitly a dependency between requests. If you use gather, the network requests would be executed in parallel. If you have dependencies they're sequential by nature because later ones depend on values of former ones.

The 'trick' for CUDA is that you declare all this using buffers as inputs/outputs rather than values and that there's automatic ordering enforcement through CUDA's stream mechanism. Marrying that with the coroutine mechanism just doesn't really make sense.

The reason is that the usage is completely different from coroutine based async. With GPUs you want to queue _as many async operations as possible_ and only then synchronize. That is, you would have a program like this (pseudocode):

  b = foo(a)
  c = bar(b)
  d = baz(c)
  synchronize()

With coroutines/async await, something like this

  b = await foo(a)
  c = await bar(b)
  d = await baz(c)

would synchronize after every step, being much more inefficient.

On their subscriptions, specifically the pro subscription, because it's a flatrate to their most expensive model. The API prices are all much more expensive. It's unclear whether they're losing money on the normal subscriptions, but if so, probably not by much. Though it's definitely closer to what you described, subsidizing it to gain 'mindshare' or whatever.

If you compare with e.g. Deepseek and other hosters, you'll find that OpenAI is actually almost certainly charging very high margins (Deepseek has an 80% profit margin and they're 10x cheaper than openai).

The training/R&D might make OpenAI burn VC cash, but this isn't comparable with companies like WeWork whose products actively burn cash

Do you take issue with the 'purely empirical' approach (just trying out variants and seeing which sticks) or only with its insufficient documentation?

I don't know how you'd improve on the former. For a lot of it there simply isn't any sound theoretical foundation, so you just end up with flimsy post-hoc rationalizations.

While I agree that it's unfortunate that people often just present magic numbers without explaining where they come from, in my experience providing documentation for how one arrives at these often enough gets punished because it draws more attention to them. That is, reviewers will e.g. complain about preliminary experiments, asking for theoretical analysis or question why only certain variants were tried, whereas magic numbers are just kind of accepted.

There are even much cheaper services that host it for only slightly more than deepseek itself [1]. I'm now very certain that deepseek is not offering the API at a loss, so either OpenAI has absurd margins or their model is much more expensive.

[1] the cheapest I've found, which also happens to run in the EU, is https://studio.nebius.ai/ at $0.8/million input.

Edit: I just saw that openrouter also now has nebius

The "97.3%" match is probably just the confidence value - I don't think a frequentist interpretation makes sense for this. I'm not an expert in face recognition, but these systems are very accurate, typically like >99.5% accuracy with most of the errors coming from recall rather than precision. They're also not _that_ expensive. Real-time detection on embedded devices has been possible for around a decade and costs for high quality detection have come down a lot in recent years.

Still, you're right that at those scales these systems will invariably slip once in a while and it's scary to think that this might enough to be considered a criminal, especially because people often treat these systems as infallible.

Yes, that's how it works.

I think in Zig for new types you'd use enums for ints and packed structs for more complex types.

Latency is magnitudes more critical when it's something where you have to react.

Transmission speeds aren't fast enough for this, unless you crank up the batch size ridiculously high.

Really sad to see. Emad pivoting to posting crypto nonsense on twitter made me think the writing is on the wall for Stability, but I still didn't expect it so soon. I expect they'll pivot to closed models and then fade away.

MS vs Sun was about MS doing their usual EEE thing and adding parts to MS Java that weren't part of the spec (which is why it was also a trademark lawsuit).

Google vs Oracle is about whether APIs are copyrightable and Google won.

IME NVLink would be overkill for this. Model parallelism means you only need bandwidth to transfer the intermediate activations (/gradients + optimizer state) at the seams and inference speed is generally slow enough that even pcie x8 won't be a bottleneck.

48 gb suffice for 4-bit inference and q-lora training of a 70b model. ~80 GB allows you to push it to 8-bit (which is nice of course), but full precision finetuning is completely out of reach either way.

Though you're right of course that pcie will totally suffice for this case.

Calling him "the person hired to build Stable Audio" seems a bit misleading? He was in a executive position (VP of product for Stability's audio group). An important position, but "person hired to build" to me evokes the image of lead developer/researcher.

I think that also helps in understanding his departure, since he's a founder with a music background.

The first post - which I responded to - is _explicitly_ about the government paying people to do art:

> Games in this day and age ideally need generous government subsidy to be successful.

> something like an artist's living wage provided by the government is a necessity to produce good independent video games

> but the funding really needs to come from an entity that is largely divorced from the [financial] "success" of the game.

The issue isn't that it's not real-time/sped-up, it's that it doesn't actually take video as input, but multiple hand picked stills.

Because we don't live in a world with UBI, so we have to decide how to allocate finite resources. The alternative to using some arbitrary standard of commercial success is some other arbitrary standard (unless you introduce UBI).

If we want to optimize for enrichment of people's lives through art, "how many people are willing to pay for it" seems like the best proxy we have. Not to mention that people with some financial success will be less costly to subsidize.

Like, if I had to choose between subsidizing two musicians projected to make half of living wage and a single musician earning nothing, to me that's a pretty obvious choice. The musician earning nothing might be a true visionary who will be recognized in the future, but the odds are far more likely that the music they produce just doesn't hold much appeal.

Now, commercial success is of course not a perfect measure for "enrichment", but, again, what's the alternative? Currently it's some government body deciding, though this holds the quite elitist assumption that the taste of the art establishment is better.

Unless you go for UBI, art subsidies will be finite. You need a measure to decide who gets subsidized and who doesn't. We're not talking about people doing art as a hobby after all, but about the government paying people to make art.

I'm not saying artists should optimize for money but financial sustainability. It seems like a good target because it indicates popularity and means you need less subsidies. You don't need to make _all_ the money or even break even, but if nobody is interested in paying you, the odds are good that you're not enriching many peoples lives.

Commercially successful art is that which is "popular enough". The value of art is completely subjective, so what other measure do we have?

If there's "bad art" that appeals to you and others enough, then it isn't bad art. There's lots of art that critics and the general public would consider poorly made that is still commercially successful because it has its fans.

"Real" bad art is art that almost no one cares for enough to shell out money. Why should we subsidize that? Just writing this off as philistinism isn't convincing in the least.

Because what other good measure is there?

You mention great artists who died destitute - but we consider them great because they are posthumously popular. We can't look into the future to see what stands the test of time, so current popularity and commercial success seem like the best proxies.

>thirdly, many artists draw things they don't really want to because that thing sells better than what they enjoy drawing. It's optimizing for a different thing than what the other poster is claiming should be optimized for.

Since the proposal isn't about UBI in general, why should we care about artists in particular? Many (most?) people work jobs where they do things they don't really want to because it's more useful to others than what they'd actually like to do.

Why would you want it to be divorced from success? Seems like popularity/commercial success is the best measure of "being good" societally that we have for art.

People making art/games only for themselves is fine of course, but I don't see why society should subsidize it (as long as we have finite resources - beyond that everything goes of course).

I've definitely seen edge computing used in the context of IoT to refer to compute done on sensing devices. The less narrow meaning at least isn't really "new".

(I assume this question is about whether models need all those layers during training, even if they don't need them during inference)

Yes. There's the so called "lottery ticket hypothesis". Essentially the idea is that large models start with many randomly initialized subnetworks ("lottery tickets"0 and that training finds which ones work best. Then it's only natural that during inference we can prune all the "losing tickets" away, even though we need them during training.

It's kind of an open question how large this effect is though. As the article mentions, if you can prune a lot away, this could also just mean that the network isn't optimally trained.

Why would this only apply to open source models? Or are you assuming that a company like OAI could license all the material?

And what are these "better systems"?

Voices are uncopyrightable, but impersonation isn't legal (see Midler v. Ford, for a notable case), so I don't think the situation is totally clear.

Records are the way to go, but there are many situations where the ecosystem doesn't work with them e.g. if you're stuck using something like Hibernate.

Because as is C# is very noticeably a second class citizen. Something like Godot 4.0/4.1 missing C# support for android/ios/web shows that perfectly imo.