boomskats

I too am exceptionally happy with Niri. Could you expand on your comment around functionality and the basic features you're talking about?

What it did you try, and what are your use cases? Genuinely curious.

100% agree. A local model might not have the same stretch goal capabilities, but at least I don't have to constantly second guess myself in terms of the inconsistencies I think I'm noticing.

No longer am I constantly made to feel like Ingrid Bergman in that movie with all the lights.

Which only _they_ do. And _we_ don't.

After all, _their_ AI is just trained on stolen IP, right? Unlike _ours_?

I still can't get over how irresponsible it is of _them_ to release these weights for free. That means _anyone_ could gain access to their AI model now.

At least _we_ make sure access to _ours_ is fully controlled by _us_.

I remember seeing somewhere credible that it was going to be Thursday this week (tomorrow?), but don't remember where.

Assume you've seen the Apsara conf announcement / the planned 'tiers'[0]. Do you have a planned use case for a 4b / 9b? Hopefully the overthinking will get better on the smaller models.

[0]: https://www.yottalabs.ai/post/qwen-4-max-vs-plus-vs-flash-vs...

Hmm.

I do wonder if all of this would be possible had they not spent all that time secretly distilling of all our superior, western, closed-source AI models.

I was expecting there to be at least one yo dawg xzibit meme there.

I sent you a docs PR, hope it helps.

I'm really sorry but I've read this comment 3 times now and I still can't make sense of it

> The US withdrew its signature all the way back in 2002.

Yeah, I guess international law was rather incompatible with their ambitions at the time. Fair play to them though, they certainly made the most of it.

I expect the majority of Oxide's customers are actively trying to escape Broadcom's VMWare hell. Can't see how something like that would make sense.

This needs a side-by-side config comparison with something like echarts config schema format. I really don't see the point.

I'm 99% sure the verbosity required in the system prompt to teach non-M$ models this new ever-so-slightly-different-but-not-obviously-necessary chart def abstraction format, and the iterations required to get it right, will outweigh any supposed efficiency gains resulting from using it.

Just stating the obvious.

Right, so the 1.4 TB weight footprint straight away makes me wonder how much more important load/unload time becomes in terms of the neocloud product here. With 1.4TB of weights, a fast-enough cold start would basically be _the_ product; fast enough weight-swapping that can make K3 into something you could usefully rent by the hour/workday.

Interesting because again, the license Kimi shipped under [0] defines "Model as a Service" as

> giving a third party access to language model inference or fine-tuning (e.g., via API) in a manner that allows such third party to exercise meaningful control over the inputs, parameters, or training data

and then they have that clause around if you operate such a business above $20M aggregate revenue you need a separate agreement with Moonshot before commercial use, which presumably captures the majority of the larger neoclouds best placed to optimise this.

But where's the line? Say you offer infra optimised for GPU inference, warm pools, isolation per customer, exposed control plane, billed per GPU-hour rather than token, the invoice says compute rather than calls. The customer arguably 'self-hosts', you're probably fine? And if you as a provider run the serving stack and hand your customer an inference API, you're inside the definition regardless of whether you charge by the second or by the token. But what about if you give them direct hardware access, but have the weights cached on NVMe / ramdisk hyperlocal to the infra they're renting so that their hf cli pull only takes a few seconds? Sure, a managed warm pool of GPUs with K3 pre-loaded probably isn't ok, but a local hugging face lru cache holding 'whatever your customers pull down most often', superoptimised for fast weight swaps that the customer controls... is? Is it?

Again, where's the line? Is it materially different from a local docker registry mirror? What about safetensors checkpoints pre-sharded for the specific hardware topology you're offering? Does it matter whether you perform the checkpoint optimisation yourself and make it available, or merely cache one published on HF that happens to target exactly the hardware you rent out? What if you published that checkpoint yourself?

I'm definitely overthinking this, and I'm sure there's been conversation here about this already, but the other kimi threads[1] are enormous. And I am curious.

I'm also curious to know whether Moonshot would actually be against a setup like this. Guessing they would if it was AWS (not quite elastic but not that dissimilar), but what about others? Realistically I guess it'd be easier to just talk to them, especially if you were doing it in a way that targets a slice of the pie they never would have gotten anyway due to data residency requirements etc..

[0]: https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE [1]:https://news.ycombinator.com/item?id=49065752

Or against all those old books that they ingested and then literally destroyed

No! It's just not fair!! What about our financial bubble!

An act of G̶o̶d̶ Congress?

It's lightweight, capability-secured, services start up way faster, smaller overhead than VMs, better isolation than containers, faster replication, etc.

The original Wasmtime 1.0 blog post explains it all really well: https://bytecodealliance.org/articles/wasmtime-1-0-fast-safe...

Thanks for that link! There goes my Saturday morning.

After a bunch of middle clicking I landed here [0]. So if I understand correctly, the current stack switching proposal depended on exception handling to be implemented first for resume.throw, so that bit was blocked until now.

But it's also further along than you might assume [1], i.e. you can already invoke Wasmtime with `-W stack-switching` and hit the boundaries where the experimental implementation breaks.

Sadly though, like many ambitious Wasm things that don't have revenue directly attached to them, it now seems partly a case of finding someone willing to sponsor or finish the remaining work [2].

I also found the list of open stack-switching issues[3] useful.

(also, minor nit - Wasm != Wasmtime)

[0]: https://github.com/bytecodealliance/wasmtime/issues/10248

[1]: https://github.com/bytecodealliance/wasmtime/issues/12941

[2]: https://github.com/bytecodealliance/wasmtime/issues/12941#is...

[3]: https://github.com/bytecodealliance/wasmtime/issues?q=state%...

I would suggest you look at it a little more often than that - to improve the quality of discussion here, if for nothing else.

Ready for what?

WASM is already running in production at a whole bunch of financial services orgs and government infra.

The thing is, it's not running anywhere near HTML, CSS or JavaScript. It's running serverside, mostly on Wasmtime - which, as it happens, is what this post is all about.

Right, Pravda. Four year old Hacker News account and this is the first comment?

The Guardian is only 'left leaning' in that it provides mildly controversial but reassuringly inconsequential opinions and sound bytes for people to regurgitate so they can convince themselves they've done something about that uncomfortable feeling they're carrying around. It's like a sports team. Sponsors and all.

Hah I guess in that way it is just like Pravda.

Maybe this is another one of Garry's 30k LOC/day vibecode specials that he's so proud of

Some of you might want to check out throwaway96/dejavuln-autoroot or the older throwaway96/faultmanager-autoroot, www.webosbrew.org/rooting and cani.rootmy.tv. Looks like it's patchy for the very latest models, but you might find the literature useful anyway.

And image tokenisation.

I have a personal fork of hyprvoice[0] which I use almost everywhere now (w/ the big cohere-transcribe running on a local vLLM instance). It does a similar thing, but that's not why I'm mentioning it; I think it's worth looking at because it's a clean reference for the few elegant ways you can implement text injection in modern Linux (wayland).

It supports ydotool[1], wtype[2] and "clipboard fallback with clipboard restore". The first two you can probably think of as AHK equivalents - they wire in at the input layer and inject keystrokes when injecting text. wtype is wayland-only and a bit less invasive, ydotool supports non-wayland also apparently, but I haven't tried it. Neither approach provides 'instant text' - you have to watch the text get typed out, and you don't touch your keyboard while it's happening; the clipboard implementation is fallback for a reason as it's the least reliable. The first two work 'well enough' though, and are fairly tunable.

The other thing hyprvoice does in probably the most linux-friendly and universal way is the 'hotkey handling'. The server creates a socket in /tmp that the cli can then ping when the user triggers the start/stop/cancel, and they do this by binding whatever their DE's keyboard shortcut mapping mechanism is to trigger `hyprvoice toggle` as a background shell command. This works extremely well and is much cheaper than you'd intuitively think coming from Windows. This way you don't have to interface with DE-specific global keyboard listeners etc, but leave that to the WM (that's not to say that your installer couldn't prompt the user to configure the keyboard shortcut for them with their detected WM, you just wouldn't do it in the software itself).

I haven't actually looked at your project in too much depth yet as I have a solution for this already, so apologies if none of the above is news to you. Hope it helps though - happy to poke around and contribute something if the gap's still there.

[0]: https://github.com/leonardotrapani/hyprvoice [1]: https://github.com/ReimuNotMoe/ydotool [2]: https://github.com/atx/wtype

Not great, but also not at all surprising.

Not sure about other solutions, but one suggested workaround here would be to silently uninstall Windows without consent.

That's what I mean though.

I've not used gnome for years, but I have a vague memory of gnome/mutter running on a single main thread which used to lock up quite a lot (javascript etc). And because in X it was X that used to manage things like rendering the mouse pointer every frame, whereas in Wayland it flipped to mutter having to do it directly, the stalls were way more obvious in wayland than X, which is where I think a lot of this perception came from.

Again, not sure how much of this is accurate, but that's the point I was trying to make.

A lot of people conflate Wayland being worse than X11 with Gnome on Wayland being worse than Gnome on X11.

Wayland has been great for me for a few years now. I don't use Gnome or nvidia though.

This is great! Those analytical workloads numbers are mad - I'd love to see the benches, and I'm happy to contribute to some of the profiling.

How does your thread-per-connection model compare to Heikki's proposal[0][1] from back in 2023?

[0]: https://www.postgresql.org/message-id/31cc6df9-53fe-3cd9-af5... [1]: https://www.youtube.com/watch?v=xLLakMmVtbY

The science might be legit here, but I'm getting really, really tired of the way every single piece of writing to come out of Anthropic is written in some kind of self-aggrandising, wooey wonderous 'our model has developed a genetic mutation that makes it have feelings' bs style. Regardless of what they're trying to communicate, those undertones are always there. It's annoying and disingenuous. Homeopathy 'this-water-has-feelings' level annoying. None of the other labs write like that.

They might as well change their name to Anthropomorphic at this point.