rfoo

> It's known that such tricks reduce accuracy

AFAIU, speculative decoding (and this fancier version of spec. decoding) does not reduce accuracy.

If you compare "schema validation error count" plus "Count of Finish Reason others" then SiliconFlow and Infinigence is in the same bucket too. Maybe their API layer detected incorrect tool call and set finish reason to something else?

IMO this likely is what you get from running the model correctly as-is (i.e. using the same weight and activation dtype), so Together is not bad.

Moonshot AI themselves and Groq likely uses some sampler tricks to eliminate schema validation errors.

So really the only thing this shows is: Nebius, Chutes, AtlasCloud could be running something else (for example further quantized model). Or bugs.

Graphene just made VoLTE / NR / VoNR toggle feature built-in to their OS.

China knew that chips were security and strategic issues maybe 20 years ago if not more, I wonder why they haven't done building equally capable factories on-shore.

That's cool. For those who happened to be in China right now, there's the same listing on a Chinese marketplace (idlefish) for only 480 CNY (~68 USD). That's... better than too good to be true. I'm going to have a try.

> and that's ironically the one feature they bought out and integrated (Supermaven) instead of developing themselves

What? Cursor bought Supermaven last November and I have been using their much superior (compared to GH Copilot) completion since maybe early last year so it does not add up.

It could be a better draft model than separately trained EAGLE etc for speculative decoding.

> It still seems strange. A big part of GrapheneOS is to provide a safeguard from Googles data hoarding, yet it works primarily on Google phones.

That's the most confusing part. IMO GrapheneOS is not mainly about "provide a safeguard from Googles data hoarding", instead this is more like a side quest.

GrapheneOS is about creating a mobile OS that is more secure against advanced threats [0] than anything else, including stock Pixel OS and iOS.

[0] Currently my rule of thumb is, anyone who can find and write exploits for new memory corruption bugs for the wanted attack surface, or who can buy such capability, qualifies as advanced threat. Hence Cellebrite qualifies as a borderline "advanced threat".

Well, anyone with actual root on a secure (locked, verified boot on) Android phone can hard brick it with a single command. Yes, you can yell at the user telling them it's their fault. Still something you usually do not want to support.

I don't think having authorized temporary root is inherently insecure, but on the other hand making sure it is secure could be a huge time sink.

Now, the original request here, modifying user app (I'd assume it's not system app) data, is reasonable. Designing a properly authenticated way to allow doing so would be an interesting challenge.

Then refresh rate is a problem.

Spoiler: they went to Thailand.

You are absolutely right! But China bad Dario good Anthropic the only firm caring about AI safety /s

How what?

The fp64 GFLOPS per watt metric in the post is almost entirely meaningless to compare between these accelerators and NVIDIA GPUs, for example it says

> Hopper H200 is 47.9 gigaflops per watt at FP64 (33.5 teraflops divided by 700 watts)

But then if you consider H100 PCIe [0] instead, it's going to be 26000/350 = 74.29 GFLOPS per watt. If you go look harder you can find ones with better on-paper fp64 performance, for example AMD MI300X has 81.7 TFLOPs with typical board power of "750W Peak", which gives 108.9 GFLOPS per watt.

The truth is the power allocation of most GPGPUs are heavily tilted for Tensor usages. This has been the trend well before B300.

That's all for HPC.

And Pezy processors are certainly not designed for "AI" (i.e. linear algebra with lower input precision). For AI inference starting from 2020 everyone is talking about how many T(FL)OPS per watt, not G.

[0] which is a nerfed version of H200's precursor.

<rant>The official Arrow C++ implementation is just ergonomic warts, full of `const std::shared_ptr<T>&` bs. Trying to use it to manipulate data always give me headache telling apart WTH is an Array, ArrayData, Buffer, and the typed Array interfaces are barely usable. The original official Rust port inherited all the mis-designs too. On the Rust side someone created arrow2 [0] to fix it.</rant>

And I'm glad there's a good C++ impl too.

[0] https://github.com/jorgecarleitao/arrow2

This is the extracted Arrow C data interfaces as documented in https://arrow.apache.org/docs/format/CDataInterface.html

It's not how you interact with the data in your own C++ code, it's for passing this data to other in-process consumers (libraries etc). While in the example it calls the release function, this is usually just passed to a downstream consumer and it's their responsibility to call it.

I agree that having such an example as the first one is confusing. Given that a large part of the point of Apache Arrow is passing data columnar data between libraries in different languages in memory, it makes some sense.

It's more of a culture thing. People just hate the concept of "idk how much I'm going to pay let's just try this and find out later".

Also people would be confused as they expect things to be prepaid, so if you let them use the service they'd think it's a free trial or something, unless you literally put very big, clear price tag and require like triple confirmation. If not and you ask them to pay later they would perceive this as unfair deceptive tricks, and may scam you by report the loss of their credit card (!), because apparently disputing transactions in China is super hard.

You can try your solutions here: https://wf25open.kattis.com/contests/wf25open

I believe it runs with the same test data as in the actual contest.

meret is really good tho.

For a whim I read this as "us regular boeing engineers" and it was really funny.

IMO the correct thing to do to make these people happy, while being sane, is - do not build llama.cpp on their system. Instead, bundle a portable llama.cpp binary along with unsloth, so that when they install unsloth with `pip` (or `uv`) they get it.

Some people may prefer using whatever llama.cpp in $PATH, it's okay to support that, though I'd say doing so may lead to more confused noob users spam - they may just have an outdated version lurking in $PATH.

Doing so makes unsloth wheel platform-dependent, if this is too much of a burden, then maybe you can just package llama.cpp binary and have it on PyPI, like how scipy guys maintain a https://pypi.org/project/cmake/ on PyPI (yes, you can `pip install cmake`), and then depends on it (maybe in an optional group, I see you already have a lot due to cuda shit).

Slapping Rc<T> over something that could be clearly uniquely owned is a sign of very poorly designed lifetime rules / system.

And yes, for now async Rust is full of unnecessary Arc<T> and is very poorly made.

> and frankly, it likely will only need to work until the bubble bursts, making "the long run" irrelevant

Now I get why people are so weirdly being dismissive about the whole thing. Good luck, it's not going to "burst" any time soon.

Or rather, a "burst" would not change the world in the direction you want it to be.

Scrapers do not care about having a 20% slowdown. All they care is being able to scale up. This does not block any scale up attempt.

... except when you do not crawl with a browser at all. It's so trivial to solve just like the taviso post demostrated.

This makes zero sense, this is simply the wrong approach. Already tired of saying so and been attacked. So I'm glad professional-random-Internet-bullshit-ignorer Tavis Ormandy wrote this one.

Pretty sure it's an incident.

For me it's my most used super long command line flag.

For a brief moment `--break-system-packages` surpassed it, then I discovered `pip` accepts abbrev flags so `--br` is enough, and sounds like bruh.

> There are a lot of countries still where you're limited to feature phones

I thought China fixed this for most of the world, at least for Africa it's fixed. It's the Internet access being the bottleneck now.

Or, you can say, OpenAI has some real technical advancements on stuff besides attn architecture. GQA8, alternating SWA 128 / full attn do all seem conventional. Basically they are showing us that "no secret sauce in model arch you guys just sucks at mid/post-training", or they want us to believe this.

The model is pretty sparse tho, 32:1.

> The only thing that I can say definitively is that there is overhead to doing the literal stack switch. There's a reason async I/O got us past the C10k problem so handily.

You can also say that not having to constantly allocate & deallocate stuff and rely on a bump allocator (the stack) most of the time more than compensate for the stack switch overhead. Depends on workload of course :p

IMO it's more about memory and nowadays it might just be path dependence. Back in C10k days address spaces were 32-bit (ok 31-bit really), and 2**31 / 10k ~= 210KiB. Makes static-ish stack management really messy. So you really need to extract the (minimal) state explicitly and pack them on heap.

Now we happily run ASAN which allocates 1TiB (2**40) address space during startup for a bitmap of the entire AS (2**48) and nobody complains.

Good points.

Personally I'm more annoyed of async-Rust itself than not having a blessed async solution in-tree. Having to just Arc<T> away things here and there because you can't do thread::scope(f) honestly just demonstrates how stackless coroutine is unreasonably hard to everyone.

Back to the original topic, I bring this up because I believe the performance advantages claimed in these "fibers bad" papers are superficial, and the limit is almost the same (think 1.00 vs 1.02 level almost), even in languages which consider raw performance as a selling-point. In case you need the absolutely lowest overhead and latency you usually want the timing to be as deterministic as possible too, and it's not even a given in async-await solutions, you still need to be very careful about that.

Let alone Python.