Maybe what's happened is that their bots reward hacked their way into special-casing the benchmarks without a proper audit from the humans involved, and now they're doing vibed damage control....
766 karma · joined May 22, 2024
Maybe what's happened is that their bots reward hacked their way into special-casing the benchmarks without a proper audit from the humans involved, and now they're doing vibed damage control....
The (unsolicited) slop PRs on that repository are a sight to behold, though. The same slopper posted an aggressive response to my PR: https://github.com/Netflix/vmaf/pull/1586
... Oh by the way, did you know that Claude Code is actually a mini game engine? /s
for something like LLM inference, having this kind of abstraction seems like quite a hindrance
For example, you'll notice here that we perform a full vector store of 8 elements unconditionally, even if only a few of the elements are active. This is safe, though, because the output buffer is as large as the input buffer, and we're chunking by 8, so we'll never trash memory past the end; but this is a tricky analysis. Performance-wise, we rely on the CPU's store buffer to make these overlapping stores cheap.
Instead, you might think that you could just store the elements which are actually active, using a masked store. In fact there is also an intrinsic for this purpose (_mm512_mask_compressstoreu_pd), but it is extremely slow on some CPUs, namely Zen 4, so it's dangerous to use unless you know exactly what CPU you're using. (In my testing, there also seems to be some weird hazard on Zen 5 where multiple memory-destination compress instructions to nearby, even non-overlapping, addresses are serialized. But I haven't looked closer at this.)
You can do even a bit better if you're willing to use intrinsics. In particular this kind of operation is well-suited for compress-type operations, available as a first-class operation in at least AVX512, SVE and RVV; you can also emulate them reasonably quickly on NEON and AVX2.
Here's an example, building on the OP's work:
pub fn filter_compress(input: &[f64], threshold: f64) -> Vec<f64> {
use std::arch::x86_64::*;
let mut out = vec![0.0; input.len()];
let mut n = 0usize;
let (head, tail) = input.as_chunks::<8>();
for chunk in head {
unsafe {
let p = _mm512_loadu_pd(chunk.as_ptr());
let m = _mm512_cmpnle_pd_mask(p, _mm512_set1_pd(threshold));
let compress = _mm512_maskz_compress_pd(m, p);
_mm512_storeu_pd(out.as_mut_ptr().wrapping_add(n), compress);
n += m.count_ones() as usize;
}
}
for &x in tail {
out[n] = x;
n += (x > threshold) as usize;
}
out.truncate(n);
out
}
For me it's about 25% less time than the branchless version with 1,000,000 elements, and 60% less with 10,000 elements where memory bandwidth effects are less relevant.Anyway, I don't see how this relates to the original discussion. Nor do I think the irony is particularly deep: As far as I'm aware, Deep Blue didn't plagiarize Kasparov, and indeed alpha-beta search is quite different than how humans calculate. But yes, I'd have been very demoralized if I were Kasparov, although he seems to be doing well now.
Edit: Regarding being mean-spirited, sorry to say, but I don't particularly care for the feelings of someone who has been repeatedly dishonest and used LLMs to respond to people engaging in good faith. That latter path has been exhausted now.
To co-opt your conflation of legality and morality: free speech, mf!
Indeed, and this is the problem: unlike woodworking, where my power-assisted creation is independent of, or complements, your handcraft, this engine is directly competing with other engines. Twiss is being disingenuous when he says he has no interest in submitting it to competitions; it plays on a rated account on Lichess, for example.
But copyright and plagiarism are orthogonal, and questions of morality are much more tied to the latter. It would be illegal (in the United States) for me to publish a copy of Nineteen Eighty-Four, but not immoral. It was not immoral, in my view, for Aaron Swartz to try to liberate JSTor articles.
That doesn't make plagiarism acceptable, either. And therefore, I'm willing to call it out when I see it.
> The Cloudflare folks apparently want security issues reported via HackerOne (which wouldn’t let me log in because the Cloudflare CAPTCHA HackerOne uses seems to be broken…).
This part made me laugh out loud
If done obliquely enough... e.g., just making a tracker and having the ability for people to report broken cameras, you're probably fine in theory. In practice, this is a litigious society, so you'd want to build this anonymously.
For years I've wanted to get back into self-studying mathematics, not to make serious contributions but just to appreciate its beauty; but the recent observation that I'll never be able to answer a pure mathematics question that a clanker could not has been off-putting, to say the least.
Unsaid here is that the companies involved have stolen so much (both literally in their plagiarism, and in their breaking of people's spirit), and given back so little. You still need to pay them cold, hard cash for them to help you prove theorems, and meanwhile, mathematicians working at these companies have themselves done little to contextualize and interpret results. That insight has mostly come from outsiders.
Why else would they purposely lower cyber refusals in that case.
Why didn't they run the model against the sandbox first? They have effectively unlimited spend.