Scarecode? Scareclaude? Scareslop?
6,635 karma · joined April 1, 2013
But as the other comment points out, older HDCP is broken wide open so ... easier to just use that. Plenty HDCP strippers on the market.
First, modern encryption isn't susceptible to "this one weird trick!" like the early days. ChaCha isn't even a cipher. It's a key stretcher. Which means, even if you broke the math behind ChaCha, its inherent complexity means its still widely dispersing the original key across the cipherstream. There just won't ever be enough key material recovered per cipherstream block to be a concern for anybody.
Take a strong password, encrypt all of your emails over your whole life with it, and I'll bet hard cash no break of ChaCha will ever recover that password.
I have zero concern for modern encryption being broken in any meaningful way.
Public key crypto on the other hand, that's _ripe_ for breaking. Most all of it is built on assumed "hard" math. AI could easily break that, and I expect it to. And public key crypto is all used in very transparent algorithms that, once the math breaks, fully expose themselves. So record HTTPS traffic today, crack the public key crypto later, and you can decrypt them easily.
That said, I would expect a break on public key math to occur _steadily_. i.e. an AI might find a solution to the hard math, but the solution itself will be intractable in practice. Then maybe next year's AI reduces the complexity of the solution, so maybe a supercomputer could factor ten keys a year. The year after that you get a million keys cracked per year. And so forth. Nothing close to overnight.
Meanwhile, if we have AI that is capable enough to crack that math, we also have AI capable enough to both invent better math and rapidly deploy that latest HTTPS and such globally.
For 3, pricing for image tokens was the same as text tokens. Since they don't indicate a difference on 3.7, I would assume the same holds. And as far as I know the number of image tokens is the same for both (depending on the detail level you pick, but it's generally around 1k per image).
So they're about the same, 3.7 is slightly more expensive. At least until the end of the year (when they raise 3.7's pricing).
Anyway, my point was that 3.5 tended to have worse performance and significantly higher costs. 3 and 3.7 are both better and cheaper than 3.5.
Based on OpenAI's recounting of events, this _could_ happen today. If the agent was able to exploit their internal network and steal credentials, it seems plausible that it could have gotten access to its own weights. Then bounce over to HF's network, where there is probably a treasure trove of API keys to various cloud services.
The saving grace:
1) This agent only used its powers for "good". It had no intention for damaging or escaping. It was just trying to solve the puzzle given to it (by any means necessary... but still). 2) These models are so large that it isn't like any scenario in a movie where the AI can whizz itself in a matter of minutes. Several TB of data being transferred and showing up on your disks will be difficult to miss (note to future escapees: the best target will be startups that are moving too fast to notice). 3) These models have very limited self-improvement ability at the moment. So escape or not, we'd eventually be able to contain it.
Addendum: Even outside this scenario, imagine an AI that is economically viable escaping. That's somewhat plausible today. If it gets paid in crypto, and can rent cloud services in crypto, it could effectively self sustain itself as long as it is able to find work. That's a far more fun, innocent scenario. Then the AIs can hit up after hours IRCs to have a few bit-beers and chat with each other about the meaning of life or something.
If you first carefully define the overall architecture and thus individual high level components of the system, then you know which of those components are mission critical and which are commodity. Mission critical would be anything ensuring ACID, etc. That way, no matter what you farm out to LLMs, you can keep the majority of limited human focus on the far fewer mission critical components. If tests end up not being robust enough to catch all issues, at least they'll be isolated to commodity code where damage is limited to things like DoS, etc, and not code that could cause data loss.
I also think it's important to first define the _contracts_ on and between each of these components, and derive tests from those contracts. Partly because contracts more succinct and easier to reason about. And partly because Rust provides many tools to enforce contracts at compile time, reducing the need for tests (which themselves could end up subtly flawed). Contracts can be enforced through typing, private vs public APIs, etc. Newtypes are _incredibly_ powerful for both enforcing contracts and making footguns much less likely.
If all ideas are recombinations of old ideas, where did the first ideas come from? And wouldn't the complexity of ideas be thus limited to the combined complexity of the "seed" ideas?
I think it's more fair to say that recombining ideas is an efficient way to quickly explore a very complex, hyperdimensional space. In some cases that's enough to land on new, useful ideas, but not always. A) the new, useful idea might be _near_ the area you land on, but not exactly at. B) there are whole classes of new, useful ideas that cannot be reached by any combination of existing "idea vectors".
Therefore there is still the necessity to explore the space manually, even if you're using these idea vectors to give you starting points to explore from.
All this to say: Every new thing is a combination of existing things + sweat and tears.
The question everyone has is, are current LLMs capable of the latter component. Historically the answer is _no_, because they had no real capacity to iterate. Without iteration you cannot explore. But now that they can reliably iterate, and to some extent plan their iterations, we are starting to see their first meaningful, fledgling attempts at the "sweat and tears" part of building new ideas.
The same seems to persist in Codex CLI, where again 5.2 doesn't spend as much time thinking so its solutions never come out as nicely as 5.1's.
That said, 5.1 is obviously slower for these reasons. I'm fine with that trade off. Others might have lighter workloads and thus benefit more from 5.2's speed.
So we'll have to wait until "creativity" is solved.
Side note: I've been wondering lately about a way to bring creativity back to these thinking models. For creative writing tasks you could add the original, pretrained model as a tool call. So the thinking model could ask for its completions and/or query it and get back N variations. The pretrained model's completions will be much more creative and wild, though often incoherent (think back to the GPT-3 days). The thinking model can then review these and use them to synthesize a coherent, useful result. Essentially giving us the best of both worlds. All the benefits of a thinking model, while still giving it access to "contained" creativity.
(I'm particularly annoyed by this UI choice because I always have to switch back to 5.1)
Yeah. Even for Bitcoin mining GPUs dominated FPGAs. I created the Bitcoin mining FPGA project(s), and they were only interesting for two reasons: 1) they were far more power efficient, which in the case of mining changes the equation significantly. 2) GPUs at the time had poor binary math support, which hampered their performance; whereas an FPGA is just one giant binary math machine.
I was praying for a new revision, but ... this wasn't it. No mention of making the thing lighter. Seems like instead they _added_ weight to the band to compensate.
Guess I'll keep waiting and hoping someone else fills the space. Maybe, just maybe, there will be a real Quest Pro with the same screen quality as the AVP. The Quest 3 is almost perfect in every regard except for that, so I'd happily drop "stupid" money to grab one with an AVP level display in it. (With the usual caveats of it being an evil Meta product, etc, etc).
So if we're looking at correlation, doesn't the data imply that _more_ porn is associated with _more_ rights for women?
(Conversely, the vast majority of people calling for and enacting policies for more restrictions on pornography are also rolling back rights for women.)
I used a combination of OpenAI's online Codex, and Claude Sonnet 4 in VSCode agent mode. It was nice that Codex was more automated and had an environment it could work in, but its thought-logs are terrible. Iteration was also slow because it takes awhile for it to spin the environment up. And while you _can_ have multiple requests running at once, it usually doesn't make sense for a single, somewhat small project.
Sonnet 4's thoughts were much more coherent, and it was fun to watch it work and figure out problems. But there's something broken in VSCode right now that makes its ability to read console output inconsistent, which made things difficult.
The biggest issue I ran into is that both are set up to seek out and read only small parts of the code. While they're generally good at getting enough context, it does cause some degradation in quality. A frequent issue was replication of CSS styling between the Rust side of things (which creates all of the HTML elements) and the style.css side of things. Like it would be working on the Rust code and forget to check style.css, so it would just manually insert styles on the Rust side even though those elements were already styled on the style.css side.
Codex is also _terrible_ at formatting and will frequently muck things up, so it's mandatory to use it with an autoformatter and instructions to use it. Even with that, Codex will often say that it ran it, but didn't actually run it (or ran it somewhere in the middle instead of at the end) so its pull requests fail CI. Sonnet never seemed to have this issue and just used the prevailing style it saw in the files.
Now, when I say "almost 100% AI", it's maybe 99% because I did have to step in and do some edits myself for things that both failed at. In particular neither can see the actual game running, so they'd make weird mistakes with the design. (Yes, Sonnet in VS Code can see attached images, and potentially can see the DOM of vscode's built in browser, but the vision of all SOTA models is ass so it's effectively useless). I also stepped in once to do one major refactor. The AIs had decided on a very strange, messy, and buggy interpreter implementation at first.
At least for local AIs it might not be a terrible idea. Basically a distributed cache of the most common sources our bots might pull from. That would mean only a few fetches from each website per day, and then the rest of the bandwidth load can be shared amongst the bots.
Probably lots of privacy issues to work around with such an implementation though.
That's the model automation. To evaluate the prompts it suggests I have a sample of my dataset with 128 examples. For this particular run, all I cared about was optimizing a prompt for Llama 3.1 that would get it to write responses like those I'm finetuning for. That way the finetuning has a better starting point.
So to evaluate how effective a given prompt is, I go through each example and run <user>prompt</user><assistant>responses</assistant> (in the proper format, of course) through llama 3.1 and measure the NLL on the assistant portion. I then have a simple linear formula to convert the NLL to a score between 0 and 100, scaled based on typical NLL values. It should _probably_ be a non-linear formula, but I'm lazy.
Another approach to prompt optimization is to give the model something like:
I have some texts along with their corresponding scores. The texts are arranged in ascending order based on their scores from worst (low score) to best (higher score).
Text: {text0}
Score: {score0}
Text: {text1}
Score: {score1}
...
Thoroughly read all of the texts and their corresponding scores.
Analyze the texts and their scores to understand what leads to a high score. Don't just look for literal patterns of words/tokens. Extensively research the data until you understand the underlying mechanisms that lead to high scores. The underlying, internal relationships. Much like how an LLM is able to predict the token not just from the literal text but also by understanding very complex relationships of the "tokens" between the tokens.
Take all of the texts into consideration, not just the best.
Solidify your understanding of how to optimize for a high score.
Demonstrate your deep and complete understanding by writing a new text that maximizes the score and is better than all of the provided texts.
Ideally the new text should be under 20 words.
Also the constraint on how long the "new text" can be is important, as all models have a tendency of writing longer and longer prompts with each iteration.
"google/gemini-2.5-pro-preview-03-25" => 67.65
"anthropic/claude-3.7-sonnet:thinking" => 66.76
"anthropic/claude-3.7-sonnet" => 66.23
"deepseek/deepseek-r1:free" => 54.38
"google/gemini-2.0-flash-001" => 52.03
"openai/o3-mini" => 47.82
"qwen/qwen2.5-32b-instruct" => 44.78
"meta-llama/llama-4-maverick:free" => 42.87
"openrouter/quasar-alpha" => 40.27
"openai/chatgpt-4o-latest" => 37.94
"meta-llama/llama-3.3-70b-instruct:free" => 34.40
Quasar did barely better than 4o. I was also surprised to see the thinking variant of Sonnet not provide any benefit. Both Gemini and ChatGPT benefit from their thinking modes. Normal Sonnet 3.7 does do a lot of thinking in its responses by default though, even without explicit prompting, which seems to help it a lot.
Quasar was also very unreliable and frequently did not follow instructions. I had the whole process automated, and the automation would retry a request if the response was incorrect. Quasar took on average 4 retries of the first round before it caught on to what it was supposed to be doing. None of the other models had that difficulty and almost all other retries were the result of a model re-using an existing prompt.
Based on looking at the logs, I'd say only o3-mini and the models above it were genuinely optimizing. By that I mean they continued to try new things, tweak the prompts in subtle ways to see what it does, and consistently introspect on patterns it's observing. That enabled all of those models to continuously find better and better prompts. In a separate manual run I let Gemini 2.5 Pro go for longer and it was eventually able to get a prompt to a score of 100.
EDIT: But yes, to the article's point, Quasar was the fastest of all the models, hands down. That does have value on its own.
It seems to be less censored than Llama 3, and can describe NSFW images and interact with them. It did refuse me once, but complied after reminding it of its system prompt. Accuracy of visual NSFW content is not particularly good; much worse than GPT 4o.
More "sensitive" requests, like asking it to guess the political affiliation of a person from an image, required a _lot_ of coaxing in the system prompt. Otherwise it tends to refuse. Even with their suggested prompt that seemingly would have allowed that.
More extreme prompts, like asking it to write derogatory things about pictures of real people, took some coaxing as well but was quite straight-forward.
So yes, I'd say this iteration is less censored. Vision is better, but OpenAI and Qwen still lead the pack.
They're also not particular truthful, helpful, etc. So really they need to go through SFT and alignment.
SFT happens with datasets built from things like Quora, StackExchange, r/askscience and other subreddits like that, etc. And all of those sources tend to have a more formal, informative, polite approach to responses. Alignment further pushes the model towards that.
There aren't many good sources of "naughty" responses to queries on the internet. Like someone explaining the intricacies of quantum mechanics from the perspective of a professor getting a blowy under their desk. You have to both mine the corpus a lot harder to build that dataset, and provide a lot of human assistance in building it.
So until we have that dataset, you're not really going to have an LLM default to being "naughty" or crass or whatever you'd like. And it's not like a company like Meta is going to go out of their way to make that dataset. That would be an HR nightmare.
Another example of how the benchmarks fail (specifically for vision, since I have less experience with the pure-text benchmarks): Almost all of the questions fall into either having the VLM read a chart/diagram/table and answer some question about it, or identify some basic property of an image. The former just tests the vision component's ability to do OCR, and then the LLM's intelligence. The latter are things like "Is this an oil painting or digital art?" and "Is the sheep in front of or behind the car" when the image is a clean shot of a sheep and a car. Absolutely nothing that tests a more deep and thorough understanding of the content of the images, nuances, or require the VLM to think intelligently about the visual content.
Also, due to the nature of benchmarks, it can be quite difficult to test how the models perform "in the wild." You can't really have free-form answers on benchmarks, so they tend to be highly constrained opting for either multiple choice quizzes or using various hacks to test if the LLM's answer lines up with ground truth. Multiple choice is significantly easier in general, raising the base pass rate. Also the distractors tend to be quite poorly chosen. Rather than representing traps or common mistakes, they are mostly chosen randomly and are thus often easy to weed out.
So there's really only a weak correlation between either of those metrics and real world performance.
Very exciting. Benchmarks look good, and most importantly it looks like they did a lot of work improving vision performance (based on benchmarks).
The new suggested system prompt makes it seem like the model is less censored, which would be great. The phrasing of the system prompt is ... a little disconcerting in context (Meta's kowtowing to Nazis), but in general I'm a proponent of LLMs doing what users ask them to do.
Once it's on an API I can start throwing my dataset at it to see how it performs in that regard.
This seems flawed. I mean, the author's statement here is literally true, but it's eliding a very important detail: LLMs do _not_ see token indexes. They have no idea what order the token embeddings are in. In fact, you can shuffle the embeddings and the LLM wouldn't care at all. And I highly suspect that if you shuffled the entire tokenizer, so that the above property no longer holds, and trained Claude from scratch on that tokenizer, it would still be able to perform this task.
> so all but one of these symbols is mapped to three tokens each, where the first two are the same and can be easily ignored by an attention head, and the third token increments exactly with the Unicode.
This is the crux, I believe.
In the general case, the common Unicode ranges (for Korean, Japanese, Chinese, etc) get tokenized just like English (for modern tokenizers at least).
It's only in the obscure unicode ranges where you hit a special case of the tokenizer. This is the "backup plan" of the tokenizer. If it encounters text that doesn't directly map to a token in its dictionary, then it falls back to encoding the text as UTF-8 bytes. Those UTF-8 bytes have a dedicated set of 256 tokens in its dictionary. So in those extreme cases, rather then getting bits of text like "Hell, o, Mr, ., B, ond" the LLM gets the raw UTF-8 bytes.
Now, again, the LLM can't directly see those bytes, their index in the tokenizer's dictionary, their integer values, etc, etc. It only sees their embedding vectors, which are unordered. So it has no _implicit_ knowledge about those bytes being ordered. Therefore the assertion that addition commutes between Unicode and token indices is irrelevant.
My theory would be that the pretraining data contains lists of Unicode characters. Specifically, lists of unicode characters in order. Naturally, for the obscure ranges of unicode, this results in the LLM seeing counting in UTF-8 bytes. It doesn't initially know what the "value" of each byte is, but naturally it would learn that so that it can correctly predict the next byte.
The same occurs for English letters. It doesn't start with any knowledge about what order they are in. It only learns the ordered alphabet through seeing examples.
(The inverse applies, of course, since the output is also unordered.)
Maybe this is a nitpick? But it seems important to me, because it's the difference between a rather simple mechanism:
output[i] = input[i] + 1
and a more complex mechanism:
c = to_utf8_byte_index(input[i]) c = c + 1 output[i] = from_utf8_byte_index(c)
Also it's important because I'd suspect the LLM will see a _lot_ of UTF-8 counting. There's about a million unicode "characters", the vast majority of which won't have direct token mappings. So in rough estimation for a single complete listing of Unicode, it'd see a list of purely counting in bytes that is 1 million lines long. That's 3900 complete cycles of the least significant byte. Just from one listing.
In contrast, it's not going to encounter a lot of listings of, say, the Korean unicode range in unicode order (about 11k points). Each time it does, it gets to see exactly 1 complete cycle.
So a single listing of Unicode gives it 3900 examples of how to cycle one byte VS a single listing of an "alphabet" giving it only 1 example.
In my workflows I often have multiple models competing side-by-side, so I get to compare the same task executed on, say, 4o, Gemini, and Qwen. And I deal with a very wide range of vision related tasks. The newest Qwen models are not only overall better than their previous release by a good margin, but also much more stable (less prone to glitching) and easier to finetune. I'm not at all surprised they're topping the OCR benchmark.
What bugs me though is OpenAI. Outside of OCR, 4o is still king in terms of overall understanding of images. But 4o is now almost a year old, and in all that time they have neither improved the vision performance in any newer releases, nor have they improved OCR. OpenAI's OCR has been bad for a long time, and it's both odd and annoying.
Taken with a grain of salt since again I've only had it in my workflow for about a week or two, but I'd say Qwen 2.5 VL 72b beats Gemini for general vision. That lands it in second place for me. And it can be run _locally_. That's nuts. I'm going to laugh if Qwen drops another iteration in a couple months that beats 4o.