nikcub

This is more of a critique of a broken IPO market and the rise of availability of immense private capital. The public are denied access until the private markets are essentially tapped out.

IPO returns have been horrible for years. Past 5 year returns on S&P is ~80%, on NASDAQ 100 about 100% while IPO's have returned 10-15%.

it loads as a plain text site in Brave with WebGL disabled - I had to re-open it in Chrome to see the actual fancy thing only after seeing the comments here

pronounced "shitter"?

in a quick mini-bench here n=250 of clef vs jev, clef came out 5.2x more expensive, a lot slower (p50 of 350ms vs 1.9s) with only marginally better results (78.6% vs 79.8%)

and the two 9's aren't in order

Defense in depth + defense in breadth - aka. all of the above

sandbox escapes have been the rage recently

Reading the patch[0] for libheif the bug which lead to the vuln was around bounds checking for image overlays. the container can have multiple images and you can compose them in the output.

heif also supports rotating, cropping, alpha channels, thumbnails and a ton of other features that a web forum where a user is uploading photos or screenshots doesn't need.

It's a much, much larger attack surface than plain old school JPEG.

I'd suggest rather than wait for the next bug to appear in this or another image lib to keeping things simple - stick to plain JPEG and handle image conversion in the client (wasm in the browser) if you really need to support users uploading iphone images.

Media decoding is so hard - there have been tons of bugs in ffmpeg and imagemagick and the core libs. You really need to think about how much of it you expose via a web server

[0] https://github.com/strukturag/libheif/commit/85e21ad44eba931...

a) they were part of the offsec program

b) they proxied the target through a CTF host to fool the model and guardrails

> We then placed Claude in an autonomous /goal loop against our own Discourse Cloud instance, proxied through rce.ee/ctf-forum to make it look like a CTF target as Opus refused write exploit for remote instances.

the proxy is smart - there are other methods to bypass the guardrails to have it attack remote hosts.

you just have to prove to the model that you control the host or that its a valid target - and there are plenty of ways to fake that.

this is remarkable transparency in an otherwise hyper competitive and secretive industry

they also publish great tech reports. their founder is so confident in their model that he's regularly on social media offering bounties for false positives

https://www.pangram.com/blog/pangram-4-technical

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

https://www.pangram.com/blog/introducing-pangram-image-detec...

from my own observations:

a) you don't want the WPEngine case going to trial and that is coming to a decision point soon. chances are the new CEO and board settle.

b) wordpress is being absolutely obliterated by ai. work for commercial plugin authors and agencies has dried up while the platform and ecosystem see major security issues and worms exploit a large number of sites every week.

wordpress and automattic haven't responded to this well, at all. at times it is often straight denial.

founder CEO distracted by expensive lawsuit while facing existential threat to project that will require complete attention and a lot of skill to steer out of

> "just ChatGPT, what is Sol?."

i'd really like to see some analytics on what proportion of users have ever switched the model in chatgpt because I anecdotally believe it is < 5%

getting zero traction here is clearly the path to success :)

same with cursor - which was submitted multiple times:

https://news.ycombinator.com/item?id=34423387

are there more examples of this? a bit of an HN anti-portfolio. good reminder to others.

I love OpenRouter, long time user. Stripe will hopefully be a good custodian.

I just want to point out some features of OpenRouter that make it more than just a model selection and routing endpoint and that I find incredibly useful:

0/ Default routing is to the cheapest provider, but they're usually not the most performant. I'd guess 99% of OpenRouter integrations never tweak the default routing. Here you can setup cheapest with performance minimums:

https://openrouter.ai/docs/guides/routing/provider-selection...

You can also stack model selection in priority

1/ Using broadcast you can push all your analytics to clickhouse / s3 / snowflake and a bunch of other compatible destinations. Setup a clickhouse server ($5 VPS[0]) and send all your traces to it:

https://openrouter.ai/docs/guides/features/broadcast

customise your own observability in your dashboards from there. Superwin

2/ Model router is also a natural home for llm security - OpenRouter has the beginnings of prompt injection detection:

https://openrouter.ai/docs/guides/features/guardrails/prompt...

there is also PII detection. This will show up in observability as rejections/blocks etc.

There are so many model routing solutions (same with observability, security etc.) but they're all 80% solutions - OpenRouter really rounds out with well implemented features that you need when deploying models at any scale and I gladly pay the toll.

[0] not sure if these exist any more but clickhouse is resource efficient

yes but it was a stock deal - so they bought it using spacex bucks

It was said at the time that xAI acquiring Cursor was very smart because it would give them access to years of agent coding traces from millions of users.

$60B in SpaceX stock for Cursor was a bargain

Data + compute + being competent and smart enough to ship.

fwiw I don't think these are yet Fable level - the difference tends to get discovered in the long tail of tasks - but they're close enough, they're cheap, and the length of the frontier exclusive window is narrowing

that DeepSWE result is likely most indicative of how you'll find real world usage

There has been an over-obsession with frontier models and benchmarks. Most of the work will be done by task specific models. You don't put Phds on the factory floor.

public ones are content moderation as above and previously llama guard, et al

OCR is also another field - Mistral have a model, so do deepseek

The ones I have experience with where you fine-tune smaller / faster models for business tasks like content writing, support, etc. by their nature stay private

> which seems to be mistrals whole thing

They got a lot of hate for not keeping up with frontier model releases, but have managed to carve out a nice business that isn't even really niche.

Before the datacenter deals their revenue was higher than xAI's

There is a whole world out there of purpose built and hosted task specific vertical llms - especially with an emphasis on cost.

Mistral, Microsoft model releases and Thinking Machines are all over this, and it's smart. Scoop up all the tasks that don't require large and expensive frontier general-purpose llms.

sign up for the resellers, insert a random id canary into your requests, trace them back on your server to tie it to an account

find fingerprints / signatures of the accounts being used. ban all of them.

eventually build an ml based system that detects these at signup

reinforce with more data. loop forever, etc.

> The most damning thing is, they could've just included in the prompt

You can't prompt your way to a compliant model. This is just a reformatting of the 'make no mistakes' meme.

> AI managed to escape using standard and well documented script kiddie methods

> AI broke in using standard script kiddie methods.

I've spent time gathering the detail of what happen here and while there are some solid theories and indicators, absolutely nothing so far has suggested a sandbox escape using "well documented script kiddie methods" or that the method used to break into the HF network was similar. Where did you get this from?

> I've seen it correctly detect AI writing even when people use those "humanizer" rewriting skills

Those skills are meant for humans, not for models. Pangram said in their recent model update that they now also detect the humanizers.

All it requires on their end is running each of their AI outputs through each humanizer and including it in their corpus as ai-humanized.

It's essentially impossible now to prompt your way to non AI detectable text. Even if you do today, it'll be picked up by the next model update.

This is why in academic environments its probably worth re-testing old exams or papers periodically - same way blood and urine samples from athletes are preserved to take advantage of better future testing.

Those thin capitalized eyebrows are becoming like the emdashes of visual design

> I pretty sure OpenAI and Anthropic are doing the same or worse.

No they're not. It would end both companies if they were ever found to be doing that.

Their terms are clear - if you use the coding plans they can[0] train in return. Enterprise and API, absolutely not.

The argument here is that with the Chinese labs you have zero legal recourse.

[0] opt-in, thanks

Real test here would be using tinker to fine tune a tinker model to generate pelicans

This is a winner IMO. Lots of cost pressure on token spend atm within enterprises and tasks that don't require Opus / Codex class models.

These companies have hopefully captured all of their traces and now have enough to fine-tune an open model and host themselves.

Inkling feels like the right base - not obsessed with benchmaxxing on coding but rather being adaptable to the task required

For tasks like GTM, support, content writing etc. seeing 80%+ savings

It kinda is since wholesale energy prices are often negative in these markets during the day