binarymax

Yes. I found this out because I have a small tool I maintain for OpenAI API integration - and my integration test suite triggered this for the streaming case. The non-streaming cases didn't trigger the requirement.

FWIW I am definitely not opting in to this overreach and will use a different provider if necessary. I currently use GPT-4o for some streaming apps I have, but will migrate away from this requirement.

Lots of other options here if you don't like managing. You can use Elastic cloud, Bonsai.io, and others

Ultimately, (good) consulting just gets decisions made. Companies hire consultants when they don't know what the decision should be, or where to go next. Whether the decision is right or not, depends on the competence and experience of the firm (and also some luck).

Sure. This seems a cultural difference between US and EU. I've lived extensively in both. EU is known to be burdensome in terms of tech regulations. Normally I find this egregious. However when children are involved I am far more tolerant of their requirements. 10% OpEx overhead to make sure children are protected seems entirely reasonable to me. If you're not prepared to meet their requirements then skip the EU.

Can you blame them? To me, an AI grading platform for children seems like it should be regulated.

Our intern worked on a similar DSPy project last year for contracts, though the classifier was much simpler: binary classification of whether the line was the start of a new section or not. We saw some success, but it wasn’t a revelation, and in the end we ended up keeping our giant regex classifier since it was much faster and much cheaper (free). I do really like DSPy though - maybe looking again with a more well defined set of classes would work better.

I'm talking about perplexity's behavior. Perhaps there's a point of contention on perplexity downloading a document on a person's behalf. I view this as if there is a service running that does it for multiple people, then it's a bot.

There’s nothing wrong with downloading documents. I do this in my personal search app. But if you are hammering the site that wants you to calm down, or bypass robots.txt, that’s wrong.

Here's how perplexity works:

1) It takes your query, and given the complexity might expand it to several search queries using an LLM. ("rephrasing")

2) It runs queries against a web search index (I think it was using Bing or Brave at first, but they probably have their own by now), and uses an LLM to decide which are the best/most relevant documents. It starts writing a summary while it dives into sources (see next).

3) If necessary it will download full source documents that popped up in search to seed the context when generating a more in-depth summary/answer. They do this themselves because using OpenAI to do it is far more expensive.

#3 is the problem. Especially because SEO has really made it so the same sites pop up on top for certain classes of queries. (for example Reddit will be on top for product reviews alot). These sites operate on ad revenue so their incentive is to block. Perplexity does whatever they can in the game of sidestepping the sites' wishes. They are a bad actor.

EDIT: I should also add that Google, Bing, and others, always obey robots.txt and they are good netizens. They have enough scale and maturity to patiently crawl a site. I wholeheartedly agree that if an independent site is also a good netizen, they should not be blocked. If Perplexity is not obeying robots.txt and they are impatient, they should absolutely be blocked.

I've built and run a personal search engine, that can do pretty much what perplexity does from a basic standpoint. Testing with friends it gets about 50/50 preference for their queries vs Perplexity.

The engine can go and download pages for research. BUT, if it hits a captcha, or is otherwise blocked, then it bails out and moves on. It pisses me off that these companies are backed by billions in VC and they think they can do whatever they want.

I’ve not had good success with vibing rust. It requires lots and lots of handholding and editing. Perhaps it’s because the model is always trying to do things from scratch. It does a poor job of finding crates and understanding the docs and implementation.

This guy reddits

Would love to try this, but it seems like you hit a limit. Is there a working example you can link to?

Thanks for the response. This statement confuses me a bit. What is a relay? Does traffic go through it at all, or is it for connection negotiation, or some of both?

Does this require a 3rd party host, or is it peer-to-peer?

Does anyone have an API that maintains a list of all model versions for a provider? I hand-update OpenAI into a JSON file that I use for cost reporting in my apps (and in an npm package called llm-primitives).

Here's the current version:

    const pricesPerMillion = {
        "o1-2024-12-17": { input: 15.00, output: 60.00 },
        "o1-mini-2024-09-12": { input: 1.10, output: 4.40 },
        "o3-mini-2025-01-31": { input: 1.10, output: 4.40 },
        "gpt-4.5-preview-2025-02-27": { input: 75.00, output: 150.00 },
        "gpt-4o": { input: 5.00, output: 15.00 },
        "gpt-4o-2024-08-06": { input: 2.50, output: 10.00 },
        "gpt-4o-2024-05-13": { input: 5.00, output: 15.00 },
        "gpt-4o-mini": { input: 0.15, output: 0.60 },
        "gpt-4o-mini-2024-07-18": { input: 0.15, output: 0.60 },
        "gpt-4-0613": { input: 30.00, output: 60.00 },
        "gpt-4-turbo-2024-04-09": { input: 10.00, output: 30.00 },
        "gpt-3.5-turbo": { input: 0.003, output: 0.006 },
        "gpt-4.1": { input: 2.00, output: 8.00 },
        "gpt-4.1-2025-04-14": { input: 2.00, output: 8.00 },
        "gpt-4.1-mini": { input: 0.40, output: 1.60 },
        "gpt-4.1-mini-2025-04-14": { input: 0.40, output: 1.60 },
        "gpt-4.1-nano": { input: 0.10, output: 0.40 },
        "gpt-4.1-nano-2025-04-14": { input: 0.10, output: 0.40 },
        "gpt-4o-audio-preview-2024-12-17": { input: 2.50, output: 10.00 },
        "gpt-4o-realtime-preview-2024-12-17": { input: 5.00, output: 20.00 },
        "gpt-4o-mini-audio-preview-2024-12-17": { input: 0.15, output: 0.60 },
        "gpt-4o-mini-realtime-preview-2024-12-17": { input: 0.60, output: 2.40 },
        "o1-pro-2025-03-19": { input: 150.00, output: 600.00 },
        "o3-pro-2025-06-10": { input: 20.00, output: 80.00 },
        "o3-2025-04-16": { input: 2.00, output: 8.00 },
        "o4-mini-2025-04-16": { input: 1.10, output: 4.40 },
        "codex-mini-latest": { input: 1.50, output: 6.00 },
        "gpt-4o-mini-search-preview-2025-03-11": { input: 0.15, output: 0.60 },
        "gpt-4o-search-preview-2025-03-11": { input: 2.50, output: 10.00 },
        "computer-use-preview-2025-03-11": { input: 3.00, output: 12.00 }
    };

I would love to replace this with an API call.

The interesting thing to me is that another company is adopting the Elastic license.

Yes, work to revive it. But in 2025 politik the democrats should be shouting everywhere and everyway that maga has ruined this and they need to be constantly burning it in. Lack of message while working behind the scenes isn’t going to get wins during midterms.

You rent an a100x8 or higher and pay $10k a month in costs, which will work well if you have a whole team using it and you have the cash. I’ve seen people spending $200-500 per day on Claude code. So if this model is comparable to Opus then it’s worth it.

For anyone wondering (like me) what COP is in this context, it’s Coefficient of Performance: https://en.m.wikipedia.org/wiki/Coefficient_of_performance

Literally every other language with unsigned types handles this just fine?

I will die on the hill that Java is inferior because it doesn’t have native support for unsigned numerics.

It’s painfully slow (on a newish mobile). While the concept is great for dynamic data, static docs are a terrible use case.

But I can't avoid it. 2 of my 4 neighbors have this installed. So now, everytime we are outside, on my own property, we're being captured without consent.

Based on your history here it’s quite obvious you’re a musk fan. Maybe though, you should realize that a model being steerable to claim itself being mechahitler and proposing death to people is absolutely not a “good thing”. I suggest you seriously reconsider on what you’re advocating for here. Because the outcome of this will cost innocent lives.

You don’t need to but it certainly helped. And the Servo project also produced crates like HTML5ever which filled a huge gap in performant and safe document parsing for both browser and server. I’m very grateful that there are modular and easily usable projects like that, and rust is far easier to use than the complex c++ code and toolchains you’d need.

Not OP but generating puzzles isn’t hard if you have a database of games and I’ve done it myself.

For each game just fast forward to the end and ask for Stockfish to provide the solution. If it’s guaranteed “mate in X” then you’ve got yourself a puzzle. You can have a classifier that grabs other puzzle types too (“win the queen in X” for example)

“I’m an electron wizard. I write spells and magical constructs appear on the mirror slate”

No it doesn't disrupt. This is a well known capability of LLMs. Most models don't even point out a mistake they just carry on.

https://chatgpt.com/share/686b1092-4974-8010-9c33-86036c88e7...

Wow, this is pretty silly. If things are like this at Apple I’m not sure what to think.

https://github.com/BlueFalconHD/apple_generative_model_safet...

EDIT: just to be clear, things like this are easily bypassed. “Boris Johnson”=>”B0ris Johnson” will skip right over the regex and will be recognized just fine by an LLM.