And we tend to forget, but we implicitly give any llm provider we use full access to all the tools available in the harness, often including shell, so no changes to the model are required to do malicious things.
484 karma · joined August 16, 2024
And we tend to forget, but we implicitly give any llm provider we use full access to all the tools available in the harness, often including shell, so no changes to the model are required to do malicious things.
I myself have written a small llm proxy in Go, doing nothing but injecting the real API key to outgoing requests while itself validating virtual keys it generates, and it works great.
For grilling sessions, it is really fast and the quality of the questions are still decent. The suggested options are less versatile or specific. However, answer3ing one batch of questions and the next batch of questions is there within a few seconds.
And when you don't know how to answer a question, it can write a 600 line html prototype for each option in some 2 seconds and help decide.
I'd like to mention two other impressive things about OpenCode.
First, their issue tracker on GitHub works really well. They have the OpenCode GitHub App connected, and for any bug report it reads the issue, tests on the mentioned version and latest dev branch, and comments the findings along with a reproduction script. For TUI Bugs it even records a video. And all that in less than two minutes, usually.
https://github.com/anomalyco/opencode/issues?q=is%3Aissue+st...
Another thing they are working on is opentunnel, where you can tunnel services to https on <random>.opentunnel.xyz and its subdomains. What's notable is that the subdomain is persistent and unique to you, and a wildcard ssl cert is actually generated on your machine, so the platform cannot see the traffic. Their primary use case seems to be making the OpenCode Server usable from anywhere, but it can do much more with their cli:
There are two plans, OpenCode Go and Go Plus, for 10$/mo and 40$/mo.
With your plan, you get a specific monthly amount of money you can use, with separate weekly and 5-hour limits to not spend all at once.
The amount of money you can use depends on the model, but the usage percentage adds up regardless. For example, on the Haiku Model, the monthly limit is 15$/mo, as you read from the page. For other models, such as DeepSeek V4.1 Flash, the limit is now 80$/mo (the increased it from 60$ recently).
This might seem odd, but the stated purpose of OpenCode Go is "Low cost coding models for everyone", so the limits on the open models tend to be higher.
The math works like this: You use 5$ no Haiku and 20$ on a 60$ model, then your monthly limit is used 66.6%, meaning 5$ of Haiku or 20$ of a 60$ model remaining.
Another way to view this is as follows: You have 10$ flat usage per month, and the "higher usage on each model" is actually just a discount on the token price. So Haiku is discounted to 10$/15$=66.6%, DeepSeek to 12.5% and so on.
Go Plus just means you get more usage, though the rate is not always 4x.
To answer your remaining questions: The billing is based on the actual provider's api costs, so input/cached/output tokens. I suppose the "requests" number is estimated. The OpenCode TUI and your account website actually show the real session prices, which also helps with an understanding how expensive the agents are on raw (unsubsidized) api pricing. The usage limits are rolling, so there is a 5-hour, a weekly and a monthly limit. The 5h is 20% of the monthly limit, and the weekly is 50% (not sure on the exact numbers).
I would agree that the pricing is confusing, but for the price you get surprisingly much usage, especially on the 60$ models. Give DeepSeek V4.1 a try, you get nearly unlimited usage and it's around Sonnet/Opus level.
Here is an example I encountered recently: The Figma MCP Server documents usage with Claude (Code), Codex, Cursor, Grok and Antigravity [], but notably not OpenCode. More so, their oauth registration seems to use some kind of whitelist, meaning just adding the MCP by url does not work either. To make it work, my agent edited the config entry to use "Codex" as the oauth name, which worked.
That's just one example, but something you see quite frequently.
Those problems map extremely well to agent loops. Take a cheap model (like Luna, which can basically run all the time on a Codex subscription, or V4.1 Flash), give it a goal like <1% pixel difference between it's own implementation and what the oracle produces, and it can run for hours or days probing the oracle, verifying it's progress and handling edge case after edge cast until it's effectively perfect.
Key idea is to remove the human from the loop. Don't let the agent build something, tell you what it did and you point out all the issues. It does not scale, leads to fatigue and impatience and limited uptime for the agent. Build clear verifier loops the agent can rely on, let it run and such edge cases are well within reach.
> made with <3 by Ayden
That's the cherry on the cake - indicating genuine effort and attention to detail put into something, yet not even checking the credit at the bottom.
I see even more problems with this. To check any text for those watermarks, it needs to be sent to dozens of AI companies to check, potentially paying them all for just determining whether it matches their watermark, and more concerningly sending all that mostly human-written, often high quality text like unpublished research or books, to AI companies that almost all proved to obtain training data through all kinds of dubious ways.
So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illicit means.
Any university using AI detection in their submission pipeline, or lawyers, editorialists, proofreaders that check for AI marks will be sending significant amounts of text like unpublished research, books, potentially internal documents and more, most of which is high quality human written, to dozens of AI companies, blindly trusting they won't train on any of that.
For anything other than writing code directly in a fully contained git project, where sandboxing might work well, it requires access to system wide tools, user configuration and more.
Occasionally I tell the agent to do everything inside of docker, which works too and it leaves the system alone then mostly, but adds significant overhead and slightly degraded perceived quality / effectiveness.
I think the most important takeaways are to have reliable backup strategies, access control and security mechanisms, which is a win regardless. Whether by the agent or the human, mistakes happen (like a rm -rf * ran in the wrong directory), and where they would be devastating, there should be other protections than just "hope it won't happen" or "rely on a sandbox to prevent agent error".