Unfortunately, the "PT" abbreviation is too short for the timezone database, so while they decide on another form they will temporarily use a bare -7 offset.
3,557 karma · joined February 1, 2019
Unfortunately, the "PT" abbreviation is too short for the timezone database, so while they decide on another form they will temporarily use a bare -7 offset.
Is it? I think the law is truly undeveloped when it comes to language models and their output.
As a purely human example, suppose I once long ago read through the source code of GCC. Does this mean that every compiler I write henceforth must be GPL-licensed, even if the code looks nothing like GCC code?
There's obviously some sliding scale. If I happen to commit lines that exactly replicate GCC then the presumption will be that I copied the work, even if the copying was unconscious. On the other hand, if I've learned from GCC and code with that knowledge, then there's no copyright-attaching copy going on.
We could analogize this to LLMs: instructions to copy a work would certainly be a copy, but an ostensibly independent replication would be a copy only if the work product had significant similarities to the original beyond the minimum necessary for function.
However, this is intuitively uncomfortable. Mechanical translation of a training corpus to model weights doesn't really feel like "learning," and an LLM can't even pinky-promise to not copy. It might still be the most reasonable legal outcome nonetheless.
The use seems to be more common in British than in American English.
I think LLMs are changing the nature of technical debt in weird ways, with trends that are hard to predict.
I've found LLMs surprisingly useful in 'research mode', taking an old and badly-documented codebase and answering questions like "where does this variable come from, and what are its ultimate consumers?" Its answers won't be as natural as a true expert's, but its answers are nonetheless useful. Poor documentation is a classic example of technical debt, and LLMs make it easier to manage.
They're also useful at making quick-and-dirty code more robust. I'm as guilty as anyone else of writing personal-use bash scripts that make all kinds of unjustified assumptions and accrete features haphazardly, but even in "chat mode" LLMs are capable of reasonable rewrites for these small problems.
More systematically, we also see now-routine examples of LLMs being useful at code de-obfuscation and even decompilation. These forward processes maximize technical debt compared to the original systems, yet LLMs can still extract meaning.
Of course, we're not now immune to technical debt. Vibe coding will have its own hard-to-manage technical debt, but I'm not quite sure that we have the countours well defined. Anecdotally, LLMs seem to have their biggest problem in the design space, missing the forest of architecture for the trees of implementation such that they don't make the conceptual cuts between units in the best place. I would not be so confident as to call this problem inherent or structural rather than transitory.
However, it is a genuine question whether the literal meanings of thinking blocks are important over their less-observable latent meanings. The ultimate latent state attributable to the last-generated thinking token is some combination of the actual token (literal meaning) and recurrent thinking thus far. The latter does have some value; a 2024 paper (https://arxiv.org/abs/2404.15758) noted that simply adding dots to the output allowed some models to perform more latent computation resulting in higher-skill answers. However, since this is not a routine practice today I suspect that genuine "thinking" steps have higher value.
Ultimately, your thesis can be tested. Take the output of a reasoning model inclusive of thinking tokens, then re-generate answers with:
1. Different but semantically similar thinking steps (i.e. synonyms, summarization). That will test whether the model is encoding detailed information inside token latent space.
2. Meaningless thinking steps (dots or word salad), testing whether the model is performing detailed but latent computation, effectively ignoring the semantic context of
3. A semantically meaningful distraction (e.g. a thinking trace from a different question)
Look for where performance drops off the most. If between 0 (control) and 1, then the thinking step is really just a trace of some latent magic spell, so it's not meaningful. If between 1 and 2, then thinking traces serve a role approximately like a human's verbalized train of thought. If between 2 and 3 then the role is mixed, leading back to the 'magic spell' theory but without the 'verbal' component being important.
"Thinking meat! You're asking me to believe in thinking meat!"
While next-token prediction based on matrix math is certainly a literal, mechanistic truth, it is not a useful framing in the same sense that "synapses fire causing people to do things" is not a useful framing for human behaviour.
The "theory of mind" for LLMs sounds a bit silly, but taken in moderation it's also a genuine scientific framework in the sense of the scientific method. It allows one to form hypothesis, run experiments that can potentially disprove the hypothesis, and ultimately make skillful counterfactual predictions.
> By asking it why it did something wrong, it'll treat that as the ground truth and all future generation will have that snippet in it, nudging the output in such a way that the wrong thing itself will influence it to keep doing the wrong thing more and more.
In my limited experience, this is not the right use of introspection. Instead, the idea is to interrogate the model's chain of reasoning to understand the origins of a mistake (the 'theory of mind'), then adjust agents.md / documentation so that the mistake is avoided for future sessions, which start from an otherwise blank slate.
I do agree, however, that the 'theory of mind' is very close to the more blatantly incorrect kind of misapprehension about LLMs, that since they sound humanlike they have long-term memory like humans. This is why LLM apologies are a useless sycophancy trap.
Yes, and that's okay because the classroom is a learning environment. However, LLMs don't learn; a model that releases the magic smoke in this session will be happy to release it all over again next time.
> LLMs are just surfacing the fact that assessing and managing risk is an acquired, difficult-to-learn skill.
Which makes the problem worse, not better. If risk management is a difficult skill, then that means we can't extrapolate from 'easy' demonstrations of said skill to argue that an LLM is generally safe for more sensitive tasks.
Overall, it seems like LLMs have a long tail of failures. Even while their mean or median performance is good, they seem exponentially more likely than a similarly-competent human to advise something like `rm -rf /`. This is a deeply unintuitive behaviour, precisely because our 'human-like' intuition is engaged with resepct to the average/median skill.
The trick is to be the Oracle of Delphi, not Cassandra.
Make the prediction once, with politeness and humility, and preferably in enough company that your opinion is noted even if (when) it is overridden. Use it as an opportunity to be seen as wise, not just smart.
Then, keep contingency plans. When the problem manifests, have a solution ready as best you can given your limited position. Even when it's too late to avoid the whole problem, you might be able to limit the blast radius. Again, be public but polite about it, and most importantly never say "I told you so" or otherwise appear smug.
You want to cultivate the reputation of "the person who is right but easy to work with, and who always has your back in a pinch."
To be nitpicky, the article doesn't say 'formal warning,' just 'warning.' That could have been anything from a gentle let-down to a reprimand.
That being said, I think your broader point is reasonably true: the author frames the 'political games' of promotion as a regrettable necessity rather than a job requirement beyond the juniormost levels. Despite their self-description as helpful and cooperative, they disdain the dyadic sport of cooperatively making their boss look good.
That's not to say that one should submit to base exploitation, of course, but there's a fine art to understanding the constraints and incentives of others and working with (and often within) that framework.
A second skill is being able to separate the person from the position, to maintain friendly or at least respectful personal relationships with people who might be professional adversaries at the moment. This is harder, but if professional hostility reads as personal contempt that will definitely destroy one's social weight in an organization.
Not necessarily. This could be considered a quotation of a trivial part of a larger work, making the use legal in the US under its fair use doctrine.
Additionally, I'm not aware of any obvious way that this use could harm the commercial market for Warcraft 3 (and the other games whose voice packs are included in this repository). The use here does not compete with the original, and if anything it might drive sales on the margin through nostalgic reminders.
This is an expected outcome of how LLMs handle large problems. One of the "scaling" results is that the probability of success depends inversely on the problem size / length / duration (leading to headlines like "AI can now automate tasks that take humans [1 hour/etc]").
If the problem is broken down, however, then it's no longer a single problem but a series of sub-problems. If:
* The acceptance criteria are robust, so that success or failure can be reliably and automatically determined by the model itself, * The specification is correct, in that the full system will work as-designed if the sub-parts are individually correct, and * The parts are reasonably independent, so that complete components can be treated as a 'black box', without implementation detail polluting the model's context,
... then one can observe a much higher overall success rate by taking repeated high-probability shots (on small problems) rather than long-odds one-shots.
To be fair, this same basic intuition is also true for humans, but the boundaries are a lot fuzzier because we have genuine long-term memory and a lifetime of experience with conceptual chunking. Nobody is keeping a million-line codebase in their working memory.
The scale there is a little bit different. If you're training an LLM with 10,000 tightly-coupled GPUs where one failure could kill the entire job, then your mean time to failure drops by that factor of 10,000. What is a trivial risk in a single-GPU home setup would become a daily occurrence at that scale.
# Read-only mode (no file modifications possible)
pi --tools read,grep,find,ls -p "Review the code in src/"
Otherwise, "yolo mode" inside a sandbox is perfectly reasonable. A basic bubblewrap configuration can expose read-only system tools and have a read/write project directory while hiding sensitive information like API keys and other home-directory files.If you're worried about a hostile agent, then indeed sandboxing is not enough. In the worst case, an actively malicious agent could even try to escape the sandbox with whatever limited subset of commands it's given.
If you're worried about prompt injection, then restricting access to unfiltered content is enough. That would definitely involve not processing third-party input and removing internet search tools, but the restriction probably doesn't have to be mechanically complete if the agent has also been instructed to use local resources only. Even package installation (uv, npm, etc) would be fine up to the existing risk of supply-chain attacks.
If you're worried about stochastic incompetence (e.g. the agent nukes the production database to fix a misspelled table name), then a sandbox to limit the 'blast radius' of any damage is plenty.
I don't think there is such a good heuristic. The user wants the agent to do the right thing and not to do the wrong thing, but the capabilities needed are identical.
> `chroot` is literally the first thing I used when I first installed a local agent, by intuition (later moved on to a container-wrapper), and now I'm reading about people who are giving these agents direct access to reply to their emails and more.
That's a good, safe, and sane default for project-focused agent use, but it seems like those playing it risky are using agents for general-purpose assistance and automation. The access required to do so chafes against strict sandboxing.
That's not the right threat model. The existing peer review process is already weak to high-effort but conflicted research.
Instead, the threat model is closer one closer to that of spam, where the submitting authors don't care about the content of their submission at all but need X publications in high-impact outlets for their CV or grant application. Predatory journals exploit this as part of a pay-to-play problem, but the low reputation of those journals limits their desirable impact factor.
This threat model relies on frequent but low-quality submissions, and a submission fee would make taking multiple kicks at the can unviable.
[ ] — Yes, I'd welcome a half-hour distraction from the thing I need to use the computer for right now!
[ ] — No, I want to get my thing done now but then be confused about missing or new buttons at random intervals for the next six to eighteen months!
> For the purposes of this post, I’ll use the figures from the 100,000 “maximum”–Claude Sonnet and Opus 4.5 both have context windows of 200,000 tokens, and I run up against them regularly–to generate pessimistic estimates. So, ~390 Wh/MTok input, ~1950 Wh/MTok output.
Expensive commercial energy would be 30¢ per kWh in the US, so the energy cost implied by these figures would be about 12¢/MTok input and 60¢/MTok output. Anthropic's API cost for Opus 4.5 is $5/MTok input and $25/MTok output, nearly two orders of magnitude higher than these figures.
The direct energy cost of inference is still covered even if you assume that Claude Max/etc plans are offering a tenfold subsidy over the API cost.
Why is it an externality? Anthropic (or other model provider) pays the electricity cost, then it's passed along in the subscription or API bill. The direct cost of the energy is fully internalized in the price.
As I read it, the implication is that a market maker in the high-P regime needs to still have an expected edge of 1.75% to profit net of fees, which means that the 'maker return' table in this article is net negative after fees for all categories save for entertainment, media, and world events.
1. The article mentions the bid/ask spread for contracts, but I believe that Kalshi also has its own fee structure. Small edges (an expected loss of 0.57¢ on a 1¢ contract implies an expected gain of 0.43¢ on a 99¢ contract, or a 5.75ppt edge) can be easily eaten by even small fees, and liquidity provision is all about small edges.
2. The article ignores the time value of money, and contracts take time to resolve. If a contract won't resolve for six months and the risk-free rate is 5%, then buying a "sure thing" over 97.5¢ is a loss net of otherwise earnable interest.
3. Long shots offer greater implied leverage to bettors, making them more attractive. This is still (sometimes) an exploitable mispricing, but it's closer to the well-understood "bet against beta" factor.
(Edit to add) Also, I think their explanation of the non-returns on finance is lacking:
> Why is Finance efficient? The likely explanation is participant selection; financial questions attract traders who think in probabilities and expected values rather than fans betting on their favorite team or partisans betting on a preferred candidate. The questions themselves are dry ("Will the S&P close above 6000?"), which filters out emotional bettors.
Financial contracts are the ones that are most perfectly hedges with existing markets. "Will the S&P close above X?" is a binary option, after all, so it's comparatively easy for a market-maker to almost perfectly offset their Kalshi positions with opposite positions in traditional markets.
I agree with your point about induced demand. The “win” wouldn’t be looking at a single press release with already-suspect numbers, but rather looking at essentially all press releases of note, a task not generally valuable enough to devote people towards.
That being said, we normally consider it progress when we can use mechanical or electrical energy to replace or augment human work.
That’s a reasonable attempt to enforce the ToS. For OpenCode, they also take the next step of additionally blocking a second line of “You are OpenCode.”
There might be more thorough ways to effect a block (e.g. requiring signed system prompts), but Anthropic is clearly making its preferences known here.