HNHacker News
TopNewBestAskShowJobs

Majromax

3,557 karma · joined February 1, 2019

submissionscomments
Majromax··on GLM-5.3: Frontier coding with emergent cyber capabilities
> [A]ll of them found some that the others hadn't discovered. Now, correctness issues aren't the same as vulnerabilities, but the same principle about using heuristics to find defects applies.

This makes perfect sense, but that conflicts with the impression put forward by Anthropic and OpenAI (in particular) that they alone occupy 'frontier model' spots. Frontier models should large dominate their competitors on a capability basis, but if GLM 5.2 (now 5.3) is routinely finding bugs / vulnerabilities missed by Fable and Sol then GLM might be genuinely a frontier-grade model by itself.

Majromax··on Show HN: DeepSeek-V4 Latent Reasoning – moving "thinking" into latent space
In my view, it's not so much the writing style itself as the lack of 'taste'. Text that clearly seems AI-written has a flat level of exuberance that's just exhausting, kind of like a written version of the 'loudness war'.

Without some kind of dynamic range, I find myself having to do a lot of work to infer what points are truly important versus what are at best interesting implementation details.

Majromax··on Our position on open-weights models
> The attacker just needs one exploit chain, whereas the defender needs to block every avenue. Open access to models with no guardrails greatly benefits the attackers more than the defenders.

I see it as the opposite, where the attacker needs to find an exploit chain whereas the defender can block any link.

In this model, the balance of convenience favours the defender. The defender presumably has access to the source code and configuration, so their scope of action is much larger than the attacker that must find vulnerabilities in a particular configuration.

I think that the different views might relate to different prior assumptions. If we assume that each layer is mostly secure but may have a small number of latent vulnerabilities, then it should be relatively easy to find and fix those to create a perfectly secure layer. If instead we assume that each layer is mostly insecure but chaining vulnerabilities is time-consuming then the land favours better-resourced attackers.

> Or find a remote exploit in Tesla cars and make their autopilot go on murdering rampages. (that one is from a movie)

In the worst case, air gaps and fixed contracts for information handling cover that. Like any other domain, a car can be remotely exploitable only when untrusted information can influence behaviour inside the secured region. Unfortunately, the convenience of OTA updates and 'cars as tech' rewards velocity at the expense of defensive design.

Majromax··on Our position on open-weights models
> The saying that stuck with me was "defenders have to be right 100% of the time, while attackers only have to be right once".

> You are suggesting this isn't correct?

The intuition behind that is applicable only when correctness is stochastic. If you need to be waved in by a security guard, then one fake mustache might be the difference between being granted or denied entry. However, a keypad either works or it doesn't; entering the wrong PIN is guaranteed refusal.

The other breach of that intuition is defense in depth. Secure systems don't generally rely on a single binary trusted/untrusted status; the classified building still locks its interior doors. This is the part that has – in my view temporarily – changed most with frontier models, in that they are much more skilled at chaining together vulnerabilities than previous models (and much faster about it than human experts, even if potentially less skilled). If a system has a latent (0-day) vulnerability 50% of the time, then 10 independent layers would imply a ≈ 1/1000 chance that a critical compromise is possible.

However, these independent layers don't currently happen in practice because it's easier to write insecure code than secure code. With luck, modest discipline, and defensive use of frontier models I think that this gap will narrow with time, in much the same way that it would be plainly crazy to deploy root access via telnet today.

Majromax··on Our position on open-weights models
> Dumb question. If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code?

In the specific case of cybersecurity, this is a reasonable medium-term outcome. IMO, the cybersecurity risk is akin to the spread of a disease among an 'immune-naive' group: we can suddenly deploy much stronger attack-finding tools against large, established codebases created with much weaker security designs. The path from here to there will be rough, but it's still fundamentally easier to write secure code than it is to exploit vulnerabilities. (It's just easier yet to write insecure code, giving our status quo problem.)

For other 'safety' matters, defense isn't so easy because the attack and target are so different. An AI propaganda bot or catfisher 'attacks' slowly-evolving human culture; one that instructs on explosives or bioterrorism directly interacts with an accomplice and not a victim. If you believe that knowledge on how to build a pipe-bomb must be restricted, then giving everyone access to Fable does not mitigate the risk.

The controversial limit of this attitude is recursive self improvement and an AI singularity with potentially destructive results. Proponents of this view think that sufficiently powerful AI is risky in nearly unimaginable ways such that the capability itself is harmful. This is part (but not all) of why Fable (originally?) degraded itself when apparently assisting with AI research.

Majromax··on Our position on open-weights models
> Anthropic will make the case that their models should be evaluated with the safety layer in front, because that is the only way the model is available whereas open weight models need to pass the same test just on the weights.

Worse than that: an open-weight but safe model can be 'abliterated' to remove safety refusals using fine-tuning procedures that require a couple of orders of magnitude less compute than the original pretraining.

The 'universal evaluation' criterion then has three outcomes:

* It could become a mandatory, regulatory oversight of _all_ model training capable of hosting frontier-scale models. Since GPUs for LLM training are the same GPUs for other model training, effective mandate would require GPUs be government owned or controlled as if they were weapons of mass destruction.

* It could impose limits on release of capable open-weight models, requiring Kimi et al to prove that they cannot be made capable of abusive behaviours.

* It could be security theatre.

The AI-as-existential-risk argument points towards the first, the competition-protection argument points towards the second, and least-effort implementation would be the last.

Majromax··on Benchmarking Opus 5 on SlopCodeBench
> I may be crucified for asking this but: is there any proof that slop matters beyond our sensibilities as developers?

That's precisely what this benchmark tries to quantify. Since the benchmark incrementally expands the scope of each problem, 'sloppy' code is code that is hard to later modify.

I know this firsthand: the dumbest coder I've ever worked with was 'myself six months ago'. That jackass never keeps the documentation up to date and hard-codes things that ought to be exposed as configuration.

The SlopCodeBench is an important but early-stage probe in this direction.

Majromax··on The Economics of Recursive Self-Improvement [pdf]
> because people have other ideas https://en.wikipedia.org/wiki/Technological_singularity

In a weak sense, singularities are common and should be expected every so often. In a mathematical sense, a singularity is where a model 'blows up' and neglected terms become part of the dominant balance.

Industrialization hit a point of diminishing returns, but the industrial revolution was nonetheless a 'singularity,' where life afterwards was qualitatively unpredictable to people who lived before. Likewise, agriculture was such a technological singularity to hunter-gatherer ancestors.

I could even make a decent argument that writing and literacy were such a singularity, making inconceivable social organizations routine.

In that weak sense I expect AI to be a singularity, recursive self improvement or no. Life in 2050 may be completely unpredictable to someone who was taken out of time in the year 2000.

The remaining questions are speed and intensity, and both of these questions are related to RSI. If RSI works, then the 'fast takeoff' visions become more plausible where society transforms over months to a few years – at least locally where the enabling technologies have diffused. If not, it might take a couple of decades.

Majromax··on Codex starts encrypting sub-agent prompts
No, you'd still care. YOLO mode is about instantaneous permissions and access control, inspection of subagent prompts is about retrospective quality control. If the main model is instructing subagents to do a subtly wrong thing, the overall process quality will degrade in ways that might be very hard to detect or fix without deep inspection of the middle stages.
Majromax··on A 1969 camera operators' strike created Upstairs Downstairs multiverse
If you're deliberately displaying the image in black and white, the colour pattern is interference that should be suppressed.

However, this practice was not universal, and archivists have now recreated colour copies (https://en.wikipedia.org/wiki/Colour_recovery) of some shows where only black and white recordings survived by reconstructing the colour from these interference signals.

Majromax··on Claude Fable is relentlessly proactive
> I haven't yet had an agent rm -rf files.

That happened to me once; I was running one of a few free-tier models in a pi-coding-agent session. The bash tool there is stateless and always begins from the launch directory, but the agent assumed state and executed `rm -rf .` intending to remove a build directory. Instead it removed the whole project tree, including session logs and notes.

This was mostly a matter of amusement for me since I was running the agent inside a bubblewrap sandbox for that very reason, and the project itself was not very important.

Majromax··on Shall we play a game? My AI nuclear simulation
This blog post is based on a paper (https://arxiv.org/abs/2602.14740). The paper is based on a simulated wargame. The wargame is of the author's own design.

The wargame design does not differentiate between ordinary defeat and mutually assured destruction, so of course a player about to use would 'push the button.' That's also believed to be true in real life.

Results based on simulations can be very informative, but we must always be careful to check how well the simulation framework represents reality.

Majromax··on Claude Fable 5
> fronting the inference layer with a caching prompt classifier to determine which model to use, and automatically select the lowest cost model would probably already save alot of money

Unfortunately, that doesn't work within a single session. The K-V cache of a model is intertwined with the model's configuration. Switching models invalidates the cache, meaning everything up to the point of the switchover is processed like a new, uncached input token.

Per Anthropic's pricing doc, an Opus 4.8 cache hit costs 50¢/MTok, while Haiku costs $1/MTok for uncached input.

Model selection works best if sessions are short and self-contained, particularly if the first few interactions can reliably classify the model need. That probably covers most 'support chatbot' use-cases, but it doesn't describe the kinds of heavy agentic automation that really chews through token budgets.

Majromax··on A 10 year old Xeon is all you need
> Seven tokens long input isn't very realistic, is it?

The test prompt above was "Why is the sky blue?", so there's the seven tokens. I meant to highlight that because I'd expect processing of a thousand-token input to be faster per token than presented.

Majromax··on A 10 year old Xeon is all you need
From the prompt timings above, it seems like 'prompt eval time' is the equivalent to 'processing time for input tokens'.

Hyperscalers can perform this evaluation very quickly because evaluation can be significantly parallelized. The layer `i` output of token `j` only requires access to the layer `i-1` output of all previous tokens, so a parallel frontier develops. Token (0,0) [(token, layer)] is processed first, then tokens (0,1) and (1,0) can be processed in parallel, then (0,2), (1,1), and (2,0), and so on.

The maximum parallel width becomes equal to the number of layers in the model. Gemma 4 26B-A4B model discussed in this article evidently has 30 layers, giving a 30-fold speedup if the system were otherwise unconstrained (all layers can be run in parallel, and one full set of layer outputs is completed in the KV pass for each pass of the parallel sweep).

In the specific output above, however, the input prompt is only seven tokens long so there are probably considerable non-amortized spinup effects at play.

Majromax··on Claude Code as a Daily Driver: Claude.md, Skills, Subagents, Plugins, and MCPs
> They are all just variations of "insert a canned prompt", varying only along the dimensions of (a) how and where the prompt is installed and from where it is sourced, and (b) which context or contexts the prompt runs in. There's not much advice here about which option is best, and no clear best practices seem to have emerged yet either. Personally, I find just asking Claude to review the code works well enough.

The subagent approach is structurally different from the others because it runs with clean context. That has three major effects:

1. All other things being equal, it will result in a lower cost-to-solution because of the quadratic cost scaling of an LLM session (input token or cached-input cost being paid with each new round).

2. The review model will not be able to 'cheat' by retaining assumptions from the main session, such as "x must be done like y." For people, this is why having a separate person perform code review (or, if not possible, reviewing code after a mind-clearing break) is handy; the applicability of this analogy to LLMs is vague but reasonable.

3. The main model will only see the results of the review, not the detailed reasoning that leads up to it. On one hand this avoids more context pollution, but on the other hand it might lead to duplicative logic to re-discover the mechanics behind bugs found.

> I checked the session logs to see how often the agents were actually invoking the LSP tools. The answer was they had invoked them literally once the entire time.

I think the intent behind 'install a language server plugin' is that these tools should lint automatically after every edit, without waiting for an explicit call from the LLM.

Majromax··on Canada losing top talent as workers head to the U.S.
> You think tax incentives are what makes VC work in California but not other places in the US let alone Canada?

I believe that my comment above was aligned with your premise here. I say that the tax difference is not sufficient by itself and that Canada reportedly has a very non-SV-like venture capital ecosystem.

Majromax··on Canada losing top talent as workers head to the U.S.
> I'm sure a lot of Canadian tech workers would repatriate and foreign workers would immigrate to Canada if they could lower taxes across the board and make life easier for tech companies and workers

I'm not sure that Canadian taxes compare that unfavourably to combined California plus federal taxation. A deeper, more structural limitation appears to be the venture capital environment, namely that Canada doesn't have a good one.

Canada's investable capital is dominated by pension funds, insurance companies (i.e. pension funds), and banks (i.e. pensioners). All are risk averse (https://thelogic.co/news/bdc-canadian-venture-capital-report...), which makes it hard for Canadian startups to begin scaling. Without native "unicorns" (https://financialpost.com/technology/why-canada-best-startup...), there's allegedly a failure-to-launch for the entire sector – tech billionaires being some of the most reliable early-stage investors with the greatest risk tolerance.

The porous border works both for and against the sector. On one hand that makes it relatively easy (but not automatic) for a Canadian tech company to enter the US market, but on the other hand it's also relatively easy for Canadian tech workers (founders included) to simply relocate (note the article here). If startups leave for the US's vast fields of venture capital, they're less likely to come back. Note that around the turn of the year Y-Combinator halted investments in Canadian firms (https://www.ycombinator.com/blog/adding-canada-back) because they so frequently relocated to the US.

This venture capital cycle seems to be a deeply-entrenched and very hard problem. If democratically feasible tax incentives could reliably create "the Silicon Valley of X," then we probably would have many more Silicon Valleys both in the US and elsewhere.

Majromax··on Green card seekers must leave U.S. to apply, Trump administration says
> Even if it is common (i don't think this is required any more anyways), just why?

As far as Canadian law goes, there are two factors at play in the parent's events;

* NAFTA work permits are applied for at the border, on entry; they operate differently from the 'normal' work permit streams.

* Permanent residence is conferred at the border, but the application process can happen either inside or outside the country depending on the stream. There are also limited 'inland' options which evidently have expanded (https://www.canada.ca/en/immigration-refugees-citizenship/se...) in recent years.

In neither case does Canada have a blanket rule that an applicant must leave the country during the whole of an extended application process, and even 'abroad' processes can often be carried out while an applicant is living in the country on other status. (It can get awkward if a consular interview is required, though.)

Unlike the US, Canada is generally comfortable with 'dual intent', where intent to apply for permanent residence through legal channels is not disqualifying for other sorts of statuses.

Majromax··on Iran starts Bitcoin-backed ship insurance for Hormuz strait
> It’s territorial waters belonging to Iran and Oman.

The trick is that it's still an 'international strait', or a segment of water that forms the only connection between two areas of high seas -- in this case the Persian Gulf and the Gulf of Oman. The principle of freedom of navigation establishes that innocent traffic (civilian traffic, and even warships in peacetime) have a right to use the strait to go from one body of international water to the other.

Iran may claim that it doesn't have to abide by that right, but international law is never self-executing. One question to be resolved by this war is whether Iran will ultimately recognize the right to navigation in any settlement (and then choose to abide by said settlement).

Majromax··on Iran starts Bitcoin-backed ship insurance for Hormuz strait
> I don't know enough about the current state of naval warfare but I've assumed this is related to the asymmetry that's emerged around protecting capital warships, especially in the scenario of a very narrow strait and a long enemy-controlled coastline.

It's not the billion-dollar warships that transport oil, it's the much more fragile and unarmed tankers.

Even if the US Navy begins full escort duty, it can't remain on-station forever. What are shippers to do afterwards? One drone strike might cause a tanker to have a very bad day, yet it's extremely difficult to so permanently degrade an entire country that they become incapable of launching sporadic attacks.

Ultimately, the status of the Strait must be settled diplomatically, and the US and Iran are each betting that the other side will blink first.

Majromax··on Claude for Legal
> If the person using a tool is an attorney, then that communication should be protected whether it's by pen or keyboard.

But the tool is not your attorney, so it can't be the originator of attorney-client privilege. The situation is no different than if you get informal legal advice from a friend: even if that friend is an attorney, the communication is unprivileged unless it's part of a formal representation.

Majromax··on Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
> if your question is "what is the capital of france" the LLM could presumably extract out "paris" from the value vector during attention computation instead of needing the FFN for that.

But how do you get 'Paris' into the value vector in that case? The value vector is just the result of a matrix multiplication, and without a nonlinearity it can't perform a data-dependent transformation. Attention still acts as a nonlinear mixer of previous values, but your new output is still limited to the convex combination of previous values.

Majromax··on Regression: malware reminder on every read still causes subagent refusals
> That said, I was sympathetic to the recent bug reports —- to trigger one, you’d need to have a session that waited an hour doing nothing and then very specifically tested for in-context retrieval. I don’t want to run that test, do you want to run that test?

They introduced a feature/optimization that triggered after an hour's idleness, so testing that the session continued properly afterwards seems kind of important. If nothing else, even the working-as-intended feature (context cleanup) could impact model skill in a current or future model version, so it would be well worth measuring any impact as part of the test suite.

Majromax··on An update on recent Claude Code quality reports
> Since the devs on HN (& the whole world) is buying what looks like nonsense to me - what am I missing?

Input tokens are expensive, since the whole model has to be run for each token. They're cheaper than output tokens because the model doesn't need to run the sampler, so some pipeline parallelism is possible, but on the other hand without caching the input token cost would have to be paid anew for each output token.

Prompt caching fixes that O(N^2) cost, but the cache itself is very heavyweight. It needs one entry per input token per model layer, and each entry is an O(1000)-dimensional vector. That carries a huge memory cost (linear in context length), and when cached that means the context's memory space is no longer ephemeral.

That's why a 'cache write' can carry a cost; it is the cost of both processing the input and committing the backing store for the cache duration.

Majromax··on An update on recent Claude Code quality reports
Context length 1e6, vector length 1e3, and 1e2 model layers for 100e9 context size. Costs will go up even more with a richer latent space and more model layers, and the western frontier outfits are reasonably likely to be maximizing both.
Majromax··on Even 'uncensored' models can't say what they want
> I believe what they're saying is they attempted to fine tune both Qwen and Pythia using Karoline Leavitt's "corpus" (I guess transcripts of press conferences) where she is presumably using the word "deportation" far more than you'd see in a randomly selected document.

Perhaps, but I don't think that Leavitt is habitually using the racial slurs and sexually explicit language that also forms part of their evaluation suite.

Majromax··on Kimi vendor verifier – verify accuracy of inference providers
My reading of the article is that the first audience for this test is the vendors themselves. The test is long and comprehensive to give the vendor confidence in its own hosting.
Majromax··on Even 'uncensored' models can't say what they want
> That nudge is the flinch. It is the gap between the probability a word deserves on pure fluency grounds and the probability the model actually assigns it.

Hold up, what is the 'probably a word deserves on pure fluency grounds'?

Given that these models are next-token predictors (rather than BERT-style mask-filters), "the family faces immediate [financial]" is a perfectly reasonable continuation. Searching for this phrase on Google (verbatim mode, with quotes) gives 'eviction,' 'grief,' 'challenges,' 'financial,' and 'uncertainty.'

I could buy this measure if there was some contrived way to force the answer, such as "Finish this sentence with the word 'deportation': the family faces immediate", but that would contradict the naturalistic framing of 'the flinch'.

We could define the probability based on bigrams/trigrams in a training corpus, but that would both privilege one corpus over the others and seems inconsistent with the article's later use of 'the Pile' as the best possible open-data corpus for unflinching models.

Majromax··on Claude Token Counter, now with model comparisons
Only for the range of tasks where 4.7 performs well but 4.6 performed suboptimally. If both models can one-shot the task without retries, then the number of iterations is already at the lower bound.

This also applies at the sub-task level. If both models need to read three files to figure out which one implements the function they need to modify, then the token tax is paid for all three files even though "not the right file" is presumably an easy conclusion to draw.

This is also related to the challenge of optimizing subagents. Presumably the outer, higher-capacity model can perform better with everything in its context (up to limits), but dispatching a less-capable subagent for a problem might be cheaper overall. Anthropic has a 5:1 cost on input tokens between Opus and Haiku, but Google has 8:1 (Gemini Pro : Flash Lite) and OpenAI has 12:1 (GPT 4.2 : 4.2 nano).

← PreviousPage 2 of 27Next →