highfrequency

Can you elaborate on how Codex vs. CC maps onto this cathedral vs. bazaar dichotomy? They seem fairly similar to me.

> These are not railroads—we aren’t building century-long infrastructure. AI datacenters are short-lived, asset-intensive facilities riding declining-cost technology curves, requiring frequent hardware replacement to preserve margins.

Neglects the most important benefit of large semiconductor spending: we are riding the Learning Curve up Moore's Law. We are not much better at building railroads today than we were in 1950. We are way better at building computers today. The GPUs may depreciate but the knowledge of how to build, connect, and use them does not - that knowledge compounds over time. Where else do you see decades of exponential efficiency improvements?

Careful not to automatically slip into a zero sum mindset - it is possible that low interest rates benefit everyone even if the benefits disproportionately go to one class over another. (Example: lowering interest rates after the Financial Crisis - certainly good for banks, but lowering unemployment is critical for normal people too).

In the same way that UBI would disproportionately benefit poor people, but considered with its downstream effects could benefit rich people too.

Huh? There are more Chinese than Indian in the US.

"No free lunch" is not good intuition for many biological processes. There has been near zero evolutionary pressure toward improvements that primarily manifest after age 60.

Separately, evolution took place in a very energy and calorie constrained environment which is very dissimilar to today's world. Many people do not feel spontaneous desire to fast or run laps even though these are clearly beneficial.

I would guess the “secret sauce” here is distillation: pretraining on an extremely high quality synthetic dataset from the prompted output of their state of the art models like o3 rather than generic internet text. A number of research results have shown that highly curated technical problem solving data is unreasonably effective at boosting smaller models’ performance.

This would be much more efficient than relying purely on RL post-training on a small model; with low baseline capabilities the insights would be very sparse and the training very inefficient.

If they release before GPT-5, they don't have to compare to GPT-5 in their benchmarks. It's a big PR win to be able to plausibly claim that your model is the best coding model at the time of release.

What could “parallel thinking techniques” entail if not “using multiple agents in parallel”?

Approach is analogous to Grok 4 Heavy: use multiple "reasoning" agents in parallel and then compare answers before coming back with a single response, taking ~30 minutes. Great results, though it would be more fair for the benchmark comparisons to be against Grok 4 Heavy rather than Grok 4 (the fast, single-agent model).

Yes, but 40 people cannot each have 5% of the company.

Directly contradicts Garry Tan's post saying that all forty founding engineers got seven figure payouts from the Google acquisition: https://x.com/garrytan/status/1947072583092052406

Even if the OP considers the full headline number of $2.4b to be the value of the company, and taking his "1% of fair" number as truth, seven figure payouts would imply all 40 founding engineers had >4% equity which is nonsensical.

Doesn't seem that simple. They raised a total of ~$250m and acquisition price was almost 10x that. The preference cliff means that employees get nothing before investors get an X% return on their investment (100%, 150%, maybe 200%). After that, the payout should be proportional to common stock ownership. Surely the preference guarantee was not 10x?

Would be curious to see the breakdown of the $2.4b:

1. How much to the founders in Google employment incentives

2. How much in licensing fee to the company itself

3. How of the licensing fee went to immediate payout to VC investors (+ employees)

4. How much got left on the balance sheet of the remaining company

I don't understand how #3 can be so large and common stock holders walk away with almost nothing without breaching fiduciary duty?

Pricing is disappointing - no cheaper than the existing AMD server equivalent Epyc Turin 9655 ($11.5k for 96 cores). https://www.amd.com/en/products/processors/server/epyc/9005-...

Significantly worse price/core than the earlier Threadripper 64 core 3990x for $3900.

Thanks for the link. The comparison to electricity is a good one, and this is a nice reflection on why it took time for electricity’s usefulness to show up in productivity stats:

> What eventually allowed gains to be realized was redesigning the entire layout of factories around the logic of production lines. In addition to changes to factory architecture, diffusion also required changes to workplace organization and process control, which could only be developed through experimentation across industries.

More counterintuitively: why hasn't any major bacteria evolved to feast on lipids like oils? Much denser source of energy than desert dirt.

Thanks for sharing - this architecture post is also a great read: https://madebyevan.com/figma/building-a-professional-design-...:

> Our vision for the future of design tools is one where both the tool and the content are easily available to anyone, anywhere

> The reason this is hard is because the web wasn’t designed as a general-purpose computing platform.

> We fully control memory layout and can use compact 32-bit floats or even bytes when appropriate instead of JavaScript’s 64-bit doubles...The generated code is completely in control of allocation, which makes it much easier to hit 60fps by avoiding GC pauses. All C++ objects are just reserved ranges in a pre-allocated typed array so the JavaScript GC is never involved.

> One big issue for us was that certain browser configurations couldn’t allocate large ranges of continuous address space for the huge typed array that contains the entire emscripten memory space. The worst case was 32-bit Chrome on Windows which sometimes couldn’t even allocate a 256mb typed array because ASLR was fragmenting the address space. This has since been fixed.

A great example of making a single bold architectural decision (building a design tool on the web instead of as an app) in service of a vision (prioritize live collaboration), and then thoughtfully resolving all of the performance tradeoffs that stem from this route.

What is the difference between o3 pro and deep research? From a glance, both seem to take 10-15mins to respond and use o3 as the base model.

I disagree completely - consider asking a color blind person to describe the color of flowers. Conversation would only be frustrating. This is analogous to LLMs seeing the world in tokens rather than characters, so character counts are simply not part of their input spectra in the same way that a blind person doesn’t get visual inputs.

Consider also all the smart people who get obsessed with conspiracy theories and spew out endless “mock reasoning” about them. Again, if “nothing incoherent anywhere” is your benchmark for intelligence, humans ain’t it. I mean, what would a computer say about a human that forgot where he just put his keys because he was thinking about dinner - “what, you can’t even store the last 10 seconds of history and search it?” Undergrads’ hit rates on mental double digit multiplication are probably <50%. In many, many ways we look completely idiotic. Surely intelligence is defined by what we can do.

Do you accept any positive definition for AGI, as in if they can achieve X result (write a bestselling novel, solve the Riemann Hypothesis) you would consider it intelligent? I find that negative definitions, as well as theoretical arguments about the techniques rather than the results (eg “LLMs cannot be AGI because they were trained the predict the next word”) to be basically useless for discussion compared to thresholds for positive results. The former will never be achieved (it is trivial to find cases of intelligent people being dumb) and the latter is totally subjective.

"Nothing dumb anywhere" is an unreasonably high bar for AGI. Even Isaac Newton spent 1/3 of his career trying to predict future events from reading the Bible. Not to mention all the insane ego-driven decisions like Hamilton's voluntary duel with Burr.

Sure, Gemini may spit out obviously self-contradictory answers 2% of the time. How does that compare to even the brightest humans? People slip up all the time.

If Bank of America let anyone in the world easily open a checking account, this wouldn’t be a problem right? But they don’t do that, because of regulations and KYC etc.

Similarly, if BoA supported an Ethereum stablecoin but a random bank in Argentina supports a Solana stablecoin, it’s still going to be a pain to transfer money right?

I’m struggling to see how this is a blockchain issue rather than a regulatory + agreement on international standard issue.

May be looking at this the wrong way, but I thought the point of stablecoins (as compared to normal USD) is that the banking system is slow, expensive and cumbersome to transfer USD due for regulatory reasons and perhaps legacy tech reasons.

If so, what is the point of stablecoins if the banks themselves are running it? If regulations and technology are no longer an issue, can’t they just make USD transfers fast and easy? Why bother with a wrapper around USD?

The magic of compound interest: buying an original Magna Carta for $27 and selling it for $21 million 80 years later is equivalent to achieving 18.5% compound interest. Roughly the same rate and duration as Warren Buffett's investing career, with a smaller starting value.

> Judge quality not category

I like this one. Would be curious to hear the author's elaboration.

> How can you finish things in a world where you can keep improving systems indefinitely? It means getting them to a point where the decision-makers at the company are happy.

Rational mindset for an employee, but the problem (for the company) is that upper management is usually unqualified to distinguish good vs. bad technical work. So under this advice, engineers do the minimal work to “deliver” lots of features in order to check off the OKR for upper management, but the engineering is done poorly. Upper management has no insight into the quality of the work, but over time the bugs and performance degradation pile up, while engineering managers get promoted and OKRs completed.

The only real resolution is to push technical expertise as high up into the org as possible, so that even if engineers are only appeasing upper management, upper management will demand good quality work. Apple is the notable example of this org structure: there are no general managers except Tim Cook; everyone who reports to Cook is a technical expert.

Intriguing perspective! Could you elaborate on this with another paragraph or two?

> We humans undervisualize until we see concrete results.

Crazy that there are now five and a half companies that all have roughly state of the art LLMs.

> We developed a new training technique which we refer to as MetaP that allows us to reliably set critical model hyper-parameters such as per-layer learning rates and initialization scales. We found that chosen hyper-parameters transfer well across different values of batch size, model width, depth, and training tokens.

This sounds interesting. Anyone have a link to the paper or other documentation on MetaP?

Are you able to formulate "generalization" in a concrete and objective way that could be achieved unambiguously, and is currently achieved by a typical human? A lot of people would say that LLMs generalize pretty well - they certainly can understand natural language sequences that are not present in their training data.

The results make the method interesting, not the other way around.

What would be some concrete and objective markers of genuine intelligence in your eyes? Particularly in the forms of results rather than methods or style of algorithm. Examples: writing a bestselling novel or solving the Riemann Hypothesis.

What is that baseline threshold for intelligence? Could you provide concrete and objective results, that if demonstrated by a computer system would satisfy your criteria for intelligence?