HNHacker News
TopNewBestAskShowJobs

red75prime

1,454 karma · joined September 24, 2015

submissionscomments
red75prime··on Why do we need human mathematicians anymore?
Brute-force search is called that specifically for its lack of sophistication. A "sophisticated brute search" is an oxymoron.
red75prime··on The LLMentalist Effect (2023)
> a basic understanding of how LLMs operate at a technical level

An LLM with CoT is Turing-complete. Training is, basically, compression (the training data gets lossily compressed into the model's weights). The information-theoretic limit of compression is an algorithm that reproduces functionality of a system that produced the training data.

No "magic" is required to get to a system that reproduces at least some facets of the human brain functionality.

Three years ago I was skeptical that stochastic gradient descent (and other known techniques) are the way. But evidence kept piling up.

red75prime··on English: A vs. An
This doesn't touch place names. The Hague, Amsterdam, The Maldives, Australia, The Netherlands, France...
red75prime··on BYD Slashes Price of Electric Car and Becomes Cheapest in Australia [video]
Freedom from those pesky car manufacturers that demand protection of domestic workforce?
red75prime··on GPT-6 Astra Solves a WWI German Radio Cipher
Because it wasn't an expert or a heuristic developed by an expert that made it possible to decrease the dictionary size.
red75prime··on GPT-6 Astra Solves a WWI German Radio Cipher
If the final set of possibilities is astronomically smaller than a naive one, calling the whole process a "brute force exercise" draws attention to an astronomically insignificant part.
red75prime··on GPT-6 Astra Solves a WWI German Radio Cipher
I'm aware. Locating a single probable key is exactly not that.
red75prime··on GPT-6 Astra Solves a WWI German Radio Cipher
Can we stop using "brute force" for designating "tour de force"?
red75prime··on How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip
> I remember the paper proving that hallucinations could never be fully solved back in 2024

The papers that use the halting problem or the Gödel's incompleteness theorem to prove something about LLMs are dime a dozen. The problem is they prove their results for any computable system. You need to also believe that the human brain contains "magic" to think that humans are exempt.

I believe I've said the same at the time this paper was published. There is no need for hindsight to notice the problem.

The required amount of compute and training data and whether the existing training methods were up to the task had the real potential to be show stoppers though.

red75prime··on GLM Built Its Own Inference Infrastructure
Buckmaster and Alpöge has found a forced finite-time singularity for the 3D incompressible Euler equations (and two other types) building on the work by Diego Córdoba and Luis Martínez-Zoroa with the assistance of Anthropic and OpenAI models. Then OpenAI found a forced finite-time singularity for the Navier-Stokes equations.

TL;DR Buckmaster and Alpöge haven't solved Navier-Stokes blow up.

How information can get so distorted when it's trivial to fact check?

red75prime··on How good are frontier models at physics?
I don't see that much difference with "A plane crashes on the border of the United States and Canada. Where do they bury the survivors?"

Fast thinking fails to activate slow thinking.

red75prime··on How good are frontier models at physics?
> they don't fundamentally "get it"

There's no clear decision criteria for this. Do trick questions demonstrate that most people don't "get it"? And, well, older model saying dumb things doesn't establish a general principle that LLMs don't "get it" in general.

> The sample efficiency is just crazy low

Autoregressive pretraining requires huge amount of data to go from a blank state to a somewhat functional model. Fine-tuning, LORA, reinforcement learning of foundation models and in-context learning are much more sample efficient.

> Chinese room

...creates a wrong intuition that by cranking a Leibniz's mill you are somehow responsible for whether it understands something or not.

red75prime··on How good are frontier models at physics?
This is unconventional benchmaxxing then, when they decrease the benchmark scores to allow models to generalize on correct solutions.
red75prime··on How good are frontier models at physics?
Yeah, it's strange that people who should be knowledgeable in experimentation ignore that. "I heard that people enjoy skiing. I've tried skiing on whatever surface happened to be on a nearby hill and it didn't work!"
red75prime··on Backprop Alternative: Augmented Lagrangian Predictive Coding
Maybe it has some interesting properties that could prevent catastrophic forgetting, but so far it looks like a biologically plausible approximation of backpropagation and it might be useful to decrease the training compute requirements.
red75prime··on Why don't machine learning research agents overfit?
Does '110001111000000011111111111' contain n>1∧∀d(d|n→(d=1∨d=n)) as well as infinite number of other generalizations?
red75prime··on Fable 5.1 Solves the Cyphral Distich, a 370-year-old cipher
It doesn't explain why it doesn't make "I'm not paid enough for this shit" more statistically likely.

LLMs' processing that reproduces statistical patterns of the training data is modified by post-training. That's why we have LLMisms, for example.

LLMs aren't simple patter-matchers/pattern-predictors. They are incredibly complex systems that capture some aspects of the systems that produce the training data.

red75prime··on Fable 5.1 Solves the Cyphral Distich, a 370-year-old cipher
What does it have to do with superstitions? Those phrases modify whichever stopping conditions a model has.
red75prime··on Stabilizing Rust's Never Type
> remember to kill all pedophile murderers like...

What's this bullshit?

red75prime··on Everyone should slow down AI development except for me
Oh! A testable prediction. Here's mine: there will surely be a lot of politicking around who qualifies to be the independent evaluator, but they'll agree on something because they are really scared.
red75prime··on Google will buy half the electricity from one of Finland's nuclear power plants
BTW, LCOE isn't that good estimate. It doesn't account for the system costs. LFSCOE (Levelized Full System Costs of Electricity) does better in estimating the amount of money you actually need to spend.
red75prime··on Astra for Coding: Why Are We Doing This Again?
> When working on something from scratch, the best an agent can do is the average of whatever is in its training set and the clarity of the text prompts.

Nah. The best it can do is to use the best writing style a model learned. Post-training might fail to prioritize it, though. Autoregressive pretraining does not average things. It creates a predictive model for variety of programming styles.

red75prime··on Google will buy half the electricity from one of Finland's nuclear power plants
1 tonne of steel requires around 5MWh of energy at 50,000 tonnes per day it's around 250GWh of batteries at 50% usage. The largest planned BESS is 19GWh. As I said solar+BESS is impractical.

Wind is more expensive. Onshore wind turbines have higher variability. Offshore ones are limited to coastal areas. Overall, there's a lot of practical problems. It's all doable, but building infrastructure takes time.

China's onshore wind potential is 12.6 PWh/year[1]. China's 2024 energy consumption is 9.85 PWh[2]. It's not much of overcapacity potential. Huh. I though it's higher.

[1] https://www.sciencedirect.com/science/article/abs/pii/S13640... [2] https://english.www.gov.cn/archive/statistics/202501/20/cont...

red75prime··on Google will buy half the electricity from one of Finland's nuclear power plants
> You install 12 hours worth of batteries at the factory

I literally laughed out loud, sorry. Providing baseload with BESS alone? That's rich. Literally. Extremely expensive. In addition to impractical amount of batteries it requires impractical overcapacity to compensate for seasonal variation.

> Then why is no other country burning that amount of coal?

China does about 30% of the world's industrial production and 40% of metal production, while having around 50% of world's coal power plants. They are working on it though. Many of their new coal plants are peaker plants.

red75prime··on Google will buy half the electricity from one of Finland's nuclear power plants
Predictability will not get a plant through the night by itself. BTW, it's about 7 units of wind energy to 1 unit of solar energy plus significant overcapacity plus increase of the grid transmission capacity plus additional measures for grid stabilization (like synchronous condensers) to get to a "fairly modest amount of storage."

> Trouble being they're still operating ~1200 coal power plants.

Yeah, it's because of the above points.

red75prime··on Google will buy half the electricity from one of Finland's nuclear power plants
Do you know how South Australia solved a problem of the Whyalla Steelworks when transitioning to 100% renewable electricity? They excluded on-site generation of the plant from the statistics. "Solarization" of the plant hadn't worked out economically. Continuous manufacturing processes have big problems with intermittent energy sources.

China is building tens of nuclear power plants though.

red75prime··on Rust is tier-1 language at Microsoft
...including rewrites into another language
red75prime··on A Tesla ran a stop sign and killed a man, Full Self-Driving/Autopilot was on
Fred Lambert is very biased against Tesla to the point of forsaking journalistic integrity. I fact checked a few of his articles and found a number of omissions that benefit his narrative.
red75prime··on More questions about whether researchers can trust OpenAI with unpublished math
I guess it's a fraction of problems on which a model produces a LEAN proof or a counterexample.
red75prime··on Programming is Art
Is it that complex? Whatever you feel stays inside you. From the outside you make noises, gestures, facial expressions. We can obtain fMRI data an so on. Operationalization is how to connect observables with the presence of soul in a subject.

What can you say besides "I don't feel a soul inside an LLM"? You don't feel it? OK, good for you, whatever.

← PreviousPage 5 of 34Next →