crystal_revenge

I'm so tired of the endless stream of articles that a.) don't know what "non-determinism" means in Computer Science and b.) don't really understand how LLMs work.

"Non-determinism" (i.e. the "N" in NP-Complete) does not mean "random", it means "following all paths at once" (there's a really elegant way to understand the List monad as representing non-deterministic computing). It's such a fundamental (and delightful) concept in computer science that refusing to understand what it means is a tragedy related to the loss of technical language itself.

But, even if we just sub "non-deterministic" for "inconsistent" or "random" or "stochastic" or whatever better term you come up with. Statements like the following just seem to not understand the basic architecture of LLMs:

> Given the same input prompt, a model's output is not guaranteed to be identical on every run, at least under normal running conditions.

"Normal running conditions"? The reproducibility of the output is 100% a property of how you choose to sample from an LLM. There are virtually endless ways you can define your sampler and whether or not the same result is reproducible is a fairly minor aspect of all the other choices you make when sampling.

At the end of the day, each step of the LLM gives you a distribution over next tokens, and you can do with that what you like.

In the end the article almost becomes a tautology: the output of an LLM represents a probability distribution over all the outputs it believes are valid with some variation around an expected output. Yes, that's literally what an LLM is.

> It is ultimately the understanding

But understanding to what end? I'm if it's for the pure aesthetic experience of engaging in the beauty of math than that's fine, but that's typically not the view from my experience.

The goal of understanding is to advance the field, since historically human understanding was the only way to do this.

For the vast majority of human history, the world we experienced was a strange and unintelligible space. Humans survived at the by the grace of forces beyond their control. For a brief window of time human existed in a world where they increasingly saw a relationship between their understanding of the world and there ability to steer their environment towards a desired outcome.

I believe now we're seeing that that was a historical anomaly. The world is once again entering a state where it becomes increasingly unintelligible to us. Nick Land already famously pointed out, many decades ago, that the global financial system is already a form of "artificial intelligence" we don't really control. We're seeing that spread wider and wider.

The world making sense peaked around 1960s and has continued to breakdown ever since.

If you're going to wrote a blog post on the topic, probably worth spending a few minutes double checking your understanding of the "thing". I don't doubt that the author may have been taught the wrong thing, but to write an entire post starting from and remaining in a state of misunderstanding is not particularly useful.

> YOLO generated code straight into prod

Most of my team's time is spend carefully reviewing PRs and iterating on improving new ways we can ensure the product works well, the product is hardly "YOLO'd"

> then frustrate your customers/users when things aren't working or keep shifting around.

All of these products come at the request of customers and they are generally quite delighted with the results and equally delighted with how fast we can deliver.

> For all of AI being touted as the best thing since sliced bread

I don't think it's the best thing since sliced bread, but I am telling you that your understanding is weirdly out of touch. I know HN doesn't have people working startups anymore but what I'm experiencing at work is a lot of serious engineering work and discussion around delivering quality products rapidly (as well as improving process so we can get ahead of transformations in what a 'product' is).

It sounds like you have a view of the world and want to stick to it, in which case there's not much point in arguing. If you search my comment history you can easily find around 8 months ago I would have largely agreed with you, which is why I opened mentioned that your view is "outdated". This space has changed dramatically in the last year, and continues to change in ways that surprise me.

> I believe most current improvement in speed is just moving from idea to demo in a few days

This is an outdated view.

Current timelines I'm facing are to be going from "thought", through customer trials and being fully live in the product and ready for sales in ~3 weeks (from kick off to live in app is a bit more than a week). This is for a full product feature that could easily be standalone. In 2023 I would say the timeline for a similarly shaped feature at another startup was around ~3 months (and the team at the time agreed that was an aggressive timeline). Bug rates are not noticeably different than other teams I've been on in the past 20 years.

Nobody I know working in startups is still building demos with AI like they were a year or more ago (for work), that's seen as largely a waste of time since you can just ship the feature and be experimenting with customers much faster.

On top of that everyone working in startup land knows that SaaS's days are numbered, so you need to be shipping working software fast enough you can get ahead of the curve to navigate where things are going next.

I take it you're not working at a startup?

> A good engineer, without LLM assistance, will still produce great stuff.

Not fast enough to keep their job these days.

Time was always the limiting factor to code quality, good engineers satisfied the classic "good", "fast" but not "cheap" selection of those three classic options.

I very sincerely doubt it is possible for even an incredible engineer to keep up with the delivery schedules required to ship products now. Not to mention that frontier models do ship pretty consistently good code. By far the biggest source of issues I see is not "poorly coded" but "problem poorly specified". We still need good engineers, because they can understand how to do decompose problems well, but I don't know anyone who writes code by and anymore (other than for fun).

I feel like I've seen this headline (followed a week later by its inversion) countless times in the last 6 months. I have little doubt this is a bubble, but also very little certainty as to when or how it will "correct".

The KOSPI is almost not worth talking about as any serious signal. It has a circuit breaker drop almost weekly (again followed in no time with an equally high rebound) and basically has come to represent how insanely the South Korean market has become pure gambling (with retail investors absurdly leveraged). Sure , it's hard to image this doesn't lead to some disaster in the long run, but these fluctuations have become par for the course.

I don't personally know anyone who writes code any more (for work). Is this not the case for you?

The distinction I see now is between teams that read PRs and teams that don't. I still think the former is a good approach... for now, but I don't expect this will necessarily be the case in a year (or less).

This article was such an inspiration to me when I was younger, and the advice, for that time, was very correct. However seeing it today really drives home how big the gap is going to be in really understanding code in just a few more years.

> predicting the future is hard

As I mentioned in another comment, this can also be rephrased as "predicting data with values outside the range you trained on typically doesn't go well". If you tried to predict some health metric based on weight and height but you only had people under 4' 10" and less than 120lbs you wouldn't be shocked at all if it worked terribly when applied to American football players.

Time-series forecasting is hard because you are always going to be predicting based on data outside of your observed range ("forecasting" does go much better when you're trying to fill-in-the-blanks of things that happened in the past).

> I’ve been thinking recently about what makes time series forecasting problems so difficult compared to other sequence learning tasks

Whenever I teach people time series forecasting, I always point out that one of the biggest challenges is that you will always have values at prediction time that are out side the range of values observed during training (specifically the value of t).

In plenty of other machine learning and statistical modeling tasks this is not the case. You can train on every token you'll ever see and every pixel value you'll ever see, you can do regression analysis on every categorical value you include and an least an observation from within a range of every continuous and discrete value you'll observe. But with forecasting you will always have values you predict that are outside the range of anything you trained on.

You would run into similar problems if you tried to create a statistical model of the density of water given a temperature but your training data only included values between 0-100 C and you went out and started predicting values covering all the temperatures found on Earth.

For whatever reason, when time is a variable we somehow think it is immune from the obvious limitation of predicting on values outside of the range of values you trained on.

This comment is a great example of how large and strange the skills gap in AI is right now.

Curious why your first impulse is not simply to point your favorite agent at a few examples and start brainstorming/planning from there?

Multiple times I’ve built a purpose specific bespoke tool starting this way. In fact, it’s a great way to learn how specialized tools are built.

On the path we're on I don't think we'll have to worry too much about future generations.

The EU has already seeing 10,000 excess deaths from climate changed caused heat waves and this is a minuscule taste of what's to come.

A very large percentage of mass extinction events have their roots in increased atmospheric CO2, but all of them on dramatically increased time scales. The closest thing in the history of the planet to what's happening to day was PETM [0] and that was only a lessor extinction event because the Earth was already quite warm (for example, there was already no polar ice at the time).

0. https://en.wikipedia.org/wiki/Paleocene%E2%80%93Eocene_therm...

> EU basically throws money anything "green"

Including paying to have wood pellets shipped across the Atlantic using bunker fuel and then calling it "bioenergy"

The EU is 'clean' largely out of it's own limited access to fossil fuels and other energy resources rather than because they are "doing their part".

> There’s an uncensored model floating around that you can run locally with llama.cpp

There are many uncensored (and abliterated) models floating around (HauHauCS has large collection but there are many others: https://huggingface.co/HauhauCS). I'm using `Qwen3.6-35B-A3B-Uncensored-Q4_K_M` (the one referenced in your link) because I find it's writing style much more interesting when you push go off the guardrails a bit, and because I think self-censoring when effectively using an advanced journal is variety of dystopian I'm not ready to accept

> it’s annoying to use since you run out of context window quickly, and it’s certainly not able to be deployed in production (i.e. Tom Riddle’s diary as a service).

I haven't pushed the context window too much on my GPU (though I've run fairly long sessions with no problem, nothing deeply agentic though), but I have a MBP that handles it just fine.

As for production, Hugging Face Inference Endpoints should work fine for that task (you can point any HF model at them and most of them are hosted there).

> For better or worse, fun is no longer allowed.

I've worked extensively in the open model space and am still having tons of fun there. If anything it's gotten aggressively better in recent months.

> understand theorems for which we comprehend

I don't know what your distinction between "understand" and "comprehend" but my point was not about these words, but about being "useful" and being "understandable".

I'm saying there's no relationship between a mathematical statement being useful and it being understandable.

If it is true that "understanding is a prerequisite for usefulness" (where "understanding" means that a statement can be proven in a way that is intelligible to humans) was a property of mathematical expressions, then this fact would certainly be useful (we could exclude any statements that no human understand from the world of useful mathematical expression). But, by that definition, we would need to understand that statement, so you would need to be able to prove that "understanding is a prerequisite for usefulness" in a human intelligible way.

Now what I just wrote is in itself not a proof that we can't know, but proving the above statement would involve expressing the claim in a mathematically verifiable way that was also understandable by humans, which does imply something remarkable about human cognition (something that would be intelligible no less!)

> Things that aren’t human intelligible aren’t human usable

This is objectively false, people use things every single day they don't understand. We still have plenty of things about the world we don't understand but still find useful.

You are saying anything we know to be the case, but cannot understand why cannot be used? Can we just stop sleeping because we haven't reasoned why sleep is necessary even though we know it is necessary? I mean we still don't really understand gravity (we know how but not why)

> but I would like to understand the problem, too

But why should it be the case that this is always possible?

It's entirely reasonable that the set of useful mathematical proofs is a proper superset of human intelligible useful proofs.

In fact, to argue the contrary would imply there is something incredibly remarkable about human cognition.

> Who in their right mind would merge a 200,000-line unaudited vibe-coded blob

Anyone who understands type theory and how theorem provers work? It's sort of akin to saying "how do you know that a massive C++ program that compiles to machine code compiled to the correct machine code that will actually run and it's just not a random string of bits!?!?!", you know because the compilation would have failed otherwise(this is different than saying the program behaves correctly, but that's precisely the difference between formal proofs and compiled programs).

The entire argument both you and Bessis are implying is that mathematics must be human intelligible. But there's absolutely no reason to assume that every mathematical statement must have a human intelligible representation. There is also not reason to assume that if we restrict ourselves to the subset of mathematical statements that are human intelligible that this is of any use.

Just because people who don't want this to be true, and I can understand the motivation, doesn't mean that it isn't still the case.

Most American ex-pats don't really understand that the thing that makes ex-pat life so attractive is that, for most of people's lives, being American in a foreign country has traditionally conferred a wide range of benefits (this is most clearly exemplified by the way Americans living in a foreign country refer to themselves as "ex-pats" not "immigrants"). The ex-pat solution assumes American exceptionalism as its foundation.

Historically Americans have benefited from income asymmetry and a fairly wide-spread desire by foreign nations not to cause too much legal trouble for US nationals abroad.

I have quite a few friends that do live, quite happily, abroad. But the common pattern for them is a.) fluency in the native language b.) historical association with the country c.) fairly large cash reserves so they can ignore any economic problems these countries are facing.

> AI psychosis might be the new normal for our industry

I've been fortunate enough to have worked on multiple AI intensive engineering teams (both on the product and research side) where considerable effort was spent reasoning through how AI was changing things and we were consistently evolving our practices. But they've all been orgs with 50 people less.

AI psychosis seems to effect very large tech orgs in a different way than small, high impact teams.

In small startups, at the end of the day, if the team doesn't ship a quality product, the company fails. Most importantly, every individual still bares the responsibility of their work. Personally, I've seen a lot of thoughtfulness around things like bad PRs because, on good teams, people realize we're all struggling to figure this out. But nonetheless, if something doesn't go well, there's always an individual that needs to figure out how to make it better. Virtually all the things I've learned about functionally shipping products built with and using AI have come from teams like this. Software engineering is changing, but for those of us shipping products, it reminds me a lot of the early webdev days when we were all trying to figure out the patterns to make this new world of software work reliably (anyone who recalls the pre-jQuery JavaScript days will remember how much we had to figure out before webdev could become what is today).

In large tech orgs there's a much, much larger disconnect between employee effort and concrete value delivered and similarly much larger diffusion of responsibility. When accountability is abstract and nobody is quite sure what the real value of their work is, then there is fertile ground for AI psychosis to run amok. In part this is because there is a certain latent psychosis in these larger orgs anyway; who's "productive" and what's "valuable" always requires a bit of imaginative story telling, not necessarily grounded in reality.

However, I don't think this will persist long as the "new normal". Just like in the rise of web application development, smaller teams will charge ahead and figure some of this stuff out. The MVC pattern applied to webapps, increasingly powerful JavaScript frameworks and best practices, agile practices, git and the popularization of github, the use of No SQL for scaling etc all primarily where battled tested by smaller, high velocity startups and now lay a foundation I'm sure some contemporary devs don't even realize needed to be built by anyone.

> in windows terminal

This is an aside, but I'm really struck by how many people on HN use Windows (based on repeated mentions I've seen in comments). I've worked for a pretty wide range of companies over the last decade and only one, maybe two companies even had any people that worked on Windows machines. I haven't worked at a company where devs used Windows in 15 years (and even that company eventually switched to linux).

As I've gotten deeper into LLMs/AI roles even Macs have seemed to start having equal share compared to devs running full Linux setups.

Is this just a sign of that a larger and larger portion of HN users are working for large corporations? I honestly can't even remember that last time I saw a serious developer pull out a Windows laptop.

> I hate being told what technology I can and can't use

Ever since the original GPT-2 "it's too powerful to release!" I've realized that whatever is the current state of open models represents what we really have access to.

It's shocking to me how many people on HN, who engage in long conversations about LLMs and AI, have never actually run a model on their own hardware.

All you need is a reasonably good macbook pro/studio or an RTX [3-5]090 and you can run useful models in the >= 30 tokens/second range (much higher if you choose the GPU path). The difference between what you can run on this hardware and what you can run on hardware that costs 2-5x is not that big. Don't be fooled by people on Twitter/X claiming you need some outrageous setup.

It's also increasingly clear that frontier models are nowhere near close to pushing the limits of efficiency. Quantization, MoE, and other techniques have dramatically improved even in the last year.

For work, of course use OpenAI/Anthropic models, but for anything personal, anyone who considers themselves a "real engineer" should be running local models, using open harnesses and seeing what they can accomplish with these.

Even if open releases slow down or even stop, we have the foundation, right now, for smart engineers to squeeze something quite useful out of. Hopefully we'll one day figure out how to train large models in a federated way. But either way: not your weights, not your inference.

For pure, weird, late-night LLM chats, I've recently started using Qwen3.6-35B-A3B-Uncensored just running with llama-cli and it is a very refreshing chat experience.

Uncensored model means it will not deny any requests (at least I have yet to come across one), if you grew up in the 90s it sort of feels like coming across the anarchist cookbook for the first time (though with more accurate content). Using llama-cli means the session is entirely local and entirely ephemeral. As a bonus all the reasoning steps are fully visible to the user.

The base Qwen3.6-35B-A3B is more than adequate for "weird late night brainstorming chats" and I've really started to dislike the natural tendency to self-censor when the model is willing to refuse (and potentially report) any requests it feels is "inappropriate" and all these private thoughts are stored on someone else's server.

The parent comment is describing a test they ran so they could assess their trust in the model for scenarios they don't have time to fully understand.

Do you not believe in running tests, evaluations, or experiments at all to better understand your environment?

The ROI in the case of a positive outcome is the reduced time needed to inspect the results in the future (the entire point of AI is to know what you can trust it on, so you can delegate everything at that level with less oversight). The ROI in the negative case is the tokens not wasted on tasks to ambitious for the model.

We've lived in a software bubble for so long, most software engineers have completely forgotten that the purpose of (most) software is to solve a problem. If that problem solves the problem well and reliably it doesn't matter the quality of the code.

In fact, that's the entire reason we care about "quality code", because we assume that quality code is code that does what you expect well and consistently.

I say this as someone who hand writes code pretty much every night for fun, just to experiment with computation. Which, oddly, is more fun than ever because I don't feel like there's any need to connect this type of programming with "real world software", and I can really enjoy code for it's own sake, meanwhile my job is mostly just running agent loops (which I quite like as well).

I'm going to guess you're young enough that you don't remember when HN first appeared?

For anyone that was an adult in the "web 2.0" days it's hard not to see HN as "social media". The first wave of social media sites where defined by community news aggregators that allowed commenting and, most important, up voting of comments and submissions. Digg, Reddit, HN, del.icio.us (though it lacked formal 'up voting') were all part of this first wave of social media.

The absolute key differentiator between HN and an old school internet forums, that absolutely makes it "social media", is that the community votes on your opinion and users have some way to score against each other. This is precisely the mechanism that is at the root of all problems in social media: you get a measurable reward for your content that pressures you towards saying things and sharing content that increases that reward.

Perhaps one of the best decisions HN ever made, which fights this somewhat, is removing the upvote count from being visible to other members of the community (this was not the case in the early days of HN). But for anyone that saw the rise of "social media" it's hard to imagine HN not fitting that description.

It's not interesting to say "this is a bubble!" I've heard that about virtually everything (and in many cases it's likely true). What is interesting is pointing out the mechanics that make the bubble pop.

This is precisely what makes the movie the Big Short interesting: we see that people did identify, within a reasonable time frame, when people would start defaulting and how that would cascade into a true crisis.

It's pretty clear that while the fruits of AI are quite useful, the entire thing is rife with very questionable financial engineering... but I still don't know what it is that makes all of this break. For example, it's obvious that the SpaceX IPO is a massive wealth transfer program, but it's not obvious that it will immediately end in a crash. Given how irrational the stock market has been, I don't see a reason it can't continue to be irrational for long after the bag has been handed over to the retail investors and retirement funds.

> I sincerely hope the market is not willing to value this sort of deal at a P/E ratio anywhere near 94.

It will very likely be valued much, much higher. The SpaceX IPO is, in itself, a marvelous piece of financial engineering (requiring co-operation among multiple actors) which has been a long time in the works.

- Right out of the gate nearly all retail investment platforms have dramatically reduced requirements for purchasing an IPO, most notably Fidelity, which previously required $500,000 in your account to participate in an IPO reduced (on Friday) this amount to $2,000

- Retail investment, despite being quieter in the post-WSB era, is at all time highs.

- Reports are that the SpaceX IPO is already highly oversubscribed, meaning there are many more retail investors interested than there are shares available.

- SpaceX has a wildy low float of only ~4% which means price discovery will be much slower then normal, especially with aforementioned demand

- All of these retail platforms enforce some sort of "soft lock-in" whereby you're excluded from future IPOs if you sell your shares within 15-30 days. So if you want to get out you're not going to be able to participate in Anthropic/OpenAI IPOs in a few months.

- Coincidentally, most of the major indexes (thankfully excluding the S&P 500) have adjusted their rules to require only 15 days post-IPO before inclusion and have no profitability requirements. Many also adjusted the rules so that low float IPOs have their weight multiplied despite the low float.

- Many retirement accounts, in one way or another, are required to track these indexes and will be forced to buy these SpaceX shares at a very likely frenzied price and further drive the price up.

SpaceX will very likely open with far more retail demand than shares, the insiders (VCs, employees etc) will still be legally locked from selling, retail investors are penalized if they sell, and so the demand will be high and supply very low.

If they can keep this demand hyped for just 3 weeks, price will still be elevated when retirement accounts are forced to buy... roughly the same time retail investor start seeing the penalty for selling expiring (meaning it is not irrational at all to be in the IPO, but it is irrational to sell before being listed in an index).

Fun fact: the other fascinating thing about this IPO is the terms for insider lock-in. At first earnings (Jun 30) inside investors unlock and can therefor liquidate 20% of their shares... but if the stock performs well, they can unlock and additional 10%. There are additional rules for continued unlocking of more shares depending on performance as time goes on. So everyone on the inside has a very vested interest in a spike in stock prices: not only will their stocks be worth more, but they can realize that value faster.

I would be surprised if SpaceX price doesn't explode in the first few weeks because for everyone involved this would make sense. It's only in August that we'll start seeing the really interesting things start happening.

The "good time" to discussion unionization would have been about 10 years ago when employees had much more leverage.

But I quite vividly remember any mention of that here on HN back then was responded to with "I'm paid great and can easily change jobs why would I want a union?" (with many engineers only thinking of factory worker unions as a model and forgetting that very highly paid and in demand actors also belong to a union).

You negotiate when you're in a position of strength, not while your value is rapidly falling through your fingers.

With AI and a growing population of ex-corporate workers desperate for work breaking up attempts to unionize would be easier than ever.