rnd33

I don't really see how it is as different as you say. Even if a system is stochastic, it can still be controlled, it's just a different type of engineering. Much of traditional engineering works exactly like this, with tolerances and probability rather than absolute determinism.

I think we (software engineers) are suffering from partial blindness to this aspect of engineering given that software has always been so extremely deterministic. We've trained ourselves that solving a problem means composing small fully deterministic parts into deterministic systems, and the prospect of having to work more with statistical confidence and managing failure modes feels scary and wrong.

But how do you source the observations used that would be the basis of your simulation? Or are you assuming the real world won't change? At least in my mind I don't see many domains that remain static enough where continuous sampling of the real world will not be required. Go and Chess are artificially stable domains, the rules are fixed.

Any social domain where there's competition of resources (salary, funding) will definitely not be stable, it would require constant testing against the real world.

Great article. I feel most people, even great software engineers, seem to lack the intuitive understanding that building software is essentially modeling a system, and that every model faces the same limit: the number of interactions between its part grows combinatorially as the system grows. If some of those parts or interactions produce no value (bloat), then complexity is added for no gain. This is true regardless of who created the model or wrote the code.

Instead they seem to think that AI can someone work around what is essentially laws of nature.

Risk calculations aside, don't forget to take into account how this could impact soldier morale. Just knowing that if injured a giant flying drone will attempt to airlift you to safety, regardless of how hopeless the situation on the ground looks like probably is a huge boost to morale.

> If you can direct AI and produce more value per $/time spent then a non software engineer directing AI to develop software then you still can produce a lot of value.

Even if true (which I'm leaning towards that it is), when will companies realize this?

We could be in for a decade long software career recession until the effects of poorly managed and written software trickles down to lost customers and company bankruptcies. Bad decisions in large companies take a very long time to produce clearly visible effects.

> As you develop your skills with AI, you now have the ability to develop your own software products.

I think this is a dead end. We're quickly approaching a state where software is no longer a product that can be sold, but rather a tool used by someone selling a service.

Completely agree. In my experience we're being more productive than ever but our projects are drowning in on the surface completely valid but on the larger scale meaningless changes. The bugs fixed were real, the features added are in some ways useful, but they really do not move the needle in any particular direction.

Isn't this a case of missing the trees for the forest though? The human brain is not an LLM, and an LLM is not intelligent in the same way as a human brain.

However, an LLM is a prediction machine, prediction IS at the very least one (or the most fundamental) element of intelligence. The brain most surely contains at least some kind of simulacrum of a prediction machine. How that prediction machine is used or wrapped is another matter.

If I said to you: "Blue blue blue, the color of my car is red", would you have absolute confidence in your prediction that my car is red? Or would the way I phrased that sentence make you slightly uncertain, and wonder if there's some miscommunication going on here?

Intelligence is compression, compression requires subtraction, and for some reason LLMs are not good at subtracting. To create a coherent model you kinda have to subtract correlations until only the essential parts are still there.

What I don't understand is why LLMs haven't been able to do this yet, if it's the harness or some orchestration layer above the LLM that is needed. Because fundamentally if you can identify correlations then it's just another small step to prioritize and remove lower value or irrelevant correlations.

I wonder if what's needed is to introduce subtraction tokens in some sense, and in post-training reward the model on that.

Completely agree. AI is very impressive in many ways but there is something deeply wrong that is hard to put into words. The output is probable but never true, if that makes sense.

I think this is also the mechanism behind why AI generated videos and images are so captivating at first. I remember when Midjourney first launched and it was hours and hours of a brain-melting "Wooooooow". But once you get used to it and start to identify the patterns the brain quickly labels most AI-generated content as blank space.

If the image or text wasn't created by a human, then there was no intent behind the content, there is no message or novel information conveyed, and it reads as noise.

I think the last one is by far the biggest reason for why this happens way less in humans. In every conversation, almost for every sentence, people gauge the response their words have on whoever is listening. If something didn't land as expected (frown, confusion, unpredicted response) you unconsciously adapt and try something slightly different.

It's immediately obvious by the fact that you're clearly using way different ways of speaking (tone, speed, vocabulary) when you're speaking with a friend, versus your parents, your colleagues, people you don't know, children, etc.

Maybe, or maybe you are overly pattern matching on what you what to be true?

I don't think anyone would mind having a competent manager or PM in a standup, someone that is actually contributing towards finding solutions and ways to move forward.

Isn't this only a transitionary problem though? Right now, during the transition there is no incentive for humans to write anything, it will get drowned in AI slop.

As time goes on, more and more people will recognize this problem and we'll develop new ways of measuring information quality and trustworthiness. Nothing about this problem is fundamental, it's just that we're in the middle of a very chaotic transition.

The end goal though (in my understanding) has never been for an LLM to regurgitate knowledge it ingested during its training. The end goal is to use the patterns and correlations found in internet data to generate an emergent prediction and problem-solving machine.

Whether that's possible is something we'll discover, but no one is throwing billions on AI companies for the hope of them building a giant natural language queryable internet information repository.

For well-scoped tasks I wouldn't say the code is bad, not brilliant for sure, but definitively good enough.

For a lot of problems "quick and good enough" is all that is required. I've used it a lot for managing my Home Assistant setup. Has saved me countless of hours.

In principle I could've done it myself, but I never would have, the time investment required to learn it wouldn't have been worth the value I get from it.

Subsidized from a capex perspective maybe, but the profit margins for API-priced inference tokens are likely very high.

Also, a lot of people get stuck on the idea that the main use case of LLMs is to generate more production code. I don't see it like that at all anymore. It's just as useful or even more useful as a problem-solving partner, generating throw-away prototypes, conducting experiments, verifying assumptions against code and documentation, performing one-off UI or smoke testing.

A large part of the world population is poor, and they do not have the same level of health problems, nor are they similarly obese. Not purely diet related, but a huge part of it for sure.

The wage structure is very flat in my country, the average wage is only around 15% higher than the median. The median wage is very livable on. For essential or basic needs I would estimate you would only need ~70% of the median wage.

I appreciate that I could be biased, and my guesses could be off, but we're talking about almost a factor two here. If the numbers would be correct the average person does not even have enough money for the minimum essentials every month, which is extremely far from what I see and hear.

I don't understand this data at all. All of the rich Western European countries has over 200h required work per month, but I don't know of anyone in my country (high or low income) working more than roughly 40h per week.

There is no electronic lock as far as I know, as many people seem to assume. It's a mechanical notch that you have to physically pull the switch past to operate it. The lock failures described in the air worthiness directive was about this mechanical stop or notch not being installed.

That’s an interesting perspective, and it makes sense it works. Thankfulness is known to provide a lot of psychological benefits, such as greater appreciation of the thing you are thankful for.

Where it goes wrong though is if we take it too far and start connecting this to some non-existent deity, which in turn makes us construct an incorrect model of the world (such as if we’re not thankful for the food, then next year there will be a drought as a punishment).

I suppose codifying beneficial practices into religion or spiritual beliefs is just part of being human.

Same experience, putting everything related to tech (printers, programming, Wifi, email) into the "IT department", which is seen as a support service to the core business of banking, instead of seeing automation as business critical in reducing overhead and managing risk.

This is why fintech (although a very overused word by now) is such a breath of fresh air in the banking industry, and desperately needed.

This is what I think a lot people don't understand when it comes to unruly IT systems in large organizations. Deutsche Bank does not have an IT or software engineering problem, they have a culture and corporate politics problem. Even if you have the world's best engineered systems, as soon as you have 30 different systems you'll have a giant mess, and the reason you have 30 different systems is because you have groups of people with different incentives not interested in using someone else's system.

That's a mind-boggling theory actually, that some general survival skills are encoded in the parent-child relationship. Basically you have to become a parent to level-up your survival skills.

I wouldn't put it past evolution to encode useful information wherever possible, but it would be hard to prove that theory.

I think it's more an indication of higher expectations in child-rearing. If you're struggling daily for survival and 7/10 of your children will die before age 6 I don't think can afford to spend hours every day experimenting which method will get your child to fall asleep without crying.

Yes, if unions exist at the company (which they did at Klarna), a company is required by law to negotiate with ALL employee organisations present at the company. This is one of the reasons large companies adopt kollektivavtal, it allows them to only have to negotiate with one union for employment changes.

Source (Swedish law): https://lagen.nu/1976:580#P13S2

Google translated:

> If the employer is not bound by any collective agreement at all, the employer is obliged to negotiate in accordance with § 11 with all relevant employee organizations in matters relating to dismissal due to a lack of work [...]

On average over a longer period variable rates will always be lower, since the risk premium due to future rate uncertainty will be lower. If that period is a 100 years it won't help lenders much though.

It's the same case in Sweden, all new lenders have to be able to handle rates of 6-8%.

What a ridiculous article... everyone understands that exponential growth cannot continue forever. The bigger and more interesting question is what the life of an average human looks like when growth inevitable slows down, and how fast we can get there.

Because the assumption that we will grow our energy use indefinitely is flawed. Energy use has diminishing returns, eventually we'll run out of useful work we want to perform. At least in terms of material wealth and comfort.

There's probably also major opportunities for energy savings that extends this timeframe greatly. Incandescent -> LED lights have reduced energy consumption by a factor of 10. Technology improvements in just the last 5 years have halved the energy consumption of air conditioners.

People are quite loyal to their grocery store. The shut down means all of their customers are forced to try out another grocery store, could hit them hard long-term.

This! Job-hunting is so ridiculously easy when everyone is working from home.