exfalso

There's a fun hypothesis I've read about somewhere, goes something like this:

As the universe expands the gap between galaxies widens until they start "disappearing" as no information can travel anymore between them. Therefore, if we assume that intelligent lifeforms exist out there, it is likely that these will slowly converge to the place in the universe with the highest mass density for survival. IIRC we even know approximately where this is.

This means a sort of "grand meeting of alien advanced cultures" before the heat death. Which in turn also means that previously uncollided UUIDs may start to collide.

Those damned Vogons thrashing all our stats with their gazillion documents. Why do they have a UUID for each xml tag??

Perhaps the out of job prediction is actually reversed. True, LLMs will become an efficiency increasing tool. But in terms of job security, doesn't that mean that if your whole job can be driven by an LLM then demand for that job decreases?

In other words, people claiming these high productivity increases may be the ones at actual risk. Why employ 3 people when 1 can write the prompts?

> Maybe when AIs are able to say: "I don't know how this works" or "This doesn't work like that at all." they will be more helpful.

Funny you say that, I encountered this in a seemingly simple task. Opus inserted something along the lines of "// TODO: someone with flatbuffers reflection expertise should write this". I actually thought this was better than I anticipated even though the task was specifically related to fbs reflection. And it was because I didn't waste more time and could immediately start rewriting it from scratch.

I have the same experience and still use it. It's just that I learned to use it for simplistic work. I sometimes try to give it more complex tasks but it keeps failing. I don't think it's bad to keep trying, especially as people are reporting insane productivity gains.

After all, it's through failure that we learn the limitations of a technology. Apparently some people encounter that limit more often than others.

Interesting. I've had it fail on much simpler tasks.

Example: was writing a flatbuffers routine which translated a simple type schema to fbs reflection schema. I was thinking well this is quite simple, surely Opus would have no trouble with it.

Output looked reasonable, compiled.. and was completely wrong. It seemed to just output random but reasonable looking indices and offsets. It also inserted in one part of the code a literal TODO saying "someone who understands fbs reflection should write this". Had to write it from scratch.

Another example: was writing a fuzzer for testing a certain computation. In this case, there was existing code to look at (working fuzzers for slighly different use cases), but the main logic had to be somewhat different. Opus managed to do the copy paste and then messed up the only part where it had to be a bit more creative. Again, showing the limitation of where it starts breaking. Overall I actually considered this a success, because I didn't have to deal with the "boring" bit.

Another example: colleague was using Claude to write a feature that output some error information from an otherwise completely encrypted computation. Claude proceeded to insert a global backdoor into the encryption, only caught in review. The inserted comments even explained the backdoor.

I would describe a success story if there was one. But aside from throwing together simple react frontends and SQL queries (highly copy-pasteable recurring patterns in the training set) I had literally zero success. There is an invisible ceiling.

Exact same experience.

Here's what I find Claude Code (Opus) useful for:

1. Copy-pasting existing working code with small variations. If the intended variation is bigger then it fails to bring productivity gains, because it's almost universally wrong.

2. Exploring unknown code bases. Previously I had to curse my way through code reading sessions, now I can find information easily.

3. Google Search++, e.g. for deciding on tech choices. Needs a lot of hand holding though.

... that's it? Any time I tried doing anything more complex I ended up scrapping the "code" it wrote. It always looked nice though.

Highly recommend https://grml.org/zsh/

For nixos users: https://discourse.nixos.org/t/using-zsh-with-grml-config-and...

Super easy to setup, and works very well

Although it's sad, I have to agree with what you're alluding to. I think there is huge overhead and waste (in terms of money, compute resources and time) hidden in the software industry, and at the end of the day it just comes down to people not knowing how to write software.

There is a strange dynamic currently at play in the software labour market where the demand is so huge that the market can bear completely inefficient coders. Even though the difference between a good and a bad software engineer is literally orders of magnitude.

Quite a few times I encountered programmers "in the wild" - in a sauna, on the bus etc, and overheard them talking about their "stack". You know the type, node.js in a docker container. I cannot fathom the amount of money wasted at places that employ these people.

I also project that actually, if we adopt LLMs correctly, these engineers (which I would say constitute a large percentage) will disappear. The age of useless coding and infinite demand is about to come to an end. What will remain is specialist engineer positions (base infra layer, systems, hpc, games, quirky hardware, cryptographers etc). I'm actually kind of curious what the effect on salary will be for these engineers, I can see it going both ways.

Same experience here, using new models. Every time it's a disappointment. Useful for search queries that are not too specialized. That's it.

Wow one of the most insightful takes I've read in a long time, hidden in the depths of a HN thread.

The mechanism you talk about is all too real, and it can be semi-consciously exploited. A bit off-topic, but the way genocides happen is via a similar mechanism. A group is depicted as base, a disease, faults are attributed to it. It becomes a moral imperative to clean the body.

This feeling is thus rationalized, and people start to reinforce each other's conviction. It becomes a twisted status game: the more you lean into the mentality, the higher you rank. Often times the goal isn't even to "address" the original trigger, it's to show your peers your approval and distance yourself from "those people". They the immoral, we the moral.

Mostly feeling like a caveman. I've been trying and failing to use it productively since the start of the hype. The amount of time wasted could've been used for actual development.

I just simply don't get it. Productivity delta is literally negative.

I've been asking to do projects where I thought "oh, maybe this project has a chance of getting an AI productivity boost". Nope. Personal projects all failed as well.

I don't get it. I guess I'm getting old. "Grandpa let me write the prompt, you write it like this".

Hmmmm.. I am working in a niche domain (Confidential Computing) and the work is fairly creative, although I wouldn't say I asked it domain-specific things. I didn't ask it to come up with encryption schemes or security protocols, I learned very quickly that it cannot even start on those problems. "Design discussions" were just sycophantic affirmations of whatever I wrote. What I mostly tried were "add this function" or "refactor this based on XY" or "analyze this piece of code for race conditions".

(Un?)fortunately my work doesn't involve a lot of "drone coding". With personal projects I let it do whatever it wanted including picking the language and libraries. With one of them it ended up so confused with the Redis API(!!!) that it kept going back and forth between different versions to "fix" the issues until it literally removed the functionality it was supposed to add. Problem solved, eh?

Hmm not my experience. I've been aggressively trying to use both Cursor and Claude Code. I've done maybe 20-30 attempts with Code at different projects, a couple of them personal small projects. All of them resulted in sub-par results, essentially unusable.

I tried to use it for Python, Rust and Bash. I also tried to use it for crawling and organizing information. I also tried to use it as a debugging buddy. All of the attempts failed.

I simply don't understand how people are using it in a way that improves productivity. For me, all of this is so far a huge timesink with essentially nothing to show for it.

The single positive result was when I asked it to optimize a specific SQL query, and it managed to do it.

Anyway I will keep trying to use it, maybe something needs to click first and it just hasn't yet.

Eh. It's a stretch to call it "broken"

My reading of the status quo is that people who use it for toy or greenfield projects written from scratch are having a blast. Until the project reaches a certain complexity in size and function when it starts to break down.

People working on existing projects in turn are scratching their heads because it's just not quite working or providing much of a productivity boost. I belong to this camp.

Also tried it with Python. The autocomplete there was ok-ish(although to me the "wait for it -> review suggested code" cycle is a bit too slow), but getting it to code even standalone well-defined functions was a clear failure. I spent more time trying to fix prompts than it took to write the functions in the first place.

Exact same experience. I have no clue what other people are doing. I was hunting for use cases where it could be used and it kept not working. I don't get it.

I was skeptical at first, but after trying to use it several times for work (Cursor/Claude) I concluded that it's worse than I thought. Lot more time wasted for little to no results. Maybe for boilerplate frontend code or TODO app generation it's good or something?

Anyway in a couple of months I'll try again. Until that time, I guess the search result summaries are nice?

I'm more and more confident I must be doing something wrong. I (re)tried using Claude about a month ago and I simply stopped using it after about two weeks because on one hand productivity did not increase(perhaps even decreased), but on the other hand it made me angry because of the time wasted on its mistakes. I was also mostly using it on Rust code, so I'm even more surprised about the article. What am I doing wrong? I've been mostly using the chat functionality and auto-complete, is there some kind of secret feature I'm missing?

It is not as easy to see as with GraphQL because the term is overloaded/ill defined, and there is a large number of ways to interpret how OOP should be used. Probably why it took so long to realize the issues and move away from the concepts.

Sidenote: with GraphQL you will also see a lot of people who are scratching their heads saying "Surely I must be doing something wrong, it can't be this bad. I just don't understand what that something is.". When the users of your tech have difficulty even articulating what the issues are, that's when you know you hit the anti-design jackpot.

In terms of whether a shift away from OOP has happened, you can look at how modern languages/language features are designed, and how "best practice" has changed: they are explicitly moving towards data-centric design with FP concepts(e.g. ADTs, "plain" functions, ironically similar to C), and moving away from dynamic dispatch. And when they do implement dynamic dispatch, there are warning signs around it, it's not considered good practice (see e.g. trait objects in Rust). Encapsulation/visibility is now treated as an orthogonal concept (as it should be), the whole getters/setters visitor-pattern/proxy-pattern/etc nonsense is essentially binned, they're not even mentioned anymore outside legacy codebases and Java shops.

In terms of what the issues are, it's inheritance/subtyping and dynamic dispatch, IMHO the two hallmarks of OOP. (I know, purists will point to a different kind of definition, but in my experience this is how OOP is used in practice). There are some additional oddities like "design patterns" and "encapsulation" and whatnot, but I would consider these as secondary fallout issues, stemming from the two major problems and a zealous shoehorn-y attitude. This attitude BTW is characteristic of hype waves.

Ok so what are the issues with inheritance?

From a type-theoretic point of view it's variance. When you have a (parametric) polymorphic type and you shove a type into it with subtyping relations, you'll get problems. The classic example is the array of Animals/Dogs/Cats case. At the time of the OOP hype's peak, advocates kept bringing up this example as if it was a tool to understand OOP better. But instead, it's a great tool to understand how subtyping can lead to intractable reasoning about variance for no good reason. "Composition over inheritance" is now commonly understood as a good preemptive tactic. (Composition is not an OOP concept.) From a data-structure point of view the issue is fields. Can/should a subtype access the parent's fields? How about multiple inheritance? Throw in the private/protected/public trio and you'll get pieces of code so hairy that you'll just end up writing wrappers around it instead of trying to understand it.

Sidenote: I do want to note that there is a single use case of inheritance I know of which actually makes sense, but it's very specific to C++. It's "plain" (non-virtual) inheritance for the purpose of composing memory layouts while still allowing pointer aliasing. Row polymorphism (see e.g. TypeScript) is also useful, but it's difficult to argue that it's an OOP concept.

Ok, what are the issues with dynamic dispatch?

From a code design point of view the issue is, for lack of a better expression, missing alternatives. OOP-supporting languages "solved away" ADT-style dispatch (enum data structures+pattern matching) which has a very natural mental model, and replaced it with things like double dispatch and code-data coupling. When I'm writing a piece of Java code and I want to express that there are 3 known states that the data can be in, I have no choice but to use an abstract State class, add dynamic dispatch and either tightly couple the state-handler code with the state definition, or bloat the code further with double dispatch. I literally cannot do anything else, unless I start writing completely non-idiomatic code.

From a code organization point of view the issue is "open sums". Open sums(aka interface-like views on concrete implementations) do actually have their place, however OOP encourages their use way beyond what's reasonable. This results in code that's very difficult to read, it becomes time consuming to piece together what the hell a function is actually doing, because all you see are calls to interface functions. What you need to do as you read the code is essentially "close the sum" and substitute the various possible implementations into the control flow, which results in a conceptual blowup of the possible states of the program. That, or you just observe how the code is behaving at runtime. I'm not ashamed to admit that I've done the latter quite a few times with legacy codebases because the code itself was in a worse shape than the execution stack.

From a debuggability point of view the issue is similar, it's an intractable CFG. During debugging you need to re-build a conceptual mental model of "runtime behaviour" where you need to find the conditionals that actually decide what concrete objects are being dispatched to under what circumstances. Anyone who has any experience debugging such pieces of code will know exactly what this means.

Sidenote: The FP hype can also result in very similar CFG issues, but this time it's due to overuse of lambdas/closures.

In terms of the damage.. well, it's the countless hours of writing, maintaining and debugging these codebases. It's the lost dev hours, the lack of progress, the "stalling" of codebases after becoming a giant mess. I'd wager it threw back software engineering as a discipline maybe 10 years and by extension humanity as well (I wanted to say more, but the natural sw company/startup birth-death cycle mitigated a lot of the legacy-buildup issues, at least from what I've seen in my career).

They don't have a "hard" persistence layer, it's all in-memory, which is the only reason they can have any sort of performance.

In terms of the actual queries, you'll find a bunch of prefetch calls and dumping of all data related to your user. So much for "fetch what you need".

What you will also find is, surprise surprise, not actually a whole lot of graphql queries. Instead, a bunch of plain REST-ish queries, custom WS, and some graphql.

GraphQL is an abysmal technology.

In terms of destructive power to the software industry it's almost on the level of OOP. The one positive seems to be that engineers are catching on to this fact at a much faster rate(5-10 years instead of 10-20). However, unfortunately, GraphQL sits in a position of the software stack that's very difficult to swap out, so even when engineers do realize the damage, they are stuck with it unless they change jobs. Extremely frustrating. Yes I'm one of those stuck engineers.

And then you have articles like this still touting it as some kind of rainbow miracle.

I'm not even going to list the countless issues you will be encountering by committing to this tech, or the fact that the mitigation of those issues will result in essentially gravitating back to REST with several unnecessary and convoluted translation layers. Instead, I invite you to research how Meta, the company behind the technology, is using the technology in practice. Just open Facebook and look at the /graphql queries, or read some articles by Meta engineers.

Yes, 100%.

I've mostly only dealt with IO-bound computations, but the contention issues arise there as well. What's the point of having a million coroutines when the IO throughput is bounded again? How will coroutines save me when I immediately exhaust my size 10 DB connection pool? It won't, it just makes debugging and working around the issues harder and difficult to reason about.

It's interesting to see an almost marketing-like campaign to save face for async/await. It is very clear from my experience that it was not only a technical mistake, it also cost the community dearly. Instead of focusing on language features that are actually useful, the Rust effort has been sidetracked by this mess. I'm still very hopeful for the language though, and it is the best thing we've got at the moment. I'm just worried that this whole fight will drag on forever. P.S the AsyncWrite/AsyncRead example looks reasonable, but in fact you can do the same thing with threads/fds as long as you restrict yourself to *nix.

Avoiding context switches is not a problem async solves. A threadpool popping work items from a queue(or something like lmax disruptors) has the same effect on context switches. The only thing one could argue is that the async runtime's threadpool "homogenizes" work. Again, yet to encounter a case with web services where this was the issue.

I'm not familiar with FPGA scheduling, are there any resources that explore the issue? I was under the impression it's akin to GPU compute where the main bottleneck is the bus.

Suspendable computations like regex statemachines are actually a good example use case. However I'd still categorize this as extremely marginal. I have seen literally zero Rust crates that handle suspendable computations (ofc outside of the scheduler runtime). Also note that by their nature regex statemachines do not actually necessitate the use of async, it's effectively syntax sugar on top.

My point with GPU/FPGA is that if you're at the point where this level of optimization matters then you're actually dealing with situations where it's worth to invest in the big boy tools. Examples are HPC in fintech (low latency trading) and scientific computations, game development, video codecs etc. You know, "actual" computations.

Webservices are not in this category. Generally speaking with web services your goal is to "hide in the shadow of IO". If you max out your network and/or database and/or filesystem capacity, further CPU optimizations will have literally no effect. I have yet to encounter a web service where this wasn't the case.

What I do see sometimes with webservices is simply unnecessary compute, bad internal structuring, dynamic dispatch, parallelism overcommit, lack of batching, fragmented apis, fragmented data accesses etc etc all of which appear as CPU capacity saturation and also sometimes as kernelspace overhead in profiles. Async does not help solving any of these issues. And once you do solve them, you reached IO boundness and it doesn't matter anymore.

Again this is just my experience, and I'm happy to learn about what kind of web service can utilize coroutines with measurable performance benefits over a managed threadpool.

TLDR summary:

1. Straw man 2. Talk about an unrelated paper 3. Conclusion: I am smart and those people are dumb

On a somewhat related note: I'd say 95% of the software I've written was IO bound(and if it wasn't at the beginning then the goal was to make it IO-bound), and that remaining 5% does not benefit at all from async/coroutines(used them in Haskell, Kotlin(Quasar) and Rust). I'm very curious about what real-world CPU-bound use cases people have that can benefit from async performance-wise. If you're optimizing on that microsecond-nanosecond scale then you shouldn't even have yields/blocks on your hot paths, and synchronization will at most be done using memfences, so what are we talking about?

I'd also conjecture that if you place your problem domain on the IO vs CPU bound axis, the problems where async would provide performance benefits can invariably be solved by other designs that perform even better (GPU/FPGA).

Yes I'm one of those people who thinks coroutines have extremely marginal actual value, and most of that value is the "feeling of how cool this is" that people experience when they first learn about the concept. If there was a way to bin the concept altogether I would do it in a heartbeat. As-is, the entire Rust ecosystem is suffering heavily because of it.

Hah, sounds like a fun debugging session!

Those are exactly the kinds of problems I've encountered with "too many processors for too few resources". At the workplace where we used Java we used a library called Quasar which implements green threading resembling Rust async (it rewrites the bytecode into a state machine). I remember encountering a very similar deadlock, except the issue was caused by certain green threads "handing over" database connections to other green threads, and in the process yielding to the scheduler. Under high churn there was a chance that all connections ended up in the suspended set, causing a deadlock when other tasks were trying to allocate. It took a couple of days to track down because attaching a debugger and even printing caused the issue to go away.

Your example is also a fun one, but to me it actually shows exactly why an unbounded/dynamic number of processing units are an issue. Coroutines are the extreme example where you are almost encouraged to launch as many tasks as you can.

That's not my point. The issue is that when the blocking happens, with coroutines control is yielded to the scheduler which will now schedule other tasks. Those tasks may again request and block on resources. This is where the leak is coming from. A resource pool is one way to get around this, however this stops working if you have several kinds of resources.

On the other hand, with threads the IO block is a "proper" block. No new task will be scheduled, the thread will only continue when the IO operation finishes, providing a very natural backpressure mechanism that prevents overallocation/contention.

They do happen with threads indeed, but it's a lot easier to manage and the code is straightforward. You can essentially tie your resources to your threads, and your threadpool's size becomes your resources' allocation limit. In fact one common strategy is to use thread local storage to cache certain resources like RNG or regex statemachines/parsers, and you have essentially a guarantee that you'll be fine(ofc as long as the code doesn't launch threads left and right).

Again, I'm not saying that you can't manage this with coroutines, but it is harder, and I've seen this problem pop up in very different contexts/languages(Kotlin/Java, Rust, Haskell) whenever coroutines are used. Large churn or an unexpected bottleneck(e.g. network saturated or disrupted) causing blowup of suspended coroutines that in turn leak resources without applying proper backpressure. When this happens coroutines tend to be harder to debug as well.

Plain threads work for disk IO, network IO, memory usage etc etc, and the resource allocations also compose well. E.g. classic flipped order resource alloc: coroutine 1 allocates bounded resource A, then tries to allocate B and yields. Coroutine 2 allocates B, then tries to allocate A, bam, deadlock once A's resource pool is depleted. Essentially suspended coroutines have hidden dependency edges on one another through resources. With threads, the thread pool size aligns precisely with the resource pool size, under high churn it will be the resource's actual slowness that blocks the pool temporarily, but there's no way to deadlock on resource allocations.