I enjoyed reading this article. I think it's an important article. Even though people might disagree about certain definitions like "maintainability," I liked what the author did by sticking to objective measurements which do correlate somewhat with the things we care about: Ability to implement the product fast, with few tokens and with as few lines of code as possible in the final product. These are well chosen metrics IMO.

Fewer lines of code (per piece of functionality) does, in fact, correlate strongly with maintainability. The downside of this metric to watch out for (as Goodhart's Law would have it) is to ensure that the framework isn't compromising (current or future) flexibility in an unacceptable way. I imagine if this kind of benchmark becomes popular, people would start coming up with low code frameworks which are extremely rigid and inflexible. Because rigidity is a good way to cut lines of code. There needs to be a way to ascertain the flexibility limits of the framework too.

Would be good to have a similar benchmark for a more complex application; one with authentication, access control (with individual ownership of records and group ownership).

Also would be good to compare the security of different frameworks in this way. See which frameworks leave security gaps. Would be good to set up a benchmark where an AI agent is made to impersonate different users (with different roles) and test access to all possible data types and views in an exhaustive way.

A while ago, I showed Claude a relatively complex application I had built with my framework https://saasufy.com/ and asked it to estimate how many lines of code and how long would be required to build the same application with a stack like React + Node.js + Postgres and it estimated that it would require 4 to 5 times the amount of code and amount of time so I'm keen to actually test it out with a benchmark like this.

I was not surprised that HTMX lead in a lot of the benchmarks. I suspect it would do even better with a more complex application. IMO, its design is closer in spirit to where the industry needs to go. There are a lot of maintainability and security gains to be had by standardizing frontend and backend interactions; though I believe there are other way to do this which can yield even more gains (though obviously with some tradeoffs depending on the use case).