pradn

Yes, the field is littered with imperfection.

One thing though - if you do take the time to learn the original "perfect" versions of these things, it helps you become a much better system designer. I'm constantly worried about API design because it has such large and hard-to-change consequences.

On the other hand, we as an industry have also succeeded quite a bit! So many of our abstractions work really well.

In the simple (albeit niche) case, a UI could populate a list of buttons based on the URIs/verbs that the REST API returns. So the UI would be totally dynamic based on the backend - and so, work pretty generically across REST APIs.

But for a client, UI or otherwise, to make use of a dynamic set of URIs/verbs would require it to either look for a specific keyword (hard coding the intents it can satisfy) or be able to semantically understand the API (which is hard, requires a human).

Oddly, all this stuff is full circle with the AI stuff. The MCP protocol is designed to give AIs text-based descriptions of APIs, so they can reason about how to use them.

> REST isn’t about exposing your internal object model over HTTP — it’s about building distributed systems that behave like the web.

I think I finally understand what Fielding is getting at. His REST principles boil down to allowing dynamic discovery of verbs for entities that are typed only by their media types. There's a level of indirection to allow for dynamic discovery. And there's a level of abstraction in saying entities are generic media objects. These two conceptual leaps allow the REST API to be used in a more dynamic, generic way - with benefits at the API level that the other levels of the web stack has ("client decoupling, evolvability, dynamic interaction").

You have to be careful to make a distinction being in the EU and using the Euro as your currency. You can benefit from the EU common market, with its uniform rules/standards, easy capital flows, subsidies, and industrial policy. All without using the Euro as your currency, and being subject to the monetary policy of the ECB.

Being in the EU without using the Euro has been pretty good for Poland.

I think you're right. APIs have a lot of aspects to them, so describing them is hard. API users need to know typical latency bounds, which error codes may be retried, whether an action is atomic or idempotent. HATEOAS gets you none of these things.

So fully implementing a perfect version of REST is usually not necessary for most types of problems users actually encounter.

What REST has given us is an industry-wide lingua franca. At the basic level, it's a basic understanding of how to map nouns/verbs to HTTP verbs and URLs. Users get to use the basic HTTP response codes. There's still a ton of design and subtlety to all this. Do you really get to do things that are technically allowed, but might break at a typical load balancer (returning bodies with certain error codes)? Is your returning 500 retriable in all cases, with what preferred backoff behavior?

"... between 1988 and 2004, hardware got 1600 times faster, and LP solvers got 3300 times faster, allowing for a cumulative speed-up factor higher than 5 × 106, and that was already 20 years ago!"

"The authors observed a speedup of 1000 between [the commercial MILP solvers of] 2001 and 2020 (50 due to algorithms, 20 due to faster computers)."

I wonder if we can collect these speedup factors across computing subfields, decomposed by the contribution of algorithmic improvements, and faster computers.

In compilers, there's "Proebsting's Law": compiler advances double computing power every 18 years.

It's easier to keep doing something incremental than to fundamentally change things.

One avenue these phone OS, and any consumer OS, can pursue is making it easy to string together app sub-steps. An app can be "cracked open" into sub-step a) either by developers themselves b) or by a backend process at app submission time. The backend process could look at the screens, and see what the user journeys are - this is within reach for current LLMs. An "sub-step" here is like taking a flights app and turning it into different types of search functions - search by date, location, points etc. So an app becomes a bunch of interfaces.

Once you have these "sub-steps" a local LLM can string them together, because it can understand their inputs, outputs, and behaviors.

Actually executing the sub-steps would require the OS to execute the app in the background and run the sub-steps for the user. This would be akin to what browser agents do right now

So this is a way of semantically extracting the "verbs" in existing apps.

With a library of these "sub-steps" in apps, combined with similar ones extracted from the internet - you could chain together the web and native worlds, in the service of the user.

It's easier on phone OSs because phone apps are usually already logged in. It's realistic for iOS to just do a bunch of stuff for you in the background. You really can probably get finance info from all your finance apps, for example.

I'm not saying this is necessarily the answer - but re-thinking the OS in this sort of what is what would be an actually ambitious thing for Apple or Google to do. All these small tweaks are opiates.

There's two problems being discussed in this article and thread:

1) Combining OLTP and OLAP databases into one system

2) Using an open data format to be able to read/write from many system (OLTP/PostGres, analytics engine/Spark)

> I'd argue the bigger value is keeping the data in one storage place and bringing the compute to it.

Yes, I agree with you. This observation is the idea behind #2, and why Iceberg has so much momentum now.

Their product looks promising. It looks like the PostGres schema and writes have to be "Iceberg-aware": special work to get around the fact that a small write results in a new, small Parquet file. That's not the end of the world - but perhaps ideally, you wouldn't be aware of Iceberg much at all when using PostGres. That might be a dream though.

Fully using PostGres without awareness of Iceberg would require full decoupling, and a translation layer in between (Debezium, etc). That comes with its own problems.

So perhaps some intimacy between the PostGres and Iceberg schemas is a good thing - especially to support transparent schema evolution.

DuckLake and CrunchyBridge both support SQL queries on the backing Iceberg tables. That's a good option. But a big part of the value of Iceberg comes in being able to read using Spark, Flink, etc.

The small writes problem that Iceberg has is totally silly. They spend so much effort requiring a tree of metadata files, but you still need an ACID DB to manage the pointer to the latest tree. At that point, why not just move all that metadata to the DB itself? It’s not sooo massive in scale.

The current Iceberg architecture requires table reads to do so many small reads, of the files in the metadata tree.

The brand new DuckLake post makes all this clear.

https://duckdb.org/2025/05/27/ducklake.html

Still Iceberg will probably do just fine because every data warehousing vendor is adding support for it. Worse is better.

On the data warehousing side, I think the story looks like this:

1) Cloud data warehouses like Redshift, Snowflake, and BigQuery proved to be quite good at handling very large datasets (petabytes) with very fast querying.

2) Customers of these proprietary solutions didn't want to be locked in. So many are drifting toward Iceberg tables on top of Parquet (columnar) data files.

Another "hidden" motive here is that Cloud object stores give you regional (multi-zonal) redundancy without having to pay extra inter-zonal fees. An OLTP database would likely have to pay this cost, as it likely won't be based purely on object stores - it'll need a fast durable medium (disk), if at least for the WAL or the hot pages. So here we see the topology of Cloud object stores being another reason forcing the split between OLTP and OLAP.

But how does this new world of open OLTP/OLAP technologies look like? Pretty complicated.

1) You'd probably run PostGres as your OLTP DB, as it's the default these days and scales quite well.

2) You'd set up an Iceberg/Parquet system for OLAP, probably on Cloud object stores.

3) Now you need to stream the changes from PostGres to Iceberg/Parquet. The canonical OSS way to do this is to set up a Kafka cluster with Kafka Connect. You use the Debezium CDC connector for Postgres to pull deltas, then write to Iceberg/Parquet using the Iceberg sink connector. This incurs extra compute, memory, network, and disk.

There's so many moving parts here. The ideal is likely a direct Postgres->Iceberg write flow built-into PostGres. The pg_mooncake this company is offering also adds DuckDB-based querying, but that's likely not necessary if you plan to use Iceberg-compatible querying engines anyway.

Ideally, you have one plugin for purely streaming PostGres writes to Iceberg with some defined lag. That would cut out the third bullet above.

Isn't it basically not possible for the input data set list to be listed? It's an open secret all these labs are using immense amounts of copyrighted material.

There's a few efforts at full open data / open weight / open code models, but none of them have gotten to leading-edge performance.

Wait - there's system webviews? On Mac, Windows, and Linux?

Edit: It looks like Tauri uses the following platform webview features.

https://github.com/tauri-apps/wry?tab=readme-ov-file#platfor...

Every company gets a ton of hate on Hacker News. Don't let it bother you too much. But the specific concerns may be a directional signal.

I'm hoping there will always be a good LLM option, for the following reasons:

1) The Pareto frontier of open LLMs will keep expanding. The breakneck pace of open research/development, combined with techniques like distillation will keep the best open LLMs pretty good, if not the best.

2) The cost of inference will keep going down as software and hardware are optimized. At the extreme, we're lookin toward bit-quantized LLMs that run in RAM itself.

These two factors should mean a good open LLM alternative should always exist, one without ulterior motives. Now, will people be able to have the hardware to run it? Or will users just put up with ads to use the best LLM? The latter is likely, but you do have a choice.

It's premature in the sense that you need to care about it before you do profiling, which is the usual advice for perf improvements: "you don't know where the hotspots are, so write your code, profile it, and fix the slow bits".

> Typically, perf isn't a few bad decisions. It's a very large number of independently reasonable decisions that add up to a bad result. If the team loses that discipline for even one moment then it's very very difficult to fix. I wonder if my former team still exists or if they've all been reassigned elsewhere.

This is precisely where the adage "premature optimization is the root of all evil" falls apart. You really do need everyone to care about performance to an obsessive, unreasonable degree to keep the entire, massive system performant. Companies with good engineering leadership understand this. The thousand cuts can come from language, libraries, feature creep, and pure ignorance or carelessness.

I see a good number of these articles, each with their own typographic features. Is there a "gold standard" set of recommendations for making the most beautiful type, on the web?

This is such a meme on Hacker News. How much evidence is there really for this? There might be some, but saying its the primary reason CEOs hire consultants is an even stronger claim that needs justification.

SWE-years are actually used for two things: a) measuring ongoing cost and b) for calculating if an investment will pay off.

There's a few benefits over just using a dollar figure. It's a unit that natural ties time to value. SWE-years is a natural unit for thinking about the time of SWEs. It's also convertible to other units. The conversion factors to other units can change over time, without having to pay attention to the minutiae.

All these factors make it useful for both measuring ongoing cost and making investment tradeoffs.

There's definitely some nuance around how many resources to consume at the client vs the server. Video decoding and ML inference are probably at the extreme end of what you can make a client do.

On the whole, since clients are so constrained, it usually pays to be efficient there - make websites load quickly to increase revenue, require only weak hardware to get more game/app sales, etc. Clients are also untrusted, so there's so many things you can only do on the backend.

Yes, a SWE-year is a common unit of cost.

And there are internal calculators that tell you how much CPU, memory, network etc a SWE-year gets you. Same for other internal units, like the cost of a particular DB.

This allows you to make time/resource tradeoffs. Spending half an engineer’s year to save 0.5 SWE-y of CPU is not a great ROI. But if you get 10 SWE out of it, it’s probably a great idea.

I personally have used it to argue that we shouldn’t spend 2 weeks of engineering time to save a TB of DB disk space. The cost of the disk comes to less than a SWE-hour per year!

It's the responsibility of leadership to set the correct goals and metrics. If leadership doesn't value maintenance, those they lead won't either. You can't blame people for playing to the tune of those above them.

The list price of a house going up is a big part of it.

But I don't think people actually have an idea of how much more a house of the same price costs when interest rates go from 2.5% to 7%. For a million dollar house, that's an extra thousand or two a month.

There's a great paper that collects a long list of anecdotes about computational evolution.

"The surprising creativity of digital evolution: A collection of anecdotes from the evolutionary computation and artificial life research communities"

[1] https://direct.mit.edu/artl/article/26/2/274/93255

> Teaching financial concepts in a classroom is like teaching swimming with PowerPoint slides. You can explain the theory of the butterfly stroke all day, but at some point you need to get in the water. And with money, the water is always cold, deep, and full of currents that weren’t in the textbook.

It is totally true that these extra-mathematical factors have a big part to play how we spend and how we save. If you’re in a certain culture that expects a certain level of spending, it’s hard to take yourself out of that.

I still don’t see even adults with a good idea of basic financial things. How much does it cost to buy a house? How does a monthly payment change with interest rates? How do you become eligible for Social Security?

You really need to keep these several-dimensional functions in your head. At the least, a good grasp of algebra is required.

Ah, I see.

Well, you do have to worry about customers using old client libraries / SDKs, even if your whole backend has migrated to a new API.

Many customers don't like to upgrade unless they need to. It can be significant toil for them. So, you do see some tail traffic in the wild that comes from SDKs released years ago. For a service as big as S3, I bet they get traffic from SDKs ever longer than that.

There is some queasy feeling of fake-ness when auto-completing so much code. It feels like you're doing something wrong. But these are all based on my experience coding for half my life. AI-native devs will probably feel differently.

What sort of ingredients keep well in this process? How about mushrooms and potatoes?