Hacker News Reader

Comment by felixgallo | Hacker News Reader

there is no way any competently written back end system takes 5-7 seconds for any reason whatsoever. A middling back end system with median complexity is doing 10-100K requests per second per server and is near-perfectly horizontally scalable.

View on HN

> there is no way any competently written back end system takes 5-7 seconds for any reason whatsoever.

I'd argue that even a well written system that allows arbitrary date ranges and filtering of data can easily run past that limit. You just can always know ahead of time what some client might request and some strategies for making normal queries fast can make some queries slower. Its tradeoffs all the way down.

definitely not. If you don't already know ahead of time what a client might request, and just let them request anything arbitrarily without regard for whether that data is indexed or cached, then you are not competently writing a backend system.

I didn't realize that a search engine needs to be able to predict 100% of the phrases a user might search for in order to be competent

And yet somehow Google, Bing, etc. manage to return results in much less than 3 seconds.

But then we have just one Google and one Bing rather than thousands of them.

Yeah, for anyone reading this that doesn't know these are embarrassing numbers. We call it an outage if our p95 latency goes above 100ms for more than a minute. Even if you think this is aggressive 50x slower is not normal. There will be necessarily be tasks that take longer than that but don't make your user sit there with the page just spinning. Post back when the task is finished asynchronously. Especially on mobile where those kinds of delays are assumed to be issues with the network and users will just hit the x and refresh.

I cannot fathom how you arrived at that conclusion - there are so many systems that need to display the results of complex data queries where caching or precomputing the results does not make sense - waiting several seconds for results would be normal in those cases.

I've been building backend systems for over 35 years across many industries.

I will echo what the parent said: there’s no way your backend is taking seconds (fucking seconds) to return data unless there is something terribly wrong with your system.

Complex queries should be optimized. The only time I have run into query issues is when: I was lazy, and didn’t want to optimize the query/schema/data layout and structures; the system was built by someone of questionable sanity and skillset; or the generated data is legitimately compute intensive, requiring lots of math i.e. more ops and cycles, but at that point it’s a report and the user can wait for it to gen (but in this case it is a lot of math that hasn’t taken advantage of parrallization).

In the wild, I’ve mostly seen people having zero clue what they’re doing (this includes management, who are as culpable for not caring about perf).

ChatGPT and many AI tools take several seconds to return responses.

I’m not saying it should always or even most of the time be the case (quite the opposite - I’m with you and parent on that) but to say that there is no situation like that strikes me as pretty odd.

This was not always the case; and local models respond almost instantly.

In ChatGPT's case, I'm assuming they're throttling responses to not hemorrhage money.

> there is no way any competently written back end system takes 5-7 seconds for any reason whatsoever.

Sure there is--ad networks.

Remember when 100ms was considered fatal? Funny how that went right out the window when the big companies decided that shoveling more ads at users was more profitable than anything else.

The current state of web development is all to paper over the idiocies brought along by serving ads.

Those of us who do web development and don't have to serve ads can pretty much use anything and hit metrics under 100ms and can scale for a very, very long time.

10-100k RPS is throughput, not latency. There is a well-known latency-throughput tradeoff. For example, Erlang’s BEAM VM is optimized for low latency, while JVM is optimized for high throughput.

Not everyone has the luxury to work with competently written backend systems.

Not everyone has the luxury to work with competently written front end systems.

The entire Internet apparently did for years because frontend JS monoliths only appeared halfway though my career.

There was a time where a request taking even 500ms was considered to be slow, and then all of a sudden it just became normal to see API traces where requests would take over a second, or multiple seconds, and people would just shrug at it.

Even a backend roughly cobbled together in PHP4 with Varnish running on top of it would considerably outperform some of the stuff that is accepted as 'good enough' today.

> with Varnish running on top

Like it's a trivial task. If it is... the whole system wasn't that complicated and likely similar results could be achieved without external caching.

It's not 1 request. It's a request waterfall in almost all cases. See the remix homepage for a great demo and pitch on why remix is awesome

https://remix.run/

1. Most software is NOT competently written - just look how bad your average, day to day software runs.

2. Salesforce, JIRA, Microsoft Teams, Reddit, even Gmail recently takes up to a minute to get content and be responsive, just to name a few.

I'm with you on this!

Sure, I can allow users to run reports that take multiple seconds, but I'm generally not delivering that to the browser to be rendered.

And if I was, I'd crash the browser tab. Which is surprisingly easy to do...

I've seen this with pretty normal Rails apps that some folks like to praise for the "productivity"

If you serve data from a DC in California to the west coast of Australia it will take 10 seconds of just network transfer time. Sometimes data has a high degree of cardinality and cannot be cached. Some companies cannot afford to, or for other reasons cannot, have regional instances of their compute surfaces. This is just one common scenario, there are many others.

> A middling back end system with median complexity is doing 10-100K requests per second per server and is near-perfectly horizontally scalable.

You know perfectly your SLAs without knowing anything about the business, data, or the user access pattern? You don't even need a budget? Impressive!

> If you serve data from a DC in California to the west coast of Australia it will take 10 seconds of just network transfer time.

What? Is this a typo or do you think that’s true?

> If you serve data from a DC in California to the west coast of Australia it will take 10 seconds of just network transfer time

Where did you get that calculation? Looks way way off

You have no idea what you're talking about. Perth is about a third of a second away from Los Angeles. There is no data that cannot be cached owing to 'high cardinality'. There does not exist a company that is trying to serve Perth traffic from a US CA DC that cannot afford a server that can do 100k RPS.

As a fellow Australian here in South Australia, I can say with certainty that I've never had it that bad!

I can measure the difference between US, Singapore and Sydney based DCs, but it's milliseconds.

10 seconds is abnormal and would indicate to me that something isn't functioning as expected!

Reply on news.ycombinator.com