Hacker News Reader

Comment by nine_k | Hacker News Reader

At one of my previous jobs, we designed a whole API to be slightly more contrived but requiring only one round-trip for all key data, to address the iffy internet connectivity most of our users had. The frontend also did a lot of background loading to hide the latency when scrolling.

It's really eye-opening to set up something like toxiproxy, configure bandwidth limitations, latency variability, and packet loss in it, and run your app, or your site, or your API endpoints over it. You notice all kinds of UI freezing, lack of placeholders, gratuitously large images, lack of / inadequate configuration of retries, etc.

View on HN

This reminded me of a feature request I dealt with at an employer, while working on backoffice software for a support team. The software loaded a list of all current customers on the main index page - this was fine in the early days, but as the company grew, it ended up taking nearly a whole minute before the page was responsive. This sucked.

So I was tasked with fixing the issue. Instead of loading the whole list, I established a paginated endpoint and a search endpoint. The page now loaded in less than a second, and searches of customer data loaded in a couple seconds. The users hated it.

Their previous way of handling the work was to just keep the index of all customers open in a browser tab all day, Ctrl+F the page for an instant result and open the link to the customer details in a new tab as needed. My upgrades made the index page load faster, but effectively made the users wait seconds every single time for a response that used to be instant at the cost of a one time per day long wait.

There's a few different lessons to take from this about intent and design, user feedback, etc. but the one that really applies here is that sometimes it's just more friendly to let the user have all the data they need and allow them to interact with it "offline".

There's no reason you can't have the cake and eat it too. If google can index the entire web and have search results and AI results for you in an instant, then you can give users instant customer search for a mid sized corp. A search bar that actually worked fast would have done the same as their Ctrl f workflow.

Of course if the system is a total mess then it might have been a lot of work, but what you describe is really more of a skill issue than a technical limitation.

I would not consider Google search, nor its AI overview an example of a good system.

It is an effective product, in so far as it generates revenue, but it is not an example I would use to describe a system that provides a good and useful experience to its end users - which was and generally is my goal when designing software.

I didn't say it was good, I said it was fast. Those two things are often correlated though, the app described above certainly doesn't fit my definition of good.

Computers can do unfathomable amounts of computation in the blink of an eye. If your app takes seconds or longer to do stuff it's probably because it's ass. Nearly all common operations should be measured in nano or milliseconds. If they're slow it's probably the dev's fault.

As your career goes on, you will likely eventually work on large production systems that are not idealized greenfield deployments - at this time, you will have to come to terms with the constraints of Network and I/O bounds, and your dreams of nano and milliseconds will be shattered.

In particular, I welcome you to experience the lovely word of corporate VPNs, whose maintainers and developers seem to have latency expectations that have not changed in 30 years.

I'm currently working on one of those wonderful greenfield projects where the monkeys who put it together made these mistakes I describe and everything is slow. Website isn't even in production yet and everything takes 5+ seconds to load for no reason. Because they just store the data the way they got it which is shit, instead of shaping it into something sensible that allows efficient queries.

And yes there's a corporate vpn and fire walls and vents and subnets and whatever else, but when I create a feature it doesn't take 5 seconds to load. It takes milliseconds to load. Because there's nothing in our environment that justifies this bullshit, the guys who built this app just suck. And I'm going to fix it like I always do.

I have also worked on large production systems where I've fixed lots of performance issues. Often I can see them just by reading the code, I'll find some code that looks ass then I'll run it and sure enough it's slow. So I fix it. It doesn't take a profiler or micro optimization, it just takes some basic understanding of what we're doing.

Some times slow code is justified, some times there's just a lot of processing to be done or a lot of network requests to send or something. But most of the time it's just devs who don't understand fundamentals.

I’m sure your super awesome programming skills that fix infrastructure and eliminate network I/O are very cool!

I'm not claiming to eliminate network transfer times. Sending a web request takes some time. But it doesn't take seconds, it takes anywhere from a few to a few hundred milliseconds, given that the request size is relatively small. If you're sending megabytes or gigabytes then obviously it takes longer.

So if you have a website and a backend, the most basic request will be fast. Make an endpoint that just returns hello world, a request to this endpoint will generally take a number of milliseconds. Maybe 10, maybe a few hundred, something like that assuming a decent connection. That's the round trip time, including overhead from protocols and auth etc.

Now you know the best case scenario - when you create a real endpoint that actually does something useful it will be slower. How much slower depends on what you are doing and how. That is what I am talking about. If you send 50 requests from your backend to various other services then obviously it's going to take a lot more time, so we want to avoid sending a lot of requests - especially in series, where the first request has to complete before the next one etc.

We also want to avoid doing a lot of heavy processing. You can do a large amount of processing with no discernible performance impact, but at some point and especially with bad algorithms this processing time can explode and make your system slow. For example I once looked at some code to generate a report that took 25 minutes to run. It was getting two lists of objects and combining them by iterating through the first one and linear searching the other for a match by id. The time complexity of this is O(n^2). I turned the other list into a dictionary which allows O(1) lookup, eliminating the O(n) linear search, making the overall time complexity O(n) and the report generated in less than 5 minutes. Still painfully slow by my standards, and I'm sure I could have optimized it further if I didn't have more important tasks to work on, but it's a pretty good improvement from a few minutes of work.

Another common culprit is just sending too much data. I've seen websites where a request returns huge Json documents of multiple megabytes, then uses a tiny fraction of the data. By changing the system so that the website only fetches the data it needs you can reduce the request time from seconds to milliseconds.

I hope this gives you a better idea of what I'm saying.

I don't really know how else to lead you to understand that when you query a data source you do not have control over, through a network you do not have control over, you cannot make the interface work faster than the response from that data source. No matter how good your programming skills are, your endpoint cannot return data faster than it retrieves it.

I completely understand that. You just haven't mentioned it until now.

You could also look for inefficiencies in the search. Maybe the query is inefficient, maybe you can make use of database functionality such as full-text search and/or indexes etc. If you don't have access to make those changes to the db, you can cache the data in your backend memory, your own DB, Redis or whatever you prefer, so your app can be nice and snappy regardless of how ass your dependency is.

VKontakte has a very clever but at the same time cursed solution to this — the `execute` API method. It takes JS-like code that runs server-side. You can make up to 25 API calls and transform the data any way you please before returning it to yourself, all for the cost of one network request. Working with every other API after that feels like a massive regression.

Years ago API’s and apps that used them were expected to do some work offline and on slow networks. Then, suddenly, everyone was expected to have stable Internet to do anything. The reason, I think, is the few apps that expected to be always online seemed better to users and easier to architect. So most architectures went that way.

The reason is developers are worse. A decade or two ago, developers were nerds who loved tinkering with computers. Today tech is big money so everyone wants in. Most devs don't care, it's just a paycheck. Until someone starts setting expectations and holding people responsible for their trash code, we'll continue to see code monkeys write broken codebases. Not to mention where is the mentorship? I've worked as a mechanic, I've worked in construction, I've worked in a store. In all the above, I was mentored. Not for very long in the store but still, the other two have 2 year apprenticeship programs.

The i got a degree and a dev job, apprenticeship? Nah dude here's a big legacy app for you, have fun. Mentorship? Okay I technically had a mentor. We had a lunch every couple months, talked about stuff a bit but nothing much. And I mean this is going to sound a bit pompous but I'm above average. I had mostly A's in university, I finished every single project alone and then helped others. I was a TA. I corrected the professors when they made mistakes. I wrote a lot of code in my free time. I can't imagine what it must be like for one of my peers who honestly didn't know Jack shit and still graduated somehow.

I'm working on an app right now, took over after two other guys worked on it for about a year. This app isn't even in prod yet and it's already legacy code. Complete mess, everything takes like 5 seconds to load, the frontend does a crapload of processing because the data is stored and transferred in entirely the wrong structure so basically they just send all the data and sort it out on the frontend.

I honestly think the fastest way to get this app working properly is to scrap the whole thing and start from scratch but we have a deadline in a couple months so I guess I'll see how it goes.

> Then, suddenly, everyone was expected to have stable Internet to do anything.

I had reasonable - not mega - internet in Australia at the time. But the first 5 years of Steam sucked hard. Get home from work, go to play [game of choice] only to find it will take hours to update.

You also get a surprisingly large amount of pushback for refactorings like this, because a lot of professional developers have been taught to value code quality and API purity over such frivolities as actual user experience.

I wish more developers bothered to test on flaky connections. Absolutely infuriating when an app can't keep up with your muscle memory...

Reply on news.ycombinator.com