Comment by xg15 | Hacker News Reader

Comment by xg15 | Hacker News Reader

Maybe the deeper message of this is: Pagination tokens are a shitty API and you should really offer something that gives the client more options.

Pagination tokens feel great if you're providing the API, because they let you retain maximum control over API usage and let you do all kinds of optimizations in the back-end. But they are really limiting for the clients and - as we've seen here - often fail to address even basic use cases, such as going back a page or even reliably reloading the same page you're on. (Not even starting about arbitrary seeking.)

This leads to dreaded infinite-scrolling or "click to load more" UIs, which overload my browser and will have me start again from the very beginning should I have to reload the page.

I think it's also very easy to miss for non-technical stakeholders just how constraining this API is. E.g. in the OP's story, I could easily imagine some nontechnical PM assigning tasks like this:

- back-end team: Implement data access API using pagination tokens (because the devs said this is the best way to do it)

- front-end team: Implement the data view UI, including "next", " previous" and "go to page" buttons, linkability and page numbers in the URL. (Because that's what the product owners ordered)

Without having some specific knowledge about how pagination tokens work, it's easy to miss that those requirements are contradictory - unless someone cheats, like in this case the front-end.

I see the failure mode you describe, but the point here is that if the API _does_ need to support that front-end use case, that should become an explicit part of the API contract.

I think "start off with an API that supports a minimal set of use cases without it becoming an API that de-facto supports other _accidentally_ and without the backend team ever having committed to that" is a valuable option to have.

It isn't really contradictory to have pages and pagination tokens? Just, at the low level, you want dumb API calls.

Now, I will assert you are probably better off with index like pages, anyway. Instead of numbered, jump to the "g" records. If you insist on page numbers, not too hard to somewhat precalculate what offset each page could be. Just more work, after all.

The advantage of tokenized apis is that you can prevent unbounded work on an API call. In particular, if you allow filters, it is easy to build what has to be a full table scan in one call. This keeps you from later having to build a way to cancel a call.

Offsets only make sense when you have ordered lists.

For large scale applications your really can't think about a global total order.

This is the reason why a list of pages should not be used with large amounts of data.

Also: It is a poor design choice. If you have users that see themselves go to page X, then they don't have the filtering they need.

Agreed. Is why I asserted numbered pages probably aren't best. Still, I know it is a popular ask.

> But they are really limiting for the clients and - as we've seen here - often fail to address even basic use cases, such as going back a page or even reliably reloading the same page you're on.

That's not really true. If this level of consistency is required, which it hardly is, then there's solutions with pagination tokens for both of these requirements.

What is the issue in previous page using pagination tokens? We use that. On all page lookups we provide 2 tokens: next and previous page.

> reliably reloading the same page you're on

How would you do that _without_ pagination tokens? Granted - it requires the data for that page is stable. Naturally page 3 of a list ordered by insertion order with a lot of additions will not stay stable.

The core is that the page metaphor only makes sense in a very limited cases - much more limited than would the regular UX designer thinks.

what's the solution space for these problems?

If the UI got you to a page that started with a search of some type, then you can look in your own experiences of how many times have you went to the second page on Amazon, Ebay, or Google search? Probably not often. So the answer is some combination of better search, related recommendations from the items on the remaining pages, and filtering.

I don't agree. As an example a shop and a dashboard are different things. Sometimes you are not looking for an item, but for a pattern, an outlier, or something else. And there are many use cases for a list of something.

Pagination is effectively a streaming API. Are all streaming APIs bad?

Pagination tokens are essentially a streaming API.

Pagination by LIMIT/OFFSET and "page numbers" is a false promise that the data never changes.

Reply on news.ycombinator.com