Hacker News Reader

Comment by sa46 | Hacker News Reader

The API improvement proposals (AIP, yes it's a terrible acronym) are a rich source of API design wisdom. It assumes protobufs but the majority of proposals would work for any API.

The relevant one is https://google.aip.dev/158, which recommends cursor-based pagination and covers different pagination use-cases, including:

- support for different sort orders - mentioned in article as weakness of keyset pagination

- support for skipping results - mentioned as a weakness of cursor-based pagination

- why to use opaque page tokens

View on HN

This is a good reference, but IMO page tokens need to expire, even if you aren't storing any data for them. Expiring tokens make it possible to change pagination formats over time (you only need to wait the expiry time while supporting both formats), eg to improve efficiency.

Also I'll add that if you support a pagination API that accepts args (e.g. to filter on different dimensions) you should include a hash of the args in the opaque next page token, and fail the call if the args change. This prevents some nonsense scenarios where you started paging by scanning index A but the user switched filters such that you'd instead scan index B, but you don't have the key for that column.

Your second point is noted in the https://google.aip.dev/158 guidance.

> Request messages for collections should define a string page_token field, allowing users to advance to the next page in the collection.

> * If the user changes the page_size in a request for subsequent pages, the service must honor the new page size.

> * The user is expected to keep all other arguments to the RPC the same; if any arguments are different, the API should send an INVALID_ARGUMENT error.

Expiring tokens has a section in that link:

> Many APIs store page tokens in a database internally. In this situation, APIs may expire page tokens a reasonable time after they have been sent, in order not to needlessly store large amounts of data that is unlikely to be used. It is not necessary to document this behavior.

Hey! Nice! I do think clients shouldn't store the cursor. This technique is neat, I will add it as an option to the Article (I’m the author).

Reply on news.ycombinator.com