Hacker News Reader

Comment by stevebmark | Hacker News Reader

I'd also point out that the GraphQL/Relay spec for pagination enforces cursor based pagination: https://relay.dev/graphql/connections.htm. It's also a really nice pagination spec overall, edges/node/pageInfo are well thought out and flexible.

In my professional experience, lots of devs will implement fetching APIs and not consider pagination, and then when the responses start growing (testing data, use, whatever), user experience sucks. Pagination should be a default API design for anything returning an unbounded list of items.

View on HN

How do you deal in GraphQL when you have a query hitting with multiple nodes which may return multiple cursors for the different nodes? Seems like a nightmare if you have to check and retry manually for each node, maybe there are libraries that take care of it transparently?

How is this different than hitting multiple REST endpoints for different resources with their own individual pagination? If the client needs to glue together paginated resources from two places, it's a hairy problem no matter what the protocol. A backend for frontend that does this under the hood is one solution to both, or a specialized endpoint that does it under the hood.

In the case of hitting multiple REST endpoints for different resources, on each query it's clear what pagination I'm using and combining the data is just appending it to the final list. In each case the partial results are flat and linear.

But in the case of GraphQL, like I may get an incomplete list, or I may get a complete list that will have some fields with missing data, or some combination of both, going down deep depending on the structure of the query. I don't see a clear way to request the missing data without duplicating part of what I already have, and I don't see a clear way to combine the results.

But I may be missing something about GraphQL or something about the tools that makes this simple.

For example something like this:

    {
      users(first: 10000, after: "cursor123") {
        edges {
            cursor
            node {
                id
                name
                friends(first: 10000, after: "cursor235") {
                    edges {
                        cursor
                        node {
                            id
                            name
                        }
                    }
                }
            }
        }
     }
    }

Oh. I think you’d handle it the same as rest, make a new individual field query for that friend by ID and ask for the pagination there. I think that would be manual work for graphql or rest, im not aware of anything that does that automatically. It’s also not a case I’ve personally ran into so not sure.

In the relay JS library you can avoid some duplication with fragments. Fragments are pieces of a graphql query that you can include in multiple queries. If you are using flow, the type system will also ensure that you can’t render a component unless you have used the correct fragment in your query.

That said, paginating multiple resources on the same page is still going to be complicated.

Agreed, so many times I've raised paging with an internal API provider only to be told they can retrofit it later if needed. Then you start to use production data and bam the one call doesn't work.

I disagree. I think pagination is a clumsy error prone kludge. That it performs better than the awful naive approach by additionally burdening the caller doesn’t remotely make it good. The correct approach is a buffered stream with proper flow control. Incidentally that’s what Transmission Control Protocol was invented for.

Probably the nicest way to expose this that fits contemporary sensibilities would be as a lazy sequence that does sensible prefetching and asks the server for more data as it’s iterated through.

> a lazy sequence that does sensible prefetching and asks the server for more data as it’s iterated through

Isn't this an accurate description of cursor-based pagination? You can have the server hold the cursor and associate a request for the next page based on the client ID (like in the PostgreSQL wire protocol), or you can make the flow stateless (at the network level) by handing the client a ticket for the next page.

Pagination implies many requests where as a streaming API would keep to a single request, no? No worry about multiple servers or juggling state.

It's still multiple requests on a streaming API if you're asking the client to notify you when it's ready for the next page. Most DB engines use TCP connection affinity to identify which cursor is in play, but the client still needs to send a packet asking for the next page. Both client and server need to keep the TCP connection open, which is state for both parties.

I see you mentioned gRPC streams in a sibling comment, which are a great alternative to REST-based pagination, but they are highly stateful! Streams are built into the protocol, so a generic gRPC client will handle most of the state juggling that would have been your responsibility with a REST client.

This is a bit inaccurate I think. You can make a RESTful api over gRPC. The protocol is irrelevant. An open connection is not a violation of statelessness. If anything, a streaming API where you can close the cursor upon completion and treat new requests as fully new is a lot less stateful.

We're also talking about APIs in general, not just http REST.

>It's still multiple requests on a streaming API

Perhaps, but the requests can be pipelined. You don't need to wait for the response to complete before asking for more.

The original article is talking about pagination over stateless (at the protocol level) HTTP requests. (Referring to APIs that are offered over stateless HTTP as "REST APIs" is technically incorrect but reflects common usage and is a convenient shorthand.)

gRPC is able to offer powerful streaming abstractions because it utilizes HTTP/2 streams, which are cursor based rather than strictly connection oriented. The state is still there; it's just a protocol-level abstraction rather than an application-level abstraction.

> Perhaps, but the requests can be pipelined. You don't need to wait for the response to complete before asking for more.

That sort of defeats the purpose of grpc flow control, doesn't it?

>That sort of defeats the purpose of grpc flow control, doesn't it?

Why do you say that? I don't know the full implementation details themselves but generally there's no reason you can't safely ask for even more after having asked and consumed some. If you have a buffer of M bytes and ask for M, then consume N, you could immediately ask for N more without waiting to receive all of the original M.

Although, I wasn't speaking about gRPC in that case though. I'm not sure how exactly gRPC achieves back pressure. I was only speaking abstractly about a pipelined call vs many pages. You seemed to claim that multiple requests was a requirement.

But fine, ignoring gRPC, you could possibly tune your stack such that normal http calls achieve back pressure from network stacks and packet loss. Http is built on top of TCP streams, after all. That doesn't make it inherently stateful does it?

Going all the way back, I still think its fair to say that a streaming response with backpressure can be stateless and without multiple requests. If you want to argue that multiple packets or TCP signals are needed then perhaps so, but I think that's a far cry from the many separate requests a paginated call requires and I dont think its accurate enough to conflate them.

> I don't know the full implementation details themselves but generally there's no reason you can't safely ask for even more after having asked and consumed some

I think we're saying the same thing but using different formulations. If you send an HTTP request for a list with 20 items, then get back a response with 10 items and a link for the next page, that is essentially the same as cosuming 20 items over a stream with flow control. The point of returning a cursor in the response and having the client send a new request for the next page is to support a stateful stream over a stateless protocol. In neither case are you waiting for the response to be complete before processing items, since your message indicates that "complete" here means the full result set has been sent to the client.

> But fine, ignoring gRPC, you could possibly tune your stack such that normal http calls achieve back pressure from network stacks and packet loss. Http is built on top of TCP streams, after all. That doesn't make it inherently stateful does it?

That's pretty much how streaming responses are implemented in TCP-based protocols (like the SQL query APIs exposed by Postgres or MySQL). TCP connections can be terminated for unrelated reasons, which is why you don't see this pattern very often in recent protocols. When a TCP connection is dropped and restarted due to, say, network congestion, you have to restart the paginated operation from the head of the list. H2 (and, vicariously, gRPC) streams are built to be more resilient to network noise.

But to answer your question, yes, that pattern is inherently stateful. Pagination has to be, since the server has to know what the client has seen in order to prepare the next batch of results. You can manage this state at the protocol level (with streams), or you can push it to the application level (with cursor tokens embedded in the messages exchanged). The streaming approach requires a stateful server and client, whereas the application-level approach only requires a stateful client.

I think my opinion on close enough also differs from yours (and that's ok!) but you're also forgetting that a streamed/connection based approach only involves a single client and server and state during the connection.

A paged response needs to sync state across many machines. As you have said, you need a clientID, a cursor reference, or a sticky session. That's not nothing.

I think they're inherently different approaches.

> A paged response needs to sync state across many machines. As you have said, you need a clientID, a cursor reference, or a sticky session. That's not nothing.

OK, I think this is what I was misunderstanding in your comments above. You can keep pagination state in a distributed store, but it's not necessary. The implementations I've seen and worked with have all embedded all the information needed to generate the next page of results in the cursor token itself. Kind of like with a JWT or TLS session ticket, there's no need to sync pagination state or use sticky sessions. That approach also provides some resilience in case the server that generated the previous page of results becomes unreachable (something that the client needs to handle by restarting pagination if pagination state is just held in memory by the server keeping the stream connection open).

>Probably the nicest way to expose this that fits contemporary sensibilities would be as a lazy sequence that does sensible prefetching and asks the server for more data as it’s iterated through.

gRPC's streaming API is a good example. It has backpressure built in and you just need to configure the buffer sizes. Clients just consume items off the stream and async writes will suspend as needed.

When dealing with the most common case (I think) for pagination, clients (web/native) requesting data, are you suggesting keep a long running websocket only to fetch data in response to new page requests? What benefit would that afford, given that the requests are pretty far apart in time?

Or are you focusing more on the backend machine to machine case? In that case, some kind of automatic prefetching, lazy loading/yield setup sounds pretty nice if that's abstracted away when you want to iterate over a large dataset from a separate resource server. It's not a pattern I've used before, mainly because it's not often that one server wants to know everything that another server has.

I find pagination really annoying on the client, but the majority of the time it does seem like the client doesn't actually need the whole thing so it's a good (and often very necessary) performance optimization. For endpoints where that isn't the case, it's (usually) not too hard to allow a sentinel value for page size that means infinity, so the client really can get it all in one swoop.

What are you saying that would be different in practice? What advantages would it have?

Not sure I agree with the parent but thinking it through, you have a single connection and a cleaner life cycle. Streaming writes and reads could mean back-pressure is possible. A client can consume items as needed as opposed to mapping usage to a page size.

There's no worry about wasting a full request on the last page with a single item. In fact there's no worry about round trips at all, it's just a matter of buffer back pressure that communicates the need for more or less.

Hmm, when you think about it, its a pretty good way to go.

That’s consistent with what I have in mind. I too think it’s pretty much best of both worlds. You get better efficiency than pagination with the convenience of an API that’s no different from fetching all the data.

I worked with an API like this once, which streamed NDJSON one line at a time. I loved the amount of control that it gave me on the client side!

Reply on news.ycombinator.com