Hacker News Reader

Comment by solatic | Hacker News Reader

Why are blog posts like these written as if they are the One True Way?

It's a question of state management. Sometimes, it's easier for the client to manage state. Sometimes, it's easier for the server to manage state. Sometimes, nobody really cares about holes in the state. Sometimes you need at-least-once delivery, maybe-once delivery, exactly-once delivery.

People can design communication architectures for the problems they actually have as long as they appreciate the trade-offs.

OP proposes that it's no big deal for long-polling to hold an open connection. At small scale, sure. At large scale, forget about it. It's a trade-off for not losing webhooks. But what if we don't care about dropped webhooks? If they're an optimization instead of a source of record? I guess OP is OK with that?

View on HN

Perhaps could be written better, but promulgating the idea of "you should consider /events instead of webhooks" is good. Most teams just do webhooks without considering /events.

IMO webhooks are overrated. They have trust issues (requiring signatures or shared secrets), they make code hard to test (because you have to mock the other end), and they bring up versioning issues (client can't advertise the version expected). When I consume webhooks I usually poll anyway and simply use the webhook notification that "Thing ID #123456 changed, poll it". Most of the time I'd be happier polling /events on a minutewise basis.

The bit in the article about long polling seems pointless. If you're accepting webhooks, you're accepting someone else's schedule and it's probably not that urgent. Everyone has a cron-equivalent, it's easy to poll statelessly.

And polling has its own issues. You have no real control over how often clients will poll, so you need some kind of caching layer or some kind of anti-abuse mechanism that keeps track of API keys and returns 429 when a client polls too frequently (but still permit polling from clients that poll at a more respectful, slower rate). That caching or anti-abuse layer has its own engineering cost and represents its own trade-offs.

You have no control over how often clients will poll even if you have webhooks. But it's easy enough to cache /events in cheap ephemeral storage, possibly even at the http level. Use Cloudflare if you want cheap. There shouldn't be much of an engineering cost here.

Except that webhooks are often client-private. Consider, for example, GitHub repository webhooks, particularly for private repositories. Storing /events in a global CDN cache is a privacy nightmare. The GitHub API has strict usage quotas and I'm sure that the engineering effort for doing that at GitHub's scale is non-trivial.

So you make the API /clients/CLIENTID/events. What's the issue?

Webhooks have engineering issues all their own, including job queues and failure notifications. Stashing events in a table and truncating it every now and then is relatively straightforward.

> Why are blog posts like these written as if they are the One True Way?

Because out of all the ways the OP knows this is the best for their use case and they are none the wiser. Happens all the time.

Because of how titles and attention spans work.

I've wondered this myself for a long time, before realising that titles like these get most of the attention, enough to rise to your feed.

If this was indeed 'Consider using /events', chances are you'll never see it.

Reply on news.ycombinator.com