Serverless Benchmark – An independent benchmark of serverless providers | Hacker News Reader

Serverless Benchmark – An independent benchmark of serverless providers

serverless-benchmark.com

40 pointsgitgud9 comments

When graphs are shown next to each other like this it is important to use the same scale across them. The Percentile graphs make IBM cloud functions look significantly better than the competition, despite being middle of the pack or worse.

Aside from that nit (which I see everywhere), this is awesome.

AWS' p99 is very impressive for hot functions. Pleasantly surprised by IBM's performance here. Google seems to perform the worst of the lot. It's also odd that Google's median cold start is faster than a hot function. Maybe the concurrency is triggering some sort of back-off?

PS: Adding cost per request as a comparison would be nice.

Interestingly I’ve gotten much better results with workers than I have with lambda

It doesn't say where the metrics are collected from, and whether they include the network latency from whatever point that is.

I did some testing a while back from servers globally (I used Postman w/ global monitoring) to test latency.

Pretty sure Postman uses some big cloud service (AWS or GCP) so you'd think Lambda would've had an advantage.

Interestingly, that didn't seem to be the case (workers had 3-4x less latency than Lambda).

That is interesting. I was trying to roughly say that Cloudflare stats might be artificially high/slow if measured from the center vs the "edge". Sounds like perhaps not?

Yeah I kinda understood what you were saying, but as I also surprisingly found out, it seems like they're fast in general regardless of where you're located.

I'll try to do more testing soon.

I made a url shortener with Cloudflare workers and their edge KV store. I did notice that it worked very fast in the US, but not so fast in Europe. The KV store seems to require a US->EU trip if it's not a "hot key".

Or basically, it seems "single master" with the center in the US. No such issue with the worker itself though...just the KV store. If I hardcoded the data, or used the regular page cache, it was fast everywhere.

Hi, PM of KV here. You should be seeing speeds improve in Europe within the next few weeks. Your intuition about the single master is currently correct, we're adding a second master in Europe soon, for exactly this reason.

Reply on news.ycombinator.com