Hacker News Reader

Comment by ozgune | Hacker News Reader

(Ozgun from Ubicloud)

Looks like every GitHub Actions provider is on this thread. :) I also wanted to chime in with two thoughts.

First, we recently wrote about how we enabled arm64 VMs at Ubicloud. Any feedback on the technical details is more than welcome: https://news.ycombinator.com/item?id=40491377

Second, I feel runs-on's analysis is a bit unfair to us. Ubicloud's x64 / arm64 performance and queue times above are as good as any (we deprecated the AMD EPYC 7502 line). For example, RunsOn arm64 queue times are 31|42 seconds and Ubicloud queue times are 17|24 seconds.

But the analysis says the following, "Be aware that Ubicloud has low CPU speeds and high queue times, which means they won’t necessarily end up being cheaper than competitors." I fail to see this conclusion from the numbers above. What am I missing?

View on HN

The queue time for x64 appears to vary quite a bit still. But you are correct that it is much better than what it was a few weeks back, when the analysis was written (it was not unusual to see 60s+ spikes).

Also note that I specifically mentioned that "all third-parties are good on that front, expected [sic] Ubicloud (but that may change)."

Also agree that now that you removed the outdated CPUs (was still active end of May), the analysis should mention that you have OK CPUs by now. I will fix it.

For providers that are on this page, do not hesitate to write to me if you find the analysis outdated for your specific service.

[edit] page is now up to date with new analysis

Hey there, thank you for this update. It's your analysis, so you're free to mention Ubicloud as you'd like.

Still, you updated the above statement to, "Be aware that Ubicloud has a slightly lower CPU speed and somewhat variable queue times for x64 (but improving)."

Could you clarify what you meant by variable queue times?

According to the benchmarks, Ubicloud queue times for x64 were 18|44 secs at p50|p95 over the past month. I agree that our p95 number is higher. At the same time, RunsOn's AWS numbers are 31|36 secs. So I feel that driving a conclusion based on our p95 variance number for x64 is a bit unfair to Ubicloud.

Thank you for compiling this benchmark btw. Now, we'll follow it closely.

RunsOn is the only on-premise solution of the bunch, so it is single tenant, and as such it cannot easily have pools of 10s of machines on standby to start within 10s. So on queue time I’m comparing Ubicloud against the other SaaS providers, and anything higher than 20s is not especially great in that case. Plus you have high variance, which I personally consider an issue.

Once this improves I’ll be sure to update it.

Reply on news.ycombinator.com