You can now use WebGPU in Cloudflare Workers | Hacker News Reader

You can now use WebGPU in Cloudflare Workers

blog.cloudflare.com

136 pointstodsacerdoti29 comments

Securing multiple tenants on a GPU is a really, really hard problem. As https://gpuweb.github.io/gpuweb/#malicious-use notes, there's a tremendous surface for undefined behavior.

Now, it does appear that the security model https://developers.cloudflare.com/workers/learning/security-... ensures that any nodes to which the GPU is connected aren't running the full JS Workers code, just the WebGPU functionality. So secrets that are maintained on the JS side aren't likely vulnerable to Cloudbleed-esque attacks: https://en.wikipedia.org/wiki/Cloudbleed .

But I'd be incredibly wary in sending sensitive information to the WebGPU calls themselves - assume any other WebGPU-using tenant can arbitrarily access data sent to any GPU and modify your outputs.

The most surprising aspect of this seems to be that Cloudflare would think of deploying it, given the apparent cost of GPUs. It almost suggests they are using GPUs for their own workloads already.

Honestly the clear case for this will be in online video distribution; both the video processing itself (encoding, compositing, enforcing privacy masks) and using inference to decide how to route it (including choosing bandwidth, configuring the internal distribution graph).

Help: I can wrap my head around WebGPU for in browser & on device 3D, video, and ML like face or hand tracking, etc.

I struggle to imagine the use cases where WebGPU runs on the edge / servers & makes more sense than other ways to access GPUs. Anyone have a good layman’s example use case?

They have pretty cool examples here: https://ai.cloudflare.com/?gpu

Text Generation Speech Recognition Image Classification Text Classification Text Embedding Translation

The demos in this page are using Workers AI, not WebGPU APIs. However, yes, you can perfectly run an ML inference runtime that takes advantage of GPUs using just Wasm and WebGPU APIs. That's exactly what we have here:

https://github.com/cloudflare/workers-wonnx

1. You want or have a “serverless” function. 2. You want to add some sort of ML work to that serverless function. 3. The serverless function runs on a maimed browser environment (v8)

Specific examples… you have an API that takes a picture returns if there’s a hotdog in the picture. Or an API to generate images from customer input. Etc.

Why is this better or why does it make more sense? If you’re already using the above environment for your functions it makes sense to keep using it. In this way, you don’t have to manager or provision GPUs at all.

Yeah, but why edge for that? It makes sense for something like Stadia (rip) so i guess that's one example, but is the user experience for hotdog-or-not really improved that much by the extra 50 Ms or whatever of latency that edge gives you over closest core?

Yeaaaaa I don't think most people really need that extra 50ms. I think CloudFlare is trying to build out a developer product, and needs to build all features to catch the long-tail of work. If you're using CF and suddenly you need a trendy GPU, its better for CF to sell it to you before you open a AWS account.

Serverless single shot inference

That’s a fair question and I think it can be asked more generally: “I struggle to imagine the use cases where X runs on the edge / servers & makes more sense than other ways to access X.” You are balancing the cost and speed of X in different locations, the cost and speed of accessing X over the network, and whether you have existing infrastructure deployed there already.

Cloudflare Workers and other edge services tend to make sense if you want more power than the client (e.g. phone) can provide, low latency to the service, and easy-to-scale deployments.

Hooray. Now I can run WebGPU ML on a cloudflare worker using Burn framework in Rust (https://github.com/burn-rs/burn). Burn has WebGPU backend in Rust.

We'd love to see it!

This is pretty amazing. This allows you to do all kinds of stuff around video, manipulation stands out to me, though they mention GPGPU as a use case without further elaboration, which makes me wonder if they might be coming out with something soon where you can use WebGPU APIs for GPGPU scenarios.

If by GPGPU you mean compute shaders, they do support that today according to the announcement.

> we fully support the part of the API that the compute shaders and the compute pipeline require, but we are not yet focused on fragment or vertex shaders used in rendering pipelines.

You might be interested in this https://github.com/webonnx/wonnx

That could be very interesting in workers!

https://github.com/cloudflare/workers-wonnx

I couldn't find out the VRAM limit, but I assume for video manipulation one would need quite a bit. Also, it looks like the workers themselves only have access to 128MB of RAM unless this has changed?

https://developers.cloudflare.com/workers/platform/limits/#m...

I’ve been working on and towards similar for nearly six years now. CloudFlare just eliminated so much stack complexity for me in one swoop and cleared a considerable amount of my todo list. Good work y’all!

Could this be used to run playwright.dev with GPU acceleration?

We do support Puppeteer https://developers.cloudflare.com/browser-rendering/platform...

Dawn supports OpenGL, but that was left out of the blog post.

Considering how CloudFlare was just implicated in destroying secure email at Defcon [1], the addition of GPUs to that active assault is terrifying.

1. https://www.youtube.com/watch?v=NwnT15q_PS8

I believe the secure email issue linked is exploitable by any MailChannels customer, not just Cloudflare Workers.

All Workers API calls to MailChannels must send email from a domain that has a Domain Lockdown record authorizing the Worker by its CF-Worker header. Docs are here: https://support.mailchannels.com/hc/en-us/articles/456589835...

Since you cannot forge the CF-Worker header in API requests (this header is added by Cloudflare's back end), it is not possible to send email for any domain that hasn't been locked down to your specific Worker.

The CloudFlare worker is just being used to make a POST request to MailChannels. How is this a CloudFlare problem?

It was Mailchannels, and Cloudflare was only one of the many companies that they fooled into doing an integration. As far as the CF Workers <> Mailchannels integration, the security issues are pretty much fixed now, but I still wouldn't trust Mailchannel

Watch to the end of the talk I posted. It's only just begun.

Appreciate the skepticism; it keeps us on our toes. Let's cut to the chase:

Domain Lockdown: We added this to our Cloudflare Workers integration. It mandates a DNS TXT record to authorize a Worker to send emails from a specific domain. You can't forge the CF-Worker header, so impersonation is off the table.

Pre-Lockdown Vulnerability: Yes, we were more exposed before. Thanks to the researchers who pointed it out, we've patched this up with Domain Lockdown.

SMTP Relay & Web Hosting: Domain Lockdown isn’t mandatory yet in the rest of our service for the web hosting industry. But we’re developing updates for our cPanel WHM plugin and other integrations to make this scalable for millions of domains.

Scale & Standardization: We service a broad range of configurations. Rolling out universal changes takes time. We're also working with industry groups like M3AAWG to push for improvements to DMARC and other standards to help everyone be more secure.

Tech docs for the curious: https://support.mailchannels.com/hc/en-us/articles/456589835...

Appreciate all the questions and criticism here.

Reply on news.ycombinator.com