Is this really "on" the RPi though? They mention inference on ARM Mali GPU (which is cool! But I don't think the RPis use Mali right?) but it wasn't clear to me if this was just using the RPi to take a picture and send it to an AWS server to do the actual inference, then return the results?

The last time I fiddled with inference on a RPi was several years ago when it took a couple seconds even for inference of a basic ImageNet CNN reading low-res images from the RPi camera (can't remember specifics sorry - I think it was on a RPi 2 or 3). FPS was something like 0.5-0.2FPS which made "real time" usage kinda difficult. I don't know what performance would be like now on a RPi4, but GPU accelerated inference on the Broadcom GPU would be nice.