I've tried that as well, but you need a brotli module for nginx. No problem if you're using Docker, but I just used the mainline distribution of nginx on Ubuntu 16.
I'm curious if brotli's decompression is still fast on high compression levels. This is the case for gzip IIRC.