how much harder is container escaping compared to vm escaping? i understand that containers are not truly meant to be security boundaries but they are often thought of and even used as such.
The answer heavily depends on your configuration. Unprivileged with a spartan syscall filter and a security profile is very different than privileged with the GPU bindmounted in (the latter amounts to a chroot and a separate user account).
And yes, this will most likely be a mess.