I’m really stuck on a relatively minor point: why does Joe-E have to ban the finally keyword?
This shows up a subtle detail of sandboxing schemes that are often overlooked. The guarantees Java provides around safety are tightly scoped and often little more than saying the JVM itself won't crash. It's not that hard to arrange for a stack overflow to occur whilst some standard library code is running, which means execution can abort in nearly any place. If the code you're calling into isn't fully exception-safe, it means the libraries global variables (if any) can be left in a logically corrupted state, which might be exploitable.
If Java finally blocks had a filter clause, that could help, but finally is sometimes implicit as with try-with-resources.