I have a feeling that any piece of closed-source software that becomes important enough will inevitably be reverse engineered by highly motivated people. This is doubly so for Vanced: The original team took pains to not release the source, but it merely took a few weeks for someone else to reverse engineer the reverse engineered codebase.
I've worked on preserving a few games with decompilers long after the original creators have moved on. Anecdotally, the death of Flash all but killed off a couple of assorted communities that didn't have enough motivation to hack the binaries themselves.
My guess is the only software that can perpetually maintain a community is software where the source code is available to be modified.