Sure and many feudal warlords grew the same way but at some point the Castle decayed away and became a tourist destination and what was left behind was the community and governance.

Once organized crime gets really big, it stops being crime and persists on organization alone.