I think there's a difference of opinion here about what it means for it to "work". The point I infer from the parent comment (which I happen to agree with) is that measuring success or failure in terms of the community is orthogonal to open source. If what you want is a successful community, then it makes sense to base your decision on whether to be open source or not on that, but failing to build a long-lasting community isn't a failure of open source itself but at using it as a tool to try to achieve a specific goal, and not everyone doing open source shares that goal.
As an aside, I think it's worth noting that most attempts to build long-lasting, vibrant communities for software fail, regardless of whether they're open source or not. Open source might not have a high success rate of community-building, but neither does proprietary software (or whatever middle ground you consider to be in the space between those terms, like "open core" or "source available"). If what you care about most is building a large community, the more effective choice probably depends on a lot of factors specific to what you're building and what else is going on at the time you're trying than something as simple as "always be open source" or "always be proprietary".