Intent. Do you intent to be a genuinely open community and is it set up around fostering that dynamic as a central aspect of development? It's hard to measure intent, but we can do so indirectly by looking at the project structure:

- Are there contributor guidelines?

- Do contributors have to sign a waiver before they can contribute any code?

- Is there a RFC process?

- Does the project actually respond meaningfully to that feedback, or does it simply get filed in the special complaints folder for corporate.

- Does the project encourage outside contributions in more than a cursory way?

- Does the CLA grant the company unilateral relicensing rights?

- Are governance documents public?

- Can the community vote on decisions?

- Does the community have a say in how it's moderated?

- Are community members invited to actually join the organization?

- Are architectural decisions made in open meetings?

- Is there a public ROADMAP and is the community invited to contribute to it or influence it?

- Can others build competing distributions without fear of retaliation?

- Can the project be forked if there is community disagreement about direction?

Those are just some signals off the top of my head. There's no bright line; the presence or absence of a few won't say anything one way or another. But if many of the answers to those questions are leaning toward the negative, then I don't think it can be open source.