Some contributors will be okay with that. But some won't be - and you're misleading them by telling them that it's open source when it's not. When something is called "open source", it's assumed to meet a certain criteria - the OSD - and can be used as a mental shortcut. When someone sees that your software is "open source", they'll probably believe you - but it's not. You're taking advantage of that.
I tried (apparently not successfuly) to write everywhere very clearly that the Commons Clause got applied and if somebody contributes to it he would normally have spend some time with the project and he should be aware of the license situation. Would never have expected that somebody would find out later that it is not the case and he would then be upset about it. But thanks, not sure that it would happen but still one more thing for me to consider (as you are right that it could happen).