Some contributors will be okay with that. But some won't be - and you're misleading them by telling them that it's open source when it's not. When something is called "open source", it's assumed to meet a certain criteria - the OSD - and can be used as a mental shortcut. When someone sees that your software is "open source", they'll probably believe you - but it's not. You're taking advantage of that.