If you think that the development of a space based missile system will only be developed with "defense" in mind, I have a surplus of bridges that I'd like to unload at a reasonable price.
The threat of using it as a really sucky attack platform is not why there are objections to missile defense.
Imagine that instead of flying a drone over your enemy with a missile, you instead launch a steel rod from space directly into their vehicle. Same destruction, no drone to alert them, you can target anywhere on earth.
Of course, the keywords here are "promises" and "ostensible". It will never work well enough that a rational person would depend on it. (Note, in particular, that it can never be fully tested, short of actual nuclear war.) But can we assume our leaders will always be rational?