Not necessarily. The OED actually gives the hyphenated form for the colloquiallism, and cites William James: "I am going to settle down to the composition of another small book, more original and ground-breaking than anything I have yet put forth". (The more recent citation for the colloquialism is unhyphenated.)

The OED doesn't comment on the act (ritual) of breaking ground, but Wikipedia does, and it uses the un-hyphenated form. https://en.wikipedia.org/wiki/Groundbreaking . (Wikipedia is not, of course, the most reliable possible source, but it does indicate that the un-hyphenated form is in wide use for the literal act).

My strong suspicion is that the colloquialism arose as a metaphorical back-formation of a participle from the gerund + object compound used for, you know, ritually inaugurating (inaugerating?) a building project.

Edit: removed ill-considered emoticon.