You have to be careful with this line of thinking, because as you've stated it, this justifies concepts like Trump's tariffs based on trade deficits.
The reality is that Finland won't be any richer (nor poorer, of course) if it spent 1+ billion euro on some EU or even Finnish alternative to MS; nor if it were spent on consulting and/or a government IT office. Trade is not a problem in itself.
The real concern is that letting Microsoft be the only game in town for business software of this type lets them control prices - that just having some trustworthy alternatives would allow the same services to be obtained for, say, 500M euro instead.
The other concern is related to tech dependence - that the government's reliance on MS services allows the US to use a threat of denying access to MS products as a strong-arming tactic in any negotiations.
Finally, the concern that a US company - and, implicitly, US security services - has access to potentially highly confidential data is a national security risk, though the impact is hard to measure.