If you spend money in Canada, then you are taking stuff from Canadians. If you spend your money in the US, then you are taking stuff from Americans.
You might wonder what happens at the limit - why don't Canadians just spend all their money in the US and take all America's stuff (just a thought experiment)? Because currencies adjust. Canadians would need US Dollars to buy stuff in the US, and as more and more Canadians try to do that, the exchange rate would change to devalue the Canadian Dollar against the US Dollar, effectively making things more and more expensive for Canadians until they are forced to get their stuff elsewhere.