I would need numbers to be able to agree with you that foreign investment isn't a driving factor in housing costs.
The numbers, at least historically, have always been hard to find, but the macro economic theory and indicators are generally quite boring and straight forward: employment stays high and household incomes improve, interest rates drop, lending and leveraging increases to household, supply is inelastic, house prices rise.
Everything lines up pretty well, so why look to blame foreigners (who, even at the most generous estimates on what data does exist, are still a minority).
The rba website would be my first stop in aus for aggregate finance stats available publicly. The Abs has some construction/hiding investment data, but there are some issues with definitions you'd need to take into account. And our neighbors across the pond, although I'm not a kiwi, I imagine will have very similar structures and data.
Additionally, such a simple model explains house prices quite well since the 1990s, across various markets with various levels of foreign investment, so we would need very good reasons to insert a "foreign investor" cause in there.
Many new condo developments in Vancouver and Toronto are offered for sale in Asian countries exclusively before Canadian citizens can even make an offer on them. If you drive around Toronto there are real estate signs openly advertising areas that don't have a 15% foreign buyer tax.
The RCMP recently busted an elaborate Chinese money laundering ring in Vancouver that was processing $500 million/year in foreign cash alone. Much of it was being used to purchase real estate in Canada. The last article I read mentioned Canadas laws are so lax the authorities couldn't even seize the real estate directly involved in the scam because it was owned by Chinese citizens through shell corps.
Also, Auckland has had probably more immigration than Australia, at least for the last few years.
That said, dropping prices by 10-20% would mean absolutely nothing for an average labourer.
Yet I've heard from one realtor that in some suburbs of Great Vancouver 15% of properties stay empty - pure investment.
Vancouver is not the city to live in anymore. Problem is that it is not the city to work in too. Too long commutes - 2 hours to spend on road. That means work day is not 8 hours but effectively 10. That means people asking and company offering higher salaries.
That housing situation hurts all other businesses except of real estate quite a lot.
But most foreign buyers in these markets are buying to profit, and rent out the properties, and so only really contribute to balancing rental and sale prices.
Sure, a few are left empty by owners who just don't care, but there's very little indication this number is high. Especially as in most countries there are rental management companies that makes it trivial to rent out a property even if you're an absentee landlord.
In Vancouver, the problem is so bad that local shops are going out of business for popular speculative properties because their just isn't enough residents to use them, even though prices are sky high and the developments are all bought out.
If you’re well capitalized it is often cheaper and less hassle to leave them empty. Renting is a pain in the ass and makes unloading the property harder.
Example: I want to hire 10 developers. There are no places to live with price tags less than 1.5 mln in 1 hour commute radii from here. What would be my options? To pay them enough salary so they can afford mortgages? Cheaper to move the whole business to Eastern Europe / Russia then.
The house market is split between people buying and renting. When people buy as an investment, it does drive up purchase prices, but unless they leave it empty rather than rent out, it drives down rental prices by increasing supply. Rental properties also tends to provide more density, as they are more likely to be occupied by sharers, and so increasing the rental supply is likely to soak up more demand than the reduction in purchase opportunities. Eventually the prices reach some degree of equilibrium, because bidding up the purchase prices much quickly becomes unprofitable as rental prices are pushed down.
Ultimately what is driving the cost up is mainly down to more people with enough money who wants to live in a given location, not investors, as if there wasn't demand to live there it'd quickly prove a bad investment.