Could it be that change is painful? When people settle down, they want things to mostly stay the way it is until they die. A city in growth or decline interferes with that.
The opposition to mass-immigration (the country is growing at about 1.6% annually, with the same annual level of immigration as the UK, a country with 3x the population) is completely justified and it is not in the long-term interests of the country or of individual Australians.
Its not only the formal mechanics - the whole system is rorted, with a lot of fraud. A Chinese businessman bribing federal politicians for citizenship is a current news item.
Finally Australia makes its way in the world by mining and selling off its fixed endowment of minerals. The rest of the economy is not particularly competitive, mostly just domestic services and real estate. More people means a smaller slice of natural resources per person. This is not theoretical - wages are flat and cost of living pressures are high in the country. Public healthcare, education and transportation are all overcrowded.