In Spain at the period, there were no famines because people kept emigrating to the colonies. Ireland was itself a colony. That's the difference. In Eastern Europe where countries didn't have colonies, famines were a norm.
Irish one is seen as something special because it happened in the West, and because overpopulation there built up for a considerable time being allowed by potatoes farming that for the time being, provided plenty of food allowing population to build up. Then it backfired.
As for local populations pre-existing in the colonies, sure they almost all died out. To a much larger proportion than the Irish, and sometimes, went entirely extinct. That is the normal part of absorbing new lands. It's just that Ireland was Christian almost since Christianity became a thing, and was never "discovered", that made it special. But we shouldn't pretend like it wasn't normal or in any way exceptional overall. Genocide is a natural way in which nations interact.