There was never a point in American history where the Irish were not considered white. It's just that racism is amazingly not limited to whites vs non-whites.
Also pervasive is this idea that Irish were also enslaved in the Americas when they were not. Indentured servitude yes but enslavement, no. [1]
[0] https://www.cambridge.org/core/journals/victorian-literature...