Those people are generally for higher taxes on the rich. Tariff are a flat tax that dis-proportionally affects the poor.
The original income tax was sold as 1% on mid income and 2% on high income. At the time more than half the country was not going to pay any tax.
Southern states supported an income tax because they believed it would make it possible to collect revenue for indigent people. Exactly opposite of "taxes the rich".