"It was just a joke bro" is rarely a joke. You say it was just a joke when it's so abhorrent that it gets messaged past the audience it was meant for. It certainly didn't sound like a joke in context, and it's something that large parts of his base believes because of all the demonization of higher education as a republican talking point for decades.

I wouldn't be surprised if it's a policy position that he doesn't actually believe but is willing to push because it polls well with his constituency.