The basic times tables are a big sticking point in secondary mathematics. Because polynomial work depends heavily on factorization, students who cannot readily see that 56 = 7 * 8, and then apply their elementary operations skills to say 7 * 8 = (27) (8/2) = (47) (8/4) will struggle more than those that can.

You can develop skills to compensate (e.g., starting with small primes and working up instead) but it's still an extra step in the process.

This does not necessarily affect a student's overall understanding of polynomial functions, but can hang over their head in the actual work with them.