People don't avoid open-access journals because they're low impact factor; many are quite high. People avoid them because there's a whopping big publication charge.
What we really need to think about is decreasing the number of journals. But nope, spammy new for-profit journals are launched every week.
Overall science funding is a separate issue, and so is the flawed metrics we use to determine who gets it.