There are just lots of filters. For English speaking countries at least, TOEFL scores ultimately determine who's allowed to study abroad (and where) or able to find employment elsewhere, so it could be argued that it's partially selective based on a person's financial background (and their ability to get personal training or get into good high schools).

Those with good conversational skills aren't necessarily those who score well on standardized tests, and vice versa. Moreover, to get a work visa, those who can speak a foreign language well still usually need to have another marketable skill, particularly in places where the law requires a business to prove they've made effort to hire domestically first - making things difficult for people who've majored in a foreign language or any field where there's lots of competition.

Most people I know hope to either get their TOEFL scores high enough to transfer to a foreign school, or get into a job that will let them transfer out of the country.