Hacker News Reader

Comment by supernova87a | Hacker News Reader

There was just a story today in the news about the University of Chicago completely doing away with requiring SAT/ACT for admissions. It seems like we're bending over backwards to toss out any kind of test or criterion that could be perceived as discriminatory. With one stated goal to make the student body mirror the general population.

But that is a flawed premise. Is it even the role of higher education to remedy the cards that have already been dealt, and even if so what evidence is there that it actually does what people say it does?

View on HN

To say nothing of the logical contortions required to argue that modern standardized tests discriminates based on ancestry or skin color.

They clearly do. The most explicit example is that ancestry correlates with whether you are a native speaker of the dialect the test was written in.

>The most explicit example is that ancestry correlates with whether you are a native speaker of the dialect the test was written in.

What kind of absurd "argument" is this? If you're testing for a university in America, the expectation is that not only will the test be in English, but the vast bulk of your university courses. And Asian Americans, despite a large number coming from families of non-English speakers, still do disproportionately well. But if the local language is something you struggle with, that local university isn't for you, unless we're also going to argue that degree awarding requires lower standards.

In which case, just throw university in the dump because it's worthless.

If we're at a point where having a majority language and doing important business in that language is bordering on discrimination, this society is lost.

So you disagree with the assertion that native speakers of a specific dialect score higher than non-native speakers of that dialect--all else being equal?

Or did you just decide you'd rather argue with something I didn't write?

Even if it is case that native speakers of a specific dialect score higher, it's not clear that the result is due to language barriers, and could be a correlation with socio-economic groups. That is the point of bringing up non-native speakers doing well, as the test is not even their native language, which is certainly a greater disadvantage than a different dialect.

So you don't think that there is an advantage gained by being a native speaker of a dialect a test is written in?

The academic consensus is that there is.

The clearest example is that vocabulary is a large part of most standardized tests and native speakers have larger vocabularies than proficient L2 speakers.

How about the math part? Besides, if I was on a college admissions board, I'd give consideration to an L2 speaker if his verbal scores were unusually below his math scores.

I'm less familiar with math testing in general, but as far as direct effects go (not something caused by language of instruction) my guess is that it would be limited to instructions and word problems.

I'm not saying that. What I'm saying is that it's not clear that the advantage is so much as to be a relevant factor. I grew up in a very poor minority neighborhood and the language the teachers used in school was the same as the ones on the standardized test, so it's not like I wasn't prepared.

It's almost insulting to think that dialect matters that much considering how much exposure everyone gets to the mainstream dialect.

In China they've got a standard dialect and there doesn't appear to be any advantage for those that are from the area north of Beijing where the dialect originates.

How much of an impact it has is debatable, but the academic consensus is that it is relevant. If you trust your own experiences and opinions more than the current research, there isn't much I can do to convince you otherwise.

>It's almost insulting to think that dialect matters that much considering how much exposure everyone gets to the mainstream dialect.

Whether it's insulting is irrelevant to whether it's true.

>In China they've got a standard dialect and there doesn't appear to be any advantage for those that are from the area north of Beijing where the dialect originates.

I'd be interested to know what your basing this assertion on. Gaokao scores in Beijing are distorted by families who try to move there from other areas to take the test and migration in general.

Combined with with relative lack of reliable data, I'm not sure how you could confidently make this claim.

Do you have a link to the study? I'm curious what "relevant" means in this case.

From the beginning I've not denied that dialect can have an effect, only that I believe it to overwhelmed by a sea of countless other factors that cause someone to succeed or fail at getting into college, and I haven't seen any data that shows that it is the prevailing factor and one that truly matters.

The subtext to my comments is that this a debatable point, and I interjected because others were throwing veiled insults and calls of racism at anyone who tries to discuss this, which is an unfair way to shut conversation down.

>only that I believe it to overwhelmed by a sea of countless other factors that cause someone to succeed or fail at getting into college

Yes it can be and is at an individual level.

If you removed 50 points from every Black student's SAT score just because they were Black, there would still be countless other factors that caused them to succeed for fail at getting into college. Their being Black wouldn't be the largest factor (family income still would be).

But across the entire population, you would see a significant drop in the number of Black students in college.

As to a link: Here's a link and a quote from the abstract of a relevant metastudy

https://www.ncbi.nlm.nih.gov/pubmed/26090843

"The results showed a negative and moderate relationship between dialect use and overall literacy performance (M effect size = -0.33) and for dialect and reading (M effect size = -0.32). For spelling and writing, the relationship was negative and small (M effect size = -0.22). Moderator analyses revealed that socioeconomic status and grade level were not significant predictors for relations among dialect use and literacy skills."

All languages are going to have dialects. Trying to eliminate that "bias" is going to just result in removing language entirely from exams, unless we decide to make tests for every single test taker. But the point of standardization is to have a solid, uniform goal for all to work towards. And if you're applying to an American university and you have difficulty with what's considered standard American English, you will have far worse troubles in university.

And again, people from families of non-native speakers do well. Generally better than people who only have family who speak the native language and dialect.

That people think it's discrimination is quite clearly silly. :)

It sounds like you really want to be arguing with something other than what I'm writing, so you've just decided to make that happen.

There is in all cultures a language of educated discourse. To take an extreme example, nobody ever natively spoke scholarly Mandarin and wrote eight-legged essays for fun. The SAT is less severe, but, yeah, it requires some effort and practice to be familiar with the language and the essay format. Not being able or willing to put in the effort to learn this very accessible information is not a good sign for being able to survive equally or moreso arbitrary environments in university.

What is it with this topic causing people to argue with straw men?

>Not being able or willing to put in the effort to learn

This has nothing to do with putting in effort. Across a population, people who grew up speaking a dialect will perform better on a test than people who learned to speak that dialect later.

This isn't a controversial statement.

If your goal is to ensure that speakers of other dialects have equal access to top tier higher education, you will need to account for that somehow.

If you dont care about equal access to speakers of other dialects, then you don't have to do anything.

But yet Asians perform the best on standardized tests, not whites who comes from England?

Do you have any data on white English immigrants who take the SAT? I sure don't.

The children of East Asian immigrants perform better than the children of white Americans. How does the language hypothesis you put forth account for that?

1. We don't know that. We know that Asian Americans outperform whites, but Asian American can refer to 1st, 2nd, 3rd etc... generation immigrants.

2. Asian immigrants tend to learn the prestige dialect (the dialect the SAT is written in), so if their children are native English speakers, they tend to learn that dialect natively.

4. Language differences are a factor in performance but obviously not the only factor or the largest factor.

> 1. We don't know that. We know that Asian Americans outperform whites, but Asian American can refer to 1st, 2nd, 3rd etc... generation immigrants.

A quick glance at the first graph on this page will tell you that almost all Asian Americans are either first or second generation:

http://www.pewresearch.org/fact-tank/2017/09/08/key-facts-ab...

Key stats: Asian American population in 2015 is slightly more than 20 million. In 1960 it is about 1 million and only about 2.5 million in 1980.

> 2. Asian immigrants tend to learn the prestige dialect (the dialect the SAT is written in), so if their children are native English speakers, they tend to learn that dialect natively.

Is it somehow easier for Asian immigrants to learn the prestige dialect than Hispanic immigrants?

None of that shows the rate at which Asian Americans take the SAT, the birth rates of 2.5 million Asian Americans who were here in 1980, or the ages of the Asians who've immigrated since. So saying that we have an extra 17.5 million Asian Americans, and Asian Americans outperform whites on the SAT, therefore 2nd generation Asian Americans outperform whites isn't supported by the evidence presented so far.

However none of that really matters because you ignored my last point.

Other factors such as family income are larger factors that native language.

>Is it somehow easier for Asian immigrants to learn the prestige dialect than Hispanic immigrants?

If it's easier is irrelevant because Asian Americans are more likely to speak the prestige dialect than Hispanics. But to answer your question, yes it is eaiser because Asians have much higher incomes, and they integrate with white communities faster--at least in part because Asian communities are smaller and more spread out.

Yeah, and my original point was that these tests don’t discriminate based on skin color or ancestry. What you’re saying is that different races have different aggregate outcomes, but that that is due to different aggregate income levels. That doesn’t really sound like you disagree with my original point.

No. I'm saying that income plays a larger role than dialect. Dialect still plays a role. If dialect differences were accounted for, speakers of non-standard dialects would have greater aggregate scores.

But you’re saying that dialect is caused by income!

No. I'm saying some of the differences in dialect between 2nd generation and later Hispanic and Asian immigrants are caused by income.

That is not the argument.

Tests are written by people who talk a certain way, in a way that is easier to process for people who talk the same way. Different communities use different language, and this is reflected in how quickly different students process the problems on standardized tests. There's also the issue that standardized test scores are higher for students who have the time and encouragement to study for them - things which are scarce in poorer families where the student might have to work or help take care of younger siblings. Years of institutionalized racism (loans not given, redlining, denial of employment opportunities) have economically crippled some ethnic communities.

The diversity thing is mostly a red-herring. Admissions offers are abandoning standardized tests because of Goodhart's Law.

> for students who have the time and encouragement to study for them

It's not just time and encouragement. SAT/ACT prep is serious industry. $100+/hr tutoring sessions. Owners of good SAT prep businesses can make six or seven figures in a few months.

And good prep courses really do work. Tests, like coding interviews, are a game you can hack with enough time and effort.

The signal is totally washed out; with enough money, you can learn how to take the test well enough.

The research suggests SAT test prep has an effect of about one seventh of a standard deviation – about 50 points out of 2400[1]. That's significant but not huge. Most of the perception of large gains comes from the fact that people do better as they get older with or without coaching.

(None of the research is great, but better than nothing.)

[1]: https://www.jefftk.com/p/sat-coaching-what-effect-size

The College Board released new results in 2017 showing a more than 50 point increase out of 1600 versus students who didn't practice.

This was with just 20 hours of practice. Many high achieving students practice for 5+ hours per week.

South Korean cram school style 2-3 hours per day is in a completely different league than a standard SAT prep course, but much of the research that shows no result counts them the same.

This phenomenon is known as the practice effect. It has a ceiling in terms of IQ tests, just like it does in terms of physical tests. IQ tests are designed to be as close to ungameable as possible.

This is like pointing out that I can make large gains in my 100m sprint time by practicing for half an hour a day for a month and predicting that I can make the Olympic team if I put in a thousand hours practice. Novice gains are real and significant but they peter out pretty quickly.

IQ tests are gameable to a degree. They are accurate across large populations largely because most people don't practice, not because they are actually ungameable.

SATs are different in 2 respects.

1. There is a large incentive to practice.

2. It isn't an IQ test, it only correlates fairly strongly with scores on IQ tests.

SATs are highly knowledge based--most of the math questions can be answered by memorizing algorithms, and the verbal section depends heavily on vocabulary.

>This is like pointing out that I can make large gains in my 100m sprint time by practicing for half an hour a day for a month and predicting that I can make the Olympic team if I put in a thousand hours practice. Novice gains are real and significant but they peter out pretty quickly.

It's unlikely you could become an Olympic sprinter, but with enough practice you could probably move into the top 1% of everyone in your age group because the vast majority aren't going to spend that much time practicing.

The SAT equivalent (for the 2018 test) would be somewhere in the 1500-1550 range.

The SAT is highly g loaded and it correlates with tests that are designed and marketed as IQ tests at levels sufficient that saying it isn’t one is quibbling. The lowest estimate I found for IQ SAT correlation was an r^2 of 0.76.

SATs are more knowledge based than a test designed to be culture fair, like Raven’s Progressive Matrices but they’re not meant to be culture fair either, they’re for testing educated English speaking Americans. The test population are in high school but there’s nothing on the test that you wouldn’t have known by the end of middle school.

As far as algorithms and vocabulary go, most people have limited ability to follow complex algorithms because that requires a lot of working memory, which correlates with... Intelligence!

Vocabulary correlates with IQ about as strongly as SAT score, yet vocabulary can clearly be hugley improved with practice.

Even if correlation with IQ somehow proved score durability, a correlation of 0.76 or even .8 is low enough to allow for huge swings away from IQ in a very large percentage of the population.

A big chunk of students practicing and increasing their scores 200-300 points could easily be hiding in that 0.76 correlation.

>As far as algorithms and vocabulary go, most people have limited ability to follow complex algorithms because that requires a lot of working memory, which correlates with... Intelligence!

Most algorithms at that level are designed to be written out step by step explicitly to reduce working memory requirements.

Part of SAT prep is learning algorithms for solving word problems that reduce working memory load by memorizing patterns in the questions and writing down the components in a standardized form.

You said

>As far as algorithms and vocabulary go

And then never mentioned vocabulary.

It has nothing to do with working memory and most definitely can be improved dramatically with practice.

One more thing. The SAT was rewritten in 2016 to be far more content based, so the correlation with IQ will likely fall even further.

There are SAT prep books in the thrift store for a dollar. Nobody is shut out because of poverty.

And good prep courses really do work. Tests, like coding interviews, are a game you can hack with enough time and effort.

Years ago I took the GMAT and scored very highly, 98th percentile, and GMAT-taking is definitely a learnable skill in and of itself. Then again, maybe the test is really a proxy for “willing to do prep work upfront”. Having said that obviously it filters for those who have the opportunity to do so, but I am willing to give the benefit of the doubt and say the latter is not it’s deliberate intent.

Why are people jumping through hoops trying to imagine possible reasons for differences in academic performance when the best predictor is IQ and that can be measured from just a few years of age - before they would have to spend time studying or taking care of younger siblings. We already know that east Asian children have higher IQs on average than whites. That explains their better test scores at school. We also know that blacks have lower IQs than whites which also explains their lower test scores. So if you're looking for a cause of the different outcomes, you should look for the cause of different IQs. Since that starts young, maybe black parents don't hug their kids as much as Asian parents (that affects IQ) or maybe it's genetic (that affects IQ). The popular Raven's matrices IQ test has no words, just abstract images so there's no language bias there. Researchers have studied this to death over and over again for 100 years and they keep successfully reproducing the same results. They try everything to eliminate bias, such as having the test administrators be the same race as the subjects and controlling for socioeconomic status and going to different countries and controlling for the economic status of the country and so on. It's a solved problem down to the IQ level. What's not solved is how to improve the IQ of a person after they're born. You can improve it before birth by choosing parents with high IQs.

> Tests are written by people who talk a certain way, in a way that is easier to process for people who talk the same way.

So have your test writers distributed in a way that matches the general population. Don't just blindly try and force the population of VLSI design engineers or whatever to mimic the city's demographics.

We don’t use any standardized testing in Canada and I don’t think it’s affected our higher education system at all.

Are there any proven advantages to standardized testing?

That's true for getting into undergraduate programs, but when it comes to graduate programs, dentistry, medical, law schools etc, I believe we still do use the same standardized tests as in the USA.

For grad school, GRE and the other tests exist to filter out obviously unmotivated candidates. So, a good GRE score can not get you in with any certainty, but a bad one can disqualify you.

I think it depends. When I wrote the GMAT (albeit in 1992), a good score would get you into most Canadian business schools even if you had mediocre marks.

My recollection of the GMAT's difficulty was that any high school grad with an A average could easily score higher than the 80 percentile on the test. While I was happy my result gave me the ability to choose which business school I wanted to go to (my marks, while not bad, were also nothing to brag about), I also think the GMAT was a useless selection criterion.

I am not familiar with business school admissions, but it is very rare nowadays to get into non cash-cow grad school programs in STEM fields on the basis of the GRE score alone.

Obviously unmotivated?

Your thoughtless dismissal of those less intelligent is enraging. Do you think all the people who work harder than you to get worse results are less motivated?

There are tens of thousands of people who would love to get into top graduate programmes, who love the subject they study but who just aren’t as smart as other people and you’re shitting on them as unmotivated?

Conscientiousness and intelligence are negatively correlated.

https://s3.amazonaws.com/academia.edu.documents/30661143/Con...

The GRE is designed to reward hard workers as well as intelligent people, that's why it is a widely used metric. Grad schools need hard working people as much as (probably more than, to be fair) they need intelligent people.

I am joining a CS PhD program in an US university this fall. The GRE, especially the math part, is high school mathematics. If you are not good at math, you can ace it by studying hard for a month.

That is why a low GRE quant score reflects that you haven't really bothered.

I went to grad school in Canada and I didn't have to do any standardized testing.

People recommended doing your GRE but it wasn't required.

There was a fairly long interview on NPR today with the Director of Admissions of University of Chicago on this today.

From the part I heard, a big reason given was that you can significantly improve your SAT/ACT scores by taking test prep courses, which reduces the reliability of the test. They believe that can get a better idea of an applicant's academic potential from their grades.

My high school graduating class (~15 of the ~250 were "Valedictorians" thanks to grade inflation and a lack of a weighting system; they all had 4.0s) says differently. This was in an inner city public school, mind you.

Admissions officers know individual high schools, and the different grading systems. They can separate the bullshit for the most part

In other words, if you're from a shitty high school (or even a "best in the county" high school, if it's a unexceptional county) you're shit out of luck - because now the admissions officers will just reject everyone from that school out of hand, no matter their grades.

Objective standards can really help people without the connections to succeed otherwise.

That's the literal goal of the SAT. A standardized test across all schools.

As a non-American, I always found the GPA system a little strange. In high school, our marks were always scores out of 100.

As someone who's moved between states and cities all across America, GPA really should be dumped for system like that.

I've lived in schools where an A/4.00's average score was 90-100 and I lived in another school where it ranged from 95-100. 3.00 would be from 80-89 at the one school and 90-94 at the other.

From raw GPA data, students who had a 90% average would look great if they graduated from the first school, but just OK if they graduated from the latter school. And I've had friends that went to schools that had max possible GPAs of 5.0.

It's a mess.

There is definitely a cultural divide. Among most of my white and Latino classmates, the SAT was treated like an IQ test: you just show up and see what happens. For better or worse, this is probably what colleges originally intended, for the SATs to be legal and non-controversial proxy for IQ. My Asian and Indian classmates treated it like any other test, something you can study for and improve at. They expected that they could boost their scores hundreds of points between their PSATs and SATs by taking classes and studying hard.

Studying hard certainly gives one an unfair advantage :-)

Your comment and questions are only related to the post as far as they both concern higher education. From your phrasing it's clear what you think the answer to your questions are.

> could be perceived as discriminatory

I believe "could be perceived" is incorrect; there is a large amount of evidence that it is discriminatory, and that therefore the most deserving students aren't getting opportunities and that the university is shortchanging itself on the quality of its student body.

> one stated goal to make the student body mirror the general population

Who stated that? Also, it's not a bad guide given that talent is distributed relatively equally.

> Is it even the role of higher education to remedy the cards that have already been dealt, and even if so what evidence is there that it actually does what people say it does?

What do you think, and more importantly to fellow users, on what is it based? Shall we stand still until nobody raises another question?

> It seems like ...

Sentences that begin with those words can end in almost any way the author wishes, and in almost all cases we will have learned nothing.

> It seems like we're bending over backwards to toss out any kind of test or criterion that could be perceived as discriminatory.

Yes. SAT/ACT scores are terrible signals because of Goodhart's law. Professional tutoring in wealthy suburbs starts at $100/hr. Many ambitious students opt for four-figure private prep courses.

SAT/ACT scores may have been good signals 20 years ago. But today, those scores discriminate against poor students and students aren't inculcated into the "elite institution rat race" from a young age.

That discrimination matters to U Chicago because they are interested in admitting brilliant, creative thinkers. Not perfectly groomed drones. If you don't require test scores, then there's a lot less back-and-forth about admitting that one kid who did publishable research as a high schooler but also happened to do only so-so on the associated SAT subject test.

Qualitative evaluations have many problems, but overly quantitative evaluations also have their failure modes. Especially when the candidate knows the quantitative function they're supposed to be optimizing.

Reply on news.ycombinator.com