I think the closest we’d be able to get to this is maybe smaller, more granular standardized test that do a better job of testing multiple specific dimensions over an academic career vs a big bang at the end.
Somehow that actually sounds kind of creepy, but if that’s what you meant that’s the only way I see this done more effectively.
In your experience, what other ways or ideas have you heard?
This is the heart of the concept, yes. We should be attempting to make in-classroom assessments as trustable as standardized tests which would allow them to continuously implemented during instruction. The quantity of data will provide a significantly more robust representation of a student's abilities today, and over time, compared to the infrequent snapshots of the ACT/SAT/etc.
Don’t we pretty much have this via AP tests and SAT II Subject tests?
> over an academic career vs a big bang at the end.
I hate to say this, but describing the SAT as a “big bang” strikes me as funny. It’s a test of fundamental literacy and fundamental numeracy for well-educated people. It’s not that hard.
The catch is that the preparation that yields high scores is developed over years.
For math, it’s taking the subjects and understanding the fundamentals of these subjects. Sadly, overall math education in the US is very weak and very skewed towards males. Note that most Asians (East and South) that have to take the SAT or GRE tend to and expect to get very high scores — it’s just that easy.
For the language sections, it boils down to having read and understood enough high quality “high brow” content. This high brow content covers most if not all of the vocab, reading comprehension, grammar, etc. that are needed for the test. Sadly, most young people are not exposed regularly to this type of content, and they rarely read it critically if they do have access.
I’m not sure what exactly we are trying to finesse with your suggested changes.
Maybe you should ask then instead of going on some dumb rant?
Maybe that was an attempt at a rhetorical question, but given how much you missed the mark, I took it more literally.
Big bang btw just refers to doing it all at the end all at once vs in smaller steps over time, it’s not to be taken literally. It doesn’t imply grandeur or anything like that.
Also, you missed the entire point of OPs message, as an educator, they were suggesting that perhaps the SAT doesn’t do all you suggested.
I don’t know what we’re trying to finesse either, can you explain what you mean by finesse?
Except that it does, and there is data to back it. As an example, some elite schools have stated that they really like using the SAT since it predicts success at their respective schools.
One simple Google search (there are many more):
https://www.manhattanreview.com/sat-predictor-college-succes...
“On average, SAT scores added 15% more predictive power above high school grades alone when attempting to understand how students will perform in college. Within narrow high school GPAs, SAT scores further helped predict a student's future success in college.”
> I don’t know what we’re trying to finesse either, can you explain what you mean by finesse?
What do “granular tests” tell us that the SAT, SAT IIs, and AP tests do not already tell us?
The short answer is “not very much”.
The results of these tests are accurate enough to make meaningful predictions and decisions.
Note that there is some interesting work in learning portfolios (probably an optimal variation of “granular test”) that can be useful for structuring individual learning plans, but that sort of misses the point of broad-based standardized tests, which is normalizing widely varied experiences.
Note that standardized tests fail two types of people at the margins, but these are largely addressable, and the benefits outweigh the costs imho:
1. Folks who have test anxiety. Familiarization with the test reduces this substantially for most people affected by this.
2. High-performing folks (think top 5% or better) who don’t have high attention to detail on computer adaptive tests. Also addressed by test familiarity. I’ve seen folks who should be maxing a test get lower than the max because they were careless in some early questions.
Apart from, I guess, a one-on-one interview for every student with the colleges they apply to. That wouldn't scale, and people would complain about bias with that, and not without reason.
Currently, a GPA and a few test scores (SAT, ACT, etc) are the most in depth view into a student's high school performance that colleges have, but in the future, we hope to see continuous tests throughout high school that don't just show a single letter/number, but a holistic view of a student's progress, effort, etc. (Obviously the data should be controlled by the student) These could also help a student decide where to focus their study efforts.
How does AutoMark account for disparities in quality of education?
There are already some great quality resources online to learn subjects like math, history, and science, but we believe the missing piece is grading - you're never actually evaluated whether you learned something online.
We think by giving students unlimited practice and feedback, along with quality learning materials, we can work on chipping away at this issue. AI will allow scaling access to quality tutoring, practice, and exams, all of which significantly improve eduction (currently, this costs $40-$X00/hr, but will cost much less with AI assistance). It's definitely not a quick fix to solve the issue of instructional quality.
Your business is selling an advantage to the rich students coupled with increasing student stress throughout school. Your "holistic" view can't account for "students grades are slipping because they were sleeping in their friends basement after their parent killed another parent" or "students grades are slipping because they are working 40 hours a week, illegally, at 14", but it will holistically show "student got a B+ so their dad who makes $500k a year hired a tutor for 10 hours a week".
All I get is that if you start from an extremely unfounded point you'll get to the wrong solution. But that doesn't say anything about the points people are making here
You really don’t need any more resources for SAT.
Being on this site, you're probably someone who values education. And you probably raised your kids to be capable. Or maybe they just lucked out and got the correct weights to succeed in that particular scenario, but you can't assume that the situation for everyone is the same as you!
Many people simply can't learn by just doing practice tests and watching videos. If it were so easy, then we wouldn't need teachers or schools in general. Mentorship and guidance are important, and it's the reason why so many people want to get into universities in the first place.
Essentially, yes you can do without, but having it helps. Overall, that would create a disparity that favors more wealthy people.
Sure, it's biased. Everything is. But it's biased in an open, auditable way that students of any background can prepare for. The rich will always have an advantage of course, but this is one of those areas where grit and determination can actually close the gap considerably.
Remove that, and a benchmark that anyone can understand and prepare for is just going to be replaced by something more opaque. You think a standardized test is biased? Do you think the application committee at most universities is going to be unbiased when they have nothing to go off but a student's essay and zip code?
It's incongruous to me that the same people who I think have a deep understanding of unconscious bias also seem to be the same ones wanting the standardized tests gone. I don't think it's going to help the people they think it will.
Anything that seeks to further widen the rich / poor gap like this is in my mind, completely unethical.
We’re already in a mental health crisis for students. I do not think adding more surveillance is going to do good there.
I may be wrong, but I hope the founders have thought a lot about this and considered what negative impacts, if any, this will have of students.
How would it go about monitoring the students?
Apply an AI agent to the sum total of student output? Can't wait for the entrenched racial bias to leak into the AI and a host of other issues.
If you could get an AI implemented that didn't involve reporting to the greater regulatory bodies, I can see that working, but that's the issue with things like that: management wants to see them and then the stats rapidly get watered down to meaninglessness.
It would be crazy to see if an AI can start diagnosing home issues, abuse, etc from student output and performance. I can see a lot of benefits, as long as the AI output is private.
The vision is not as grand as your last paragraph.
Higher education roughly correlated with higher salaries which roughly correlates with higher status.
I’m not sure where you are, but I’m in the US (specifically California). This dream of yours pretty much exists.
> Therefore, we wouldn’t need an arbitrary and flawed aptitude tests.
The aptitude tests are more for matching folks who will be more closely matched in terms of quantity and speed of learning. There is definitely some virtue signaling as well via elite admissions, but that doesn’t exclude anyone from learning pretty much most subjects.
Maybe I don’t understand what you’re suggesting.
Ranking and benchmarking between educational environments would require an additional psychometric evaluation layer. This would not a Day One function, so the tool would focus on maximizing utility as an assessment assistant tool for instructors. There is a bridge to be built between formative and summative assessment. If this thing doesn't make them better instructors, it would be dead in the water anyway.
Everyday, all across the country - we pay teachers to teach and assess their students. Those assessment results aren’t trustable for benchmarking and ranking students, so we have standardized testing.
Obviously, yes - one intention of this concept is that it serves the same function as standardized testing, but it does so while eliding the externalities of cost and single, high stakes exams by allowing classroom assessment - made and proctored by the instructors during their standard lessons - to become trustable.
The one thing I loved about schooling in Iran is that there wasn’t a constant system of surveillance.
I hate the idea of long term data collection during a time a kid is supposed to be able to learn.
Western students have the highest suicide rate in the world. People are quick to blame social media, but never look at the pressure we put on them.