Hacker News Reader

> The common thinking is that everyone is using ChatGPT, therefore you’ll be left behind if you don’t cheat.

So true. I am aware of classes where everyone who didn't use AI cheated.

The simple reality is that if AI makes better answers than a student, and exam scores are normalized, then students who don't use it will fail as soon as a decent proportion of students do use it.

> and exam scores are normalized

This never should've been done to begin with. Education isn't supposed to be a competition.

I think the intent is to calibrate for instruction quality

When I was grading labs as a TA, the intent was communicated to me rather as "per university teaching guidelines we mustn't have too many students get the top grade but we also mustn't have too many students fail"

Reminds me of "stack ranking" in performance reviews.

The purpose of normalized grades is to habituate future workers to this so that by the time they find it in the real world they don't resist.

It's part of reproducing the labor-captial relationship.

It also helps to avoid populist teachers that give everyone a A+++ to avoid students complains, and also idiots that give everyone a C because only God is A and only the teacher is B.

(We don't use that method here, we use other method to try to avoid both problems.)

Where is the incentive for test makers in academia to accommodate this outcome? It sounds nice but I don’t think jaded professors or overworked, inexperienced, and stressed TAs have a reason to do this. It sounds nice but it doesn’t actually seem connected.

If the purpose of a system is what it does, what is the purpose of this education system that seems to uniformly and spectacularly fail to educate children?

It's not supposed to be a competition, but there should be incentives and oversight and controls and all the features you'd want to be able to reward outliers and foster excellence and all the good things while minimizing the bad.

What we have is tragic and absurd.

https://www.astralcodexten.com/p/come-on-obviously-the-purpo...

An even-simpler reality is that, to the extent AI helps you cheat on your professional exams, you're about to enter a dead-end profession that will no longer exist a few years from now.

Interesting thought. I feel currently AI is most useful to those who have a decent understanding of the subject material and can critique any output. To wholesale trust the output of AI and remove any human in the loop, well, it needs to be really correct all the time.

Exams are a different beast and really a subset of a range of common problems.

Still, I'm very curious what happens when people who have just cheated their way through college, or these kinds of professional exams, meet the real world? Will they all get fired a few months down the track?

I lot of people who took those exams on paper are no good at actual finance jobs. It is more about gatekeeping than anything else

Still, I'm very curious what happens when people who have just cheated their way through college, or these kinds of professional exams, meet the real world? Will they all get fired a few months down the track?

They will continue to use AI to do their jobs. Eventually, the people who pay their salaries will ask themselves why they continue to pay them.

To wholesale trust the output of AI and remove any human in the loop, well, it needs to be really correct all the time.

It's not now, but it will be. Accounting is what you might call an exact science, one where creativity isn't rewarded and where hallucinations by one model can be detected and corrected by others. There is no need for humans to do this type of work.

> Accounting is what you might call an exact science, one where creativity isn't rewarded

Not sure of this. Corporations pull a lot of creative accounting all the time:

https://en.wikipedia.org/wiki/Creative_accounting

well, lets take that as true (no more humans doing accounting). Doesn't that mean that there'll be a knowledge gap there? What happens when new rules come along (laws change all the time which will affect accounting practices). How will an AI (at least the current batch) learn what needs to change when there's no prior art for it to lean on?

I mean sure, if we ever get AGI then all bets are off, but, as far as I know we're not there, and LLMs are unlikely to evolve into AGI. They're not thinking right? It doesn't actually _understand_ anything right? I mean, I'm quite probably wrong here, but, as far as I can tell it's really just very fancy backwards autocomplete.

(Shrug) Thinking, unthinking, meh. If you can perform near the top level at the International Math Olympiad you're not going to have much trouble with the tax code.

What will likely happen is that future tax codes will be written specifically with rules oriented towards automation. We won't have to train general-purpose LLMs by shoving trainloads of IRS documents, Congressional records, and tax court cases at them, as happens now. I think we'll see lots of specialized models ramp up at some point, for efficiency's sake if not just for accuracy and traceability.

>> I'm very curious what happens when people who have just cheated their way through college, or these kinds of professional exams, meet the real world

Certification questions, as well as interview questions usually quite far from the real world. The best strategy is to fake everything to pass through and then learn at work.

Basically fake it until you make it. The hardest part of swe job is to land it.

At the moment very many of the people who take those exams wouldn't get a job with me. They are only good for box ticking audit work. AI will take their jobs. Less than 1% will make audit partner and actually pay for the effort the exams require. I can see why people cheat.

A small % transition to industry from practice, and have too learn their jobs all over again. That group will still exist in my view. They are the ones who will be asking AI the right questions. God only knows how we will train that 1%!

To me exams is possibly the easiest thing to tune AIs for. You have the clearest metrics, you possibly have lot of material in training data already. And well those tests are not even supposed to be novel. Seems like a thing that LLM should really excel at.

The common thinking of often a mental pattern of that intersects somewhere between laziness and comfort.

Is this the sort of thinking of “everyone needs to be able to do calculus in their heads with calculators around” or “you still need to write in the age of computers/printers” or something different?

> Is this the sort of thinking of “everyone needs to be able to do calculus in their heads with calculators around” or “you still need to write in the age of computers/printers” or something different?

I can't tell - are you suggesting these aren't good practices/traits to be learning when people are still in the "fundamentals of education/learning" stages of their lives?

I did all my basic differential and integral calculus studying by mind only. I don't do it that way in my career day to day now - nor could I without some serious practice. But the efforts I took in learning this way in undergrad made me a much stronger student and made me much more comfortable leveraging calculus in more application driven fields of study.

Well yeah my suggestion is do you need to get the skills when the skills can be do for you? As a lot of popular thought goes as per the parent comment.

I have never been in a calculus class where a calculator would be anything other than a paperweight

There are calculators with CAS programs that can symbolically differentiate and integrate expressions or even solve certain classes of ODEs/PDEs

In 1988 my Calc 1&2 classes were partially taught using Maple on an MTS mainframe. It did NOT make any of it easier.

The TI-89 came out in 1998 and can do a lot of calculus work. It can go very far in entry level calculus courses and can be very useful for checking work even in the higher courses.

But both of those statements are true, and for the same reason. A calculator isn't a human brain capable of doing math, and writing isn't the same thing as a computer. They're different things.

I can give a 5th grader a calculator and he's not passing college calculus. I can even give him a whole ass PC and he still isn't.

As for writing, again, it's its own thing with its own benefits.

I still write all my notes, because it helps me remember. There's something specifically about using my hands on paper that makes things stick better in my brain. It's less convenient than computer notes, and much harder to organize. But they accomplish different goals. They're not for reference, no, I usually don't ever read my notes again.

> AI has taken it to the next level. Previously, with many exams you would still have to know how to identify the concepts and related keywords in a word problem to even know what words to look for in the index of the books on hand before you could get to the right page to start cheating.

Online exam cheating was easier than that.

20 years ago, for online quizzes cheaters given would simply get the three guys in their frat who took the class last year to sit nearby and act as human ChatGPT.

The solution was simple - limit easy-to-cheat means of assessment like online quizzes to 10% of the final grade, with 90% of the grade dictated by in-person exams and equally hard to cheat options.

My mom always made a couple different exams to hand out so you couldn't just copy answers. A lot of my professors had a rule if you aced the final you got an A in the class. I think that was also done to not reward cheating. Having to take the bar is probably another one like that. Cheat all you want in law school if you can't pass the bar too bad.

Or you could just have someone who could pass the exam in the same room as you.

LLMs make this way easier but you can pay someone who gives private lessons in any subject and they can easily take an exam for you.

It is indeed the same.

But in practice, having another human cheating for you was often unpractical: people don't usually like helping cheater, and simply trying to find an accomplice may get you in trouble. Because of that, it is relatively inefficient and therefore not a real problem and not a real impact on the final quality of the evaluation.

LLM is indeed just the same, except that finding an accomplice is now easy and without risk.

So before just the rich and connected could cheat, now everyone can cheat and is bad. Funny no?

No, because allowing remote online exams is a relatively new thing

The rich and connected adjacent to accounting and willing to cheat would have most probably already worked somewhere else money-related and making an order of magnitude more money.

"Everyone can cheat" is not making the poor equal to the rich, not at all. The rich gets 2 things: they can pay for stuffs and they can get away for stuffs just because they are rich. The fact that no one is paying does not mean that magically the poor can get away for stuffs.

The things they can get away with includes, for example, the fact that they don't get fired when they don't know their job. The poor still gets fired, the fact that they can now cheat more easily just mean they are shooting themselves in the foot.

> AI changes that. Now you don’t need to know anything at all.

This is democratisation. Is the cheating the problem, or is the system the problem?

If it's so easy to cheat that a person with no previous knowledge or experience can appear to be very knowledgable and/or experienced by typing a few words into a computer, I would probably suggest the system, and all the gatekeeping and profit extraction that has gone into that system over the years, is the problem.