Hacker News Reader

Comment by nigamanth | Hacker News Reader

ChatGPT could easily get 100% on the Mathematics sections, but I never thought it'd get 80% on the reading/writing section.

View on HN

I’d expect the opposite. It’s not very good at precise problem solving but it’s very good at writing verbose flowery bullshit.

"reading/writing" is actually reading comprehension/grammar. the essay was optional from [####] until june 2021 when it was discontinued. so the ai isn't writing, it's answering "writing" questions which are grammar/sentence structure/correctness/etc.

also if you click the link, it was all multiple choice.

You can have it call out to python or some other runtime to help with that. It just needs to be able to turn a problem into code.

An AI using a calculator. Ha!

It can easily do mathematic functions and SAT syllabus. Despite the word problems, by assuming ChatGPT should easily be able to solve simple arithmetic.

It failed at around day 3 in advent of code, and could only pass those as they likely had seen the exact solutions to the same, but differently asked questions.

It really can’t do any novel thought though, which is necessary for mathematics. Language models are fundamentally limited here (even a 3 years old can easily follow some easy rules “ad infinitum”, like counting. Language models have a limited window for inference)

There are similar reddit threads complaining about its inability to do basic math:

https://www.reddit.com/r/ChatGPT/comments/103bw12/chat_gpt_g...

And various discussions elsewhere about its inability to do word problems:

https://ai.stackexchange.com/questions/38220/why-is-chatgpt-...

It's so bad at math that they made it start responding that it can't do math.

Reply on news.ycombinator.com