ChatGPT Trading Algorithm Delivers 500% Returns in Stock Market

>The researchers collated 67,586 headlines pertaining to 4,138 unique companies during this time, using web scraping techniques. These headlines were subsequently filtered for relevance, with the team narrowing their focus to full articles and press releases, excluding any stock-gain or stock-loss headlines. They also removed duplicate news to ensure only fresh information was used.

67,586 headlines over more than a one-year period is under 200 headlines per day. One human analyst could trivially perform sentiment analysis on that many headlines before lunch. Practically perfect sentiment analysis on this scale is thousands of years old.

There is no "special sauce" from ChatGPT.

They tested six different strategies and only two-three of the strategies returned profit on this year. Even the ones that turned out profitable were only profitable within the first half of the year and provided no return during the second half of the year.

Seems similar to p-hacking.

The timeframe was only 1 year, therefore this is a rather inconclusive test. It warrants more investigation, but if a hedge fund understands that feeding good news or bad news to an AI makes the stock market move, eventually they will control the news factory to control the AI.

It is a neat trick though. I can't read that fast and I can't make 500%.

Interesting experiment and a good start at evaluating the concept, in particular, the fact that they compared to other methods makes it way stronger than similar concepts that forget to even compare to index. However, even though they clearly tried to avoid obvious pitfalls of backtesting, they also left lots of potential sources for bias.

Their method of collecting news is based on what popped up in their post-facto web search, it also relied on them classifying them for relevance. This can generate bias compared to, for instance, feeding all articles tweeted from a specific news source irregardless of "relevance". I also wonder what assumptions they've made about timing, slight errors in timing estimated could have huge impact on results.

The fact that the shorting seemed to make for the most part of the gains (although the long strategy was not terrible at 50% gain vs index going down) might also be something somewhat biased towards the fact that this was a downturn year and it might not fare as well in a more regular year.