A group of researchers has discovered that the GPT-4 program developed by OpenAI outperforms humans in financial analysis for large companies like Apple.
Three researchers from the University of Chicago Booth School of Business — Alex Kim, Maximilian Muhn, and Valeri Nikolaev — conducted a study to see if GPT-4 could analyze financial situations based solely on numbers, without providing any context to the OpenAI-developed program.
The analysis did not focus on the text typically included in quarterly reports, such as the Management's Discussion and Analysis (MD&A) section. The researchers wanted to see if Artificial Intelligence could interpret only the numbers to provide the most accurate financial analysis, as reported by Business Insider.
Where AI Performs Better
Over the course of a year, the study authors examined over 150,000 data points collected for each firm, from around 15,000 companies, between 1968 and 2021.
The study found that analysts had a 53% accuracy in predicting future earnings for one month.
The researchers then fed the financial situations into GPT-4 without any textual information and anonymized the data so the model wouldn't know which companies' data it was analyzing.
When GPT received a simple command that does not use "chained thinking" – meaning the researchers asked the program to respond without breaking down the request into step-by-step instructions – the model scored slightly lower than the analysts, with an accuracy of 52%.
However, AI performance improved when researchers used a type of command with chained thinking. By receiving more instructions and guidance, GPT achieved an accuracy of 60%.
The results showed that when GPT receives more instructions, approaching the analysis like a human, the AI model can outperform human analysis, even without key text information usually found in financial reports, the study reveals.
The researchers also noted that financial analysis and prediction are extremely complex tasks that require reasoning, common sense, and intuition, which can affect the performance of both humans and computers. This may explain why neither category produces an analysis with close to 100% accuracy.
Will Artificial Intelligence Replace Analysts?
The study demonstrated that GPT is better at analyzing larger companies like Apple.
One explanation could be that larger groups or more mature firms tend to be less unpredictable, according to the research author.
At a small biotechnology firm, for example, the company's profitability in the following year can be very variable due to factors such as a successful clinical study, making predicting based solely on financial situations more challenging. In this case, by interpreting only the raw numbers, AI "does not see" the broader context.
Moreover, GPT-4 could be more efficient in analysis, as humans can be biased in situations like rational information incorporation.
In these circumstances, could an AI model replace human financial analysts? "At this point, I would say no. It is still complementary. Technology will evolve over time, and then, who knows? Two years ago, we didn't even think such technology would appear," said one of the authors, Alex Kim.
He emphasized that the study does not indicate that analysts will be replaced by machines and that, for now, there are areas where human experts can excel and vice versa.
However, the research shows what tools a financial analyst could soon use to determine a company's financial situation more accurately.
T.D.