“Skills in machine learning and AI are no longer optional, with these technologies transforming data analysis and prediction paradigms.” That was the final word from financial services recruiter Selby Jennings in yesterday’s article, which covered the rise of Python programming in the industry.
But the final word in AI is far from here. The changes computing has wrought on the finance industry already render it unrecognisable from what it was when Guido van Rossum developed Python in the 1980s. And yet the conversation about AI and machine learning is only just beginning.
In the first of a follow-up series, Disruption Banking takes a look at the global phenomenon of AI in trading and its implications for the financial services industry.
AI Trading Equals Big Business
The global AI market in financial services was expected by data cruncher Gitnux to reach $22 billion by 2024. Yahoo Finance went a step further – predicting a rise from an estimated $38.36 billion that year to $190.33 billion by 2030.
Gitnux assessed that by 2023 the use of deep learning techniques in financial modeling had increased by 95% since 2019. Between that year and 2022, machine learning in quantitative finance increased by 58%.
And that’s just the tip of the iceberg. According to Quantified Strategies, in 2003 algorithmic or “algo” trading accounted for around a tenth of US equity orders. According to a 2011 study published in the Journal of Finance, by 2009 algo trading took up 73% of high-frequency trades – from “a starting point near zero in the 1990s.” In 2023, volume of order executions processed by algorithmic trading systems topped 25 billion per day globally.
That snapshot of data should tell you one thing: the future of trading is already here, and it’s dominated by AI and machine learning.
Similar but not Synonymous
Though they overlap, AI and machine learning are not quite the same. A subset of AI, machine learning allows systems to learn unsupervised from data and improve over time – without direct human intervention.
In financial trading terms, that would equate to most predictive models that draw on historical data to forecast future market movements. A more specific example would be large-language model (LLM)-based applications such as Blackrock’s Thematic Robot – which the bank’s portfolio managers use to blend human insight and big data to make better equity trading decisions.
LLMs are in fact another iteration of machine learning, and relatively new to the game, as we’ll see.
LLMs in Stock Trading
Machine learning has been used in financial trading for quite some time. The older ‘traditional’ method entails using numerical data such as price and volume to try to make accurate predictions about where the markets will move next. But the newer, shinier version involves using text-based LLMs as well – to parse useful information like news, filings, and earnings calls.
Perhaps the most famous recent experiment involving LLMs and the trading market was the Social Science Research Network (SSRN) 2023 study: Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models.
The results of this led SSRN to conclude “that sophisticated return forecasting is an emerging capability of AI systems and that these technologies can alter information diffusion and decision-making processes in financial markets.”
Debunking the Hype
However, though ChatGPT has made the headlines as a mainstream LLM, it was far from the first used to predict the stock market. And if published research is anything to go by, it may not even be the best.
The Journal of Emerging Technologies published research last September that praised the Google AI deep-learning model BERT and its financial-services spinoff FinBERT, introduced in 2019. That’s three years before OpenAI’s ChatGPT3 launched and took the world’s tech media by storm.
The journal study ran a test in 2024 that fed newspaper headline data into both LLMs to try to predict the stock market. It found that BERT and FinBERT respectively had an accuracy of 86.25% and 83.6% “after thorough comparison with benchmark algorithms.”
Study author Ritesh Tandon adds: “The findings show that the system might be used in the stock market and provide important information on LLM algorithms for stock data prediction and sentiment analysis using financial news headlines.”
BERT beats GPT
“There is no evidence that the GPT model performs that much better in predicting correlations compared to the BERT model,” agrees another study published by Cornell University in November. “With regard to an optimal strategy, we demonstrate that the BERT model […] is a better strategy.”
Simple Science, which worked off of the Cornell study and released findings of its own portfolio management tests on BERT and GPT in May this year, was even more emphatic. It calculated different allocation strategies across assets such as stocks, bonds and real estate, based on predictions from both models. The aim was to try to minimize risk and boost returns.
“While the [overall] results were promising, there was a clear distinction between the models,” it says. “The BERT model continued to shine, providing better and more stable results than GPT, especially when analyzing the [Federal Reserve] Beige Book. The simplicity of BERT’s approach allowed it to adapt to a lot of different market scenarios.”
Trad ML Struggles Too
However, Simple Science stressed that traditional models struggle to navigate the complicated landscape of stocks too, and warned of overly enthusiastic news coverage that amounts to little more than “gossip.”
“Market data is not only noisy, but it can also be complex,” it says. “This noise means that predicting how the market will behave is a major challenge. Traditional models often struggle to keep up with the chaotic nature of market data. Oftentimes, they end up fitting the noise rather than the actual trends, leading to inaccurate predictions.”
Beware Bias
We get it – predicting stocks is tough at the best of times, because if it weren’t we’d all be billionaires. But that consideration by no means lets LLMs off the hook.
Another problem highlighted with ChatGPT by the Cornell study was “look-ahead bias.” This occurs when a learning model uses data it could not possibly have had access to during the timeframe of the test period. Think of it as like giving your past self of one week ago today’s newspaper and being amazed when that past self delivers astonishing prescience.
The result is that back tests for how a model might perform at stock market predictions end up giving scores that are too good to be true. Guess what? They are.
Training vs. Trading Reality
A second problem that faces LLMs is “overfitting”. This is when a model learns data particular to your training set – but not necessarily applicable to future live forecasts. The result? Having performed well at trials run on historical data, it then fails dismally to deliver in a real-life situation.
Or as Tech HQ defines it: “Overfitting occurs when a model becomes too specialised in the training data, losing its ability to generalise in unseen scenarios. This reduces the model’s overall effectiveness and undermines its purpose.”
As at the time of writing, efforts are under way to alleviate the problems overfitting can cause. According to the MIT Technology Review, AI benchmarkers like ARC-AGI are keeping their test data dark so LLMs cannot fall foul of the overfitting trap.
LLMs Need to be Risk Assessed
Whether this kind of thing becomes industry standard is yet to be determined, but earlier this year Cornell University researchers called for greater risk analysis of LLMs.
“Standard benchmarks fixate on how well large language model agents perform in finance, yet say little about whether they are safe to deploy,” say academics, warning of an “illusion of reliability, overlooking vulnerabilities such as hallucinated facts, stale data, and adversarial prompt manipulation.”
They add: “Financial LLM agents should be evaluated first and foremost on their risk profile, not on their point-estimate performance.”
And speaking of risk: next week we’ll be looking at how AI itself is used to guard against it in finance. Until then, stay safe!
Author: Damien Black
The editorial team at #DisruptionBanking has taken all precautions to ensure that no persons or organizations have been adversely affected or offered any sort of financial advice in this article. This article is most definitely not financial advice.
#CapitalMarkets #MachineLearning #AI #FinancialMarkets #Fintech
See Also:
Python: the First Language of Finance? | Disruption Banking
The AI5: The New Powerhouse Redefining the AI Era | Disruption Banking
