By University Communications
Using a complex text mining system — which scans and analyzes huge quantities of content — McClelland Professor of MIS Hsinchun Chen has created a program that mimics the way Wall Street analysts predict stock performance.
In simulated trading, the program outperformed the market average and performed well against existing quantitative funds (quants), which analyze numeric data to issue stock predictions.
Unlike quants, Chen’s Arizona Financial Text system, or AZFinText, works by absorbing a high volume of financial news along with by-the-minute stock price data. “Our approach is to emulate what an analyst does,” said Chen. “Great analysts read papers and hone in on clues that others have not observed.” Chen’s paper is the first published work to demonstrate a promising textual analysis program.
The program analyzes the financial news and then buys or shorts every stock that it predicts will move more than one percent in 20 minutes. “There are many variables in long-term predictions,” Chen said. “But the system has an advantage if it’s just looking at the market in a five- or ten-minute window.”
Initial attempts to automate text analysis of financial information began in the 1990s, but failed to take off due to poor performance and the technical limitations presented by the need to swiftly compute huge numbers of articles, press releases, and reports. Chen and co-author Robert Schumaker of Iona College in New York overcame these challenges by developing an architecture of word categories that directs the program’s analysis.
Going head to head on an individual article, Chen said that “The system won’t be as accurate as an individual analyst. The computer is maybe 80 to 85 percent accurate when analyzing text.” But the advantage? “The computer can read maybe 100,000 times the amount of data that a single analyst can.”
Their initial results, which looked at data from a five-week period in the fall of 2005, were published in IEEE Computer Society’s Computer magazine. AZFinText had an 8.5 percent return on trades, beating the S&P and six of ten quants.
Since the publication of their original article, a team of ten doctoral students working in Chen’sArtificial Intelligence Lab has taken up the project, significantly increasing its capacity. “We’re applying the same technology that we developed for the Dark Web project, including sentiment analysis and volume of the chatter on a given company,” he said.
While the initial project focused on news from Yahoo! Finance, they’ve expanded the media that the program tracks beyond traditional outlets such as The Wall Street Journal to include social media including Twitter feeds, blogs, and online forums. In a recent project, for example, the team focused on WalMart, synthesizing some two million messages to look at risk assessment and make stock predictions.
“The market moves very quickly now,” Chen said. “We have a window of maybe three to five years to demonstrate that this is a promising commercial product. It has the potential to make the market more efficient on a large scale.”