The trouble began about nine miles northwest of the New York Stock Exchange, in a squat, unremarkable building.
There, inside a critical data node in the U.S. Financial system, a crucial piece of hardware failed. Like a downed electrical line in some faraway corner of the power grid, that breakdown effectively turned the lights out for a swath of investors across the nation. Irate traders woke up to discover they’d been locked out of their accounts at one of the nation’s leading electronic brokers, Interactive Brokers Group.
Monday’s events—from the failure across the Hudson River, in a windswept stretch of New Jersey, to Interactive Brokers’ desperate race to identify and fix the problem—underscores the new and growing strains that trading systems are experiencing as individuals jump into the markets as never before.
The changes wrought by the COVID-19 pandemic have fueled a surge of retail trading, which now comprises 20% of U.S. Stock orders—an increase of 5 percentage points in the $42 trillion market in just a year, according to Bloomberg Intelligence research.
Interactive Brokers and rivals including Charles Schwab and Morgan Stanley’s E*Trade have set records this year for customer accounts and activity as day traders, many of them stuck at home with few distractions, have ridden the more than 60% surge since March’s lows. But that has brought greater focus on how the companies operate and serve their growing legions of customers.
“As retail brokerages become a more significant aspect of this business, the quality of a retail broker’s tech becomes more important,” said Larry Tabb, an analyst at Bloomberg Intelligence. “When they’re down, they frustrate a wider array of traders, who have very little patience when they want to trade.”
When problems occur, the feedback is swift. Websites such as Downdetector.com allow any brokerage customer to report problems with their trading platform and chart how many are reported. Customers also broadcast their frustration—and, in some cases, plans to switch brokerages—on their Twitter or Facebook feeds.
As if to emphasize the new environment for 42-year-old Interactive Brokers, customers bombarded it with complaints Monday on Twitter and threatened to close their accounts.
“The system was specifically designed to minimize the likelihood of the kind of technology failure that we experienced,” said CEO Milan Galik, who apologized for the disruption in a note to clients. “It did not work as expected today.”
While the outage was far from the first for a retail brokerage, the fallout was more widespread because of this year’s successes. Interactive Brokers had more than 1 million accounts at the end of November, a 52% increase from a year earlier, and hit a record 1.8 million daily average trades in the third quarter, more than double the same period in 2019.
Steve Sanders, the firm’s executive vice president of marketing and product development, said Tuesday that all systems were “back to normal.”
Monday’s issue was caused by a hardware malfunction in a Secaucus, New Jersey-based data center called Equinix NY5, according to a person familiar with the matter. Part of a nondescript campus of warehouses that are home to the plumbing of the modern financial system—as well as servers for social media, telecoms and streaming companies—the site is the unglamorous side of today’s stock trading.
It’s the true backbone of Wall Street and, like any technology, is vulnerable to the occasional breakdown. Stock exchanges, for example, experience similar issues, though backups and the fact that there are more than a dozen bourses usually prevents any market disruption.
But for non-professionals who’ve come to expect instantaneous trades from their computers or phones, outages are a shock, and a potentially expensive one.
“Talk about infuriating,” said user Dale O. Hays, who was unable to access Interactive Brokers’ trading systems at about 11 a.m. New York time on Monday. Hays said the system didn’t restart until around 3 p.m., and even then it went down again three more times.
“They seem to have the mentality of not being proactive, not communicating the true status—as noted by their various emails, claims and ‘status boxes’—and letting their clients blow in the wind,” he said.
Interactive Brokers isn’t alone in facing new, and newly irate, customers. Robinhood Markets, the app-based broker that has perhaps benefited more than anyone else from 2020’s retail surge, has been criticized for its lack of assistance when things go wrong.
“We’ve been seeing outages from Robinhood to TD Ameritrade to Schwab,” Tabb said. “They lose clients when this happens. Clients get upset, and if problems persist they go somewhere else.”
