This post could leave you feeling strange, bewildered, or skeptical, as it does me. I know I’m outside of my comfort zone.
Tim Urban’s posts on AI
A reader sent me a series of Tim Urban papers on the coming Artificial Intelligence revolution. Reading the two-part series (Part I, and Part II) is one of those mind-bending experiences that people otherwise achieve by paying a lot of money, traveling to Bali, and using exotic mild-altering substances.
As the Keanu Reeves character once said in the movie Speed: “Whoa.”
And sure, there’s the economic impact to consider from Artificial Intelligence, which we can somewhat easily imagine. But then there’s the impact beyond that, the part that leaves me feeling shaky.
In myriad industries – data management, financial trading, the military, manufacturing, robotics, and gaming to name a few – artificial intelligence experts are training computers in intelligent functions. The economic impact is massive and promises accelerating gains.
Of course we use artificial intelligence already in everyday life in ways unimagined just ten years prior, before the widespread use of iPhone-type devices and apps. My mapping app tells me the best route to drive my kids to school. Siri responds to my voice to tell me who won the World Series in 1918. (If you need Siri to answer that last one, sigh, you know nothing.)
In a paper resulting from a National Bureau of Economic Research conference in September 2017, “Artificial Intelligence and Economic Growth” economists attempt to grapple with the Artificial Intelligence revolution. Is it something that will offer incremental improvements in automation, with the analogy that we move up the automation foodchain, like from the plow to the tractor to the combine-harvester in the 19thand 20thCenturies?
Or does AI instead change the rate at which we innovate, such that economic development occurs at an accelerating pace? Change in this second case happens exponentially – like we see in graphs of population explosions, or the spread of infectious diseases. If AI helps automate physical production – or even more compellingly – automate idea production, then an exponential economic growth explosion due to AI seems increasingly plausible.
The economics paper argues that one big effect will be economic gains to people who control capital versus people who provide labor. That makes intuitive sense since increased automation favors the investor over the worker. From a financial perspective, the innovators who achieve the biggest breakthroughs in AI will likely reap huge rewards. So, plausible long-term effects of AI include accelerating economic growth and accelerating inequality. Which, okay, sounds like a path we’re already on.
But the most intriguing part of Tim Urban’s papers has to do with the question of whether artificial intelligence advances so quickly that we achieve the Singularity? And if so, when? And then what?
The Singularity
The Singularity, if you’re not a sci-fi or artificial intelligence nerd, refers to that future moment when machine or artificial intelligence gains enough self-learning momentum that it becomes “unbound,” no longer limited to human input or human-level intelligence to advance. It simply surpasses us and keeps on gaining intelligence.
If that happens, we don’t have a good way of understanding the economic (not to mention moral, environmental, or existential) implications. On the positive side, a super-intelligent machine could solve problems that humans simply can’t solve. On the negative side, what guarantee do humans have that a super-intelligent machine wouldn’t disregard our humanity, in the pursuit of solving whatever task it’s working on? The “turn the computer off and turn it back on” hack that the IT department always tells us to do won’t work when the super-intelligent machine knows that trick in advance.
Here’s a bewildering and unsettling thought, as suggested by Tim Urban, who himself builds on the work of other futurists and data scientists. The Singularity may happen in my lifetime.
The “in my lifetime” statement is a crazy-sounding claim, except that it’s backed by a sort of rough consensus belief among computer scientists, economists, and experts on the rapidly observable change in computing power and artificial intelligence technology over recent decades.
Normally I don’t go in for this kind of fuzzy futurism, except for one thing. AI advances are driven by the mathematics of exponential growth. And the mathematics of exponential growth leads us to previously unexpected results. Bear with me here for a moment on an analogy from finance, where I am more comfortable.
Things that Compound Grow
A guiding principle of this blog, my book on personal finance, and everything I try to teach about finance is that the exponential growth of invested money leads to an unexpectedly wealthy future. Since exponential growth occurs in technology, artificial intelligence, and other areas of economic growth as well, I have to be open to the unexpected future there as well.
AI, like populations and money, responds to exponential growth math
Machine computation speeds have increased by a factor of 10 raised to the power of 11 (10 with 11 zeros after it) since World War II, comparing the giant vacuum tube machine of 1943 to today’s fastest computer. Most lay people know a popular version of what’s known as Moore’s Law, which observes that computing power doubles every 18 months. Miniaturization, memory storage, and costs all improve in similarly exponential ways. Continuing our compounding improvements in AI, futurists estimate that computers are on pace to match total human intelligence somewhere between 2045 and 2080. From there, rapid advances would likely far surpass human intellectual capacity.
As Tim Urban concludes in his paper, the big issue in artificial intelligence really isn’t economic gains.
Will It Be A Nice God?
Nor is the big issue whether China or some other country is getting ahead of the United States in research into AI. That last issue is kind of like worrying about who will finally capture the Iron throne when Winter is Coming.
No. Rather, if Urban’s summary understanding of our best minds on AI are right – if you give some credence to the Singularity possibly in our lifetime – then the real question Urban asks us to ponder about the Singularity is:
Apparantly this is me in D&D, except I think minus the pony tail
I’m trying to figure out where electronic and high frequency trading firms fit with respect to Alignment, and how I feel about that.
Alignment
I mean “Alignment” the way a Dungeons and Dragons (D&D) player means it (full disclosure: I’m a Lawful Good Half-Elf Bard in real life. I mean, in the game. I mean, outside of my writing. Whatever, you know what I mean.) All D&D characters are either Good, Neutral, or Evil, and act either Lawfully, Chaotically (law-breaking), or Neutrally (in between).
So, on our quest to understand electronic trading, it is helpful to know which alignments electronic traders fall under, including high frequency traders (HFTs)?
But First: A Definition
Instead of making human judgments about when and how much to trade stocks (or bonds, or currencies, or commodities or derivatives) electronic traders program computers to make those decisions, usually based on some set of conditions that indicate a momentarily profitable opportunity. Electronic, or ‘algorithmic’ trading, is about 25 to 30 years old. High frequency trading is a subset of electronic trading, except done at a humanly unimaginable pace – like 1,000 to 10,000 buy or sell orders per second. High frequency trading is about 10 to 15 years old. In recent years electronic trading accounts for between 50-75 percent of all stock market trading. Like SkyNet or Hal 9000, this naturally makes the humans nervous. But are they good or evil or neutral? [1]
They slyly dropped “Don’t Be Evil” from their corporate motto
Are HFTs Chaotic Evil?
If you read Michael Lewis’ 2014 book Flash Boys the most widely read story about the high frequency trading industry to date – you would develop the strong impression that these firms hew to chaotic evil on the D&D alignment compass.
In Lewis’ story, HFTs operate as predatory sharks attracting unwitting investors inside broker-sponsored ‘dark pools,’ all the better to extract trading profits through quick-strike trading against slow-footed prey. These evil creatures also use ‘spoofing’ subterfuge and aggressive ‘front running’ tactics. I expand on these tactics below.
‘Spoofing’ – in which electronic trading firms send large numbers of false orders to market exchanges, only to cancel them immediately, is a ploy (I admit I can’t explain in plain English exactly how this would work) to manipulate markets, and is clearly chaotic. It’s also illegal, and would lead to enforcement action against any firm doing this and getting caught.[2]
‘Front-running’ – in which an electronic trading firm uses prior knowledge about a customer order to buy or sell ahead of a customer for its own profit is also evil, as well as clearly illegal.[3]
And Flash Crashes
Many blame recent occurrences of “flash crashes” on algorithmic trading. Flash crashes are exactly the kind of mess that chaotic evil-doers would wish on markets.
Increasing the frequency or severity of flash crashes is the most likely way in which electronic trading causes chaotic evil effects. I don’t mean intentionally, but rather as an unintended consequence of numerous market players pursuing their own strategy. Something like: All market signals indicate to the algorithms the need to sell – all at the same time – which becomes a self-fulfilling downward spiral for prices. That type of unintended effect, however, predates the rise of HFTs. The 1987 Crash, for example, stemmed from the rise of ‘portfolio insurance’ that caused many institutions to suddenly need to sell securities, all at the same time, to limit losses. In the absence of real news, prices drop on such rush-for-the-exits stampedes.
On the issue of crashes and market glitches, there’s the not-too-infrequent case of human traders – not only computer traders – doing a bad job of ensuring orderly markets. This happened in August in a high-profile case of the floor trader on the NYSE who halved the value of publically traded KKR, a company whose markets he was responsible for trading, for about 15 minutes, for no apparent reason. The right standard for comparing human trader to computer trader is probably not “error-free,” but rather frequency and severity of mistakes and glitches like this. My point here is that human traders can probably screw up markets just as badly as programmed computers.
Or Lawful Good?
My friend Peter Kovac wrote a book last year – Flash Boys: Not So Fast, as a response to Michael Lewis’ book, in which he argues not only that Lewis got many details of the industry wrong, but perhaps the HFTs should be regarded instead as something like Lawful Good (my words, not Kovac’s.)
I’ll explore some of Kovac’s reasoning in follow-up posts, but for the moment I have in mind what I wish, and perhaps think to be true, regarding electronic traders.
Lawful Neutral
As a Dungeons and Dragons player (as well as a greedy capitalist,) I would hope for Lawful Neutral alignment among high-speed electronic traders. I mean, I don’t expect a trader to be saving the whales or reducing carbon emissions when he or she programs a computer algorithm to buy and sell securities at light speed. Their goals, as for-profit companies, are to make a profit. But I do expect them to always follow the law.
What Lawful Neutral means to me is that as long as they follow the rules – avoid conscious or even unintended evil-doing – then I’m ok with extraordinary profits accruing to electronic traders. That’s because I believe the profits of an algorithmic trading firm will mostly come at the expense of legacy Wall Street trading firms (the “old guard”) which are slower, or which operate at less efficient (meaning, wider) margins. I’ll write more about this next week as well.
[1] Assigning corporate alignments in D&D fashion is not necessarily new. Google’s previous corporate motto “Don’t Be Evil” is a seemingly simple standard, for example, from which to begin to evaluate high frequency trading. I’m not sure Google founders Brin and Page ever played D&D, but let’s just say it’s not unlikely that they know their way around a 20-sided die, right? Also, did anybody else notice Google dropped “Don’t Be Evil” as a corporate motto? Do you think it means what I think it means?
[2] My sense is that while this has happened in the past, it’s not normal market practice among electronic trading firms, any more than spamming is normal market practice among marketing companies. Sort of like: there are spammers, and there are marketers, but these are different types of firms with different business models
[3] Incidentally, front running as a business practice is probably as old as any stock brokerage business, it just happens that speedy trading could make it more easily perpetrated, and more easily hidden, at least for a time. Any firm shown to front-run as a business practice, however, would be fined and regulated out of the trading business.
I read Rishi Narang’s Inside the Black Box as a kind of primer on quantitative trading, in advance of reading Michael Lewis’ Flash Boys.
Need for a primer on quantitative trading
I need a primer before reading Flash Boys for several reasons.
I was an ‘over-the-counter’ institutional bond guy, rather than a participant in the exchange-traded stock markets that form the core of many quantitative trading strategies. In addition, my Wall Street experience is a decade old at this point, (I left Goldman in 2004) and the development of dark pool trading, high-frequency trading (HFT), and alternate electronic exchange trading has proliferated since then.
Whereas the traditional stock exchanges with real-live human specialists (like the New York Stock Exchange – NYSE) was already in decline by the time I left Wall Street, the vast majority of stock-trading volumes now happen electronically on alternate exchanges and dark pools away from the traditional NYSE.
Finally, I have a few friends in the quantitative trading world and one of them reached out to me shortly after the publication of Lewis’ book and said there are laughably wrong statements and implications in Flash Boys. In the interest of having an intelligent dialogue with him, to figure out if Lewis really screwed up, I needed some background.
Different Audience for the Books
Narang’s audience for his book differs from Michael Lewis’ audience for Flash Boys. We rely on Lewis for funny anecdotes and characters, and enough financial sense to give the general public an accurate, entertaining, and non-technical narrative.
By contrast Narang’s book targets non-expert practitioners who need more quantitative trading knowledge such as: new hires into the quantitative trading world – or current and prospective investors/asset allocators to quant-trading funds. Narang currently is an investor/allocator to quantitative strategies, who previously founded, and ran, a quantitative trading firm directly.
Narang provides a simplified ‘how to understand’ or ‘how to evaluate’ quantitative trading firms, for readers without specialized trading and computer programming background.
When I say ‘simplified’ I don’t mean the book will be ‘simple’ for people outside finance. Narang’s language is clear and precise, even full of useful analogies for explaining concepts, but it’s academic in style. He systematically reviews, engineer-like, the common components and varieties of quantitative strategies.
I learned a ton, but folks without a need to know about quant trading will not rip through this like they would a Michael Lewis book on finance.
I had a number of questions about quant trading before I read Inside the Black Box and Narang sheds light on my questions.
What is quantitative trading and what should we know about it?
Quant trading, in its simplest definition, can be distinguished from discretionary trading by the role of human decision-making. In traditional, discretionary, investing we decide to sell or buy a financial instrument for any number of good or bad reasons. With quant trading, by contrast all of the human inputs occur ahead of time, that is to say, through the programming of a trading algorithm to look for and respond to the optimal opportunities to trade. The actual order to sell or buy occurs in response to the signals, rather than any in-the-moment human decision.
How big is quant trading compared to discretionary trading in the markets?
Narang cites a study from 2009 that indicated 60% of trading volume in US equities came from quantitative trading, as well as 45% of trading volume in European exchanges. So if 6 billion US shares changed hands per day, then Hal9000 algorithm-based trades accounted for 3.6 Billion of those, leaving only 2.4 Billion shares traded by human discretion. In addition, over 90% of commodity trading firms (CTAs) are quant-trading firms, rather than discretionary traders.[1]
So how do programmers and traders set up the trading algorithm to order trades ahead of time?
Naturally, the next question is about how this programming gets done, and answering this question forms the bulk of Inside the Black Box.
Narang comprehensively reviews and explains both the common elements of building a quant trading strategy, as well as the wide variety of approaches used by the entire spectrum of quant trading firms.
Typically, programming a successful trading algorithm depends on reviewing extraordinary amounts of historical data on observed trading responses of financial instruments, and then seeking patterns among the noise. If a quant trader can find predictable enough responses, then it becomes possible to test if trading on an expectation that the pattern continues could be profitable over time. If his algorithm recognizes the pattern well enough, he can, in effect, ‘see the future’ enough to trade predictably and profitably.
So, it’s pattern-seeking?
A few simple examples of patterns may be helpful here to give an introductory idea. Some stocks – often from the same industry – predictably track each other’s direction, such that a directional move from one typically will be mirrored by a directional move in the other. A similar pattern could hold for a single stock and a ‘basket’ of similar enough stocks.
Volume data on stocks – how much is being traded over any given time period – may predictably be followed by moves in the stock or financial instrument.
Some directional trends in financial instruments could indicate a predictable pattern of a continuation of that trend, or just as likely (and somewhat paradoxically) the likely reversal of a trend.
The common denominator in this type of trading, however, posit that past trading data indicates somehow what will happen in the near future.
To the extent that markets behave in patterned ways – and in fact most of the time financial markets do behave in recognizable historical patterns, or at least recognizable to quants and their computers – quant traders can make money with trading algorithms. This doesn’t have to happen every time, with every trade or every day. In fact, quant strategies depend not so much on winning on every trade, but winning on enough trades to overcome the losses on other trades, as well as the costs, or ‘trading friction,’ of executing many trades simultaneously.
Some of the time, of course, market regimes may shift and past patterns become poor indicators of future moves. At that point, the human-element decision becomes whether to lower the trading exposure based on the algorithm, to shut down the algorithm altogether, or to reprogram the algorithm in response to updated market conditions and patterns.
My sense from reading Narang’s book, as well as from talking to friends in the business, is that quant traders must engage in constant vigilance around their algorithms: Algorithms may cease to generate positive returns as competition eliminates the opportunity, or competitors’ speed eliminates the previous advantage, or patterns simply stop working.
Research and development to find new ‘signals’ inside the noise of past trading data requires constant work.
Summer Glau, my favorite terminator ever
Besides pattern-seeking and constant R & D, what are other elements common to the wide variety of quant traders?
DATA – The acquisition of historical data, the scrubbing of data to extract useful information, and the testing of hypotheses (will following that pattern actually make money?) using the historical data are common to all quants.
COSTS – Some strategies require thousands or even millions of trades, so costs associated with trading matter a lot. Quants have to include in their model expected trading costs – not only commissions paid to brokers or exchange – but the ‘slippage’ that occurs due to trading. The scientific method analogy to this ‘slippage’ here would be the “Hawthorne Effect” or “Observer Effect,” meaning it’s difficult-to-impossible to collect ‘profit’ from the market without affecting the market opportunity itself.
When a quant finds a financial instrument momentarily ‘cheap’ according to some historical pattern, the act of purchasing the instrument to take advantage of the cheapness may cause the price to rise.
The larger the trade that the quant algorithm demands to take advantage of that momentary cheapness, the larger the ‘slippage’ due to the trade itself will actually eliminate the opportunity. Any historical model that fails to take into account this cost element will likely fail to work.
SCIENTIFIC METHOD – Quant-trading, more than discretionary trading, depends on a ‘scientific method’ approach to investing. While a discretionary trader may depend on fuzzy thinking and ‘gut feeling’ to guide his buy and sell signals, quant trading eliminates this part of trading in favor of a rigorous Hypothesis à Test à Hypothesis à Test continuous feedback loop, with consistent profitability (in trading-speak, the generation of “Alpha,” or excess return over risk) the primary indicator of whether the hypothesis has any validity in real markets. Like scientific hypotheses, the trading algorithm can never become ‘proven’ or ‘permanent law.’ In fact, many current predictors of market moves may be shown to be poor predictors in the future when markets change. A continuous testing of hypotheses about patterns through the scientific method helps distinguish quants from discretionary traders.
How different is High Frequency Trading (HFT) from other types of quant trading, and what should we know about the subset of quant trading known as HFT?
Narang devotes the final chapters of this book to explaining HFT. Narang suggests that the trading ideas of HFTs can often be much simpler than other quant trading – but depend on faster execution of the simpler idea.
Narang provides a relatively easy illustration of a typical, simple, HFT idea.
The S&P500 index consists of shares from the 500 largest market-capitalization US companies, in known relative weightings. Naturally whenever the price of an individual stock in the index changes, the value of the S&P 500 index as a whole changes slightly as well. An exchange traded fund (ETF) like the SPY, or an S&P500 futures contract (ES), both of which are meant to track the index precisely, should exactly mirror the price changes in the underlying stocks.
If, however, one of the underlying stocks goes up in value and the SPY value does not change precisely, instantaneously, to a new value based on the moves in the underlying stocks, a super-fast arbitrage can be had by buying the SPY and shorting the stock. These arbitrage opportunities are either rare or exist for very brief instances – all sorts of traders would look for precisely this simple risk-less opportunity – but trading speed makes exploiting a price difference like this possible.
Keanu: “Whoa”
The best way to explain this is that only a computer programmed to move as fast as Neo in that scene dodging Mr. Smith’s bullets, could hope to profit from the S&P/ SPY arbitrages. Nobody else has a chance.
The basic trading idea here is super-simple. On the other hand the technology involved in calculating the precise price discrepancies, in the right weightings, based on the right data feeds, and doing it faster than anyone else in the marketplace, is not easy.
The S&P 500 trade described above is probably version 1.0 of HFT trading, and most HFT firms seek out similar price opportunities with fewer competitors and less efficient instruments in myriad markets around the world.
HFTs may seek to use more complicated algorithms than this example, but at a certain point the computing time required to sift through the market signal may slow down the process, so that they begin to resemble other quant traders.
By keeping the trading idea dumb and simple, the HFTs retain the advantage that comes from technology. HFT is the Keanu Reeves of quant trading, dumb and dependent on Speed to keep the bus above 88 mph to prevent everything from exploding.
Conventional quants may have harder-to-explain patterns that they recognize and exploit, but probably do not depend on execution speed to the same extent as HFTs in order to be profitable.[2]
How likely, and with what severity, could we experience a destructive flash-crash catastrophe driven by the present and future take-over of markets by quant trading? In other words, should we the public (and regulators) be much more actively regulating SkyNet?
Narang addresses a version of this question in Chapter 10, “Risks Inherent to Quant Strategies,” although his goal is limited to showing – persuasively in my opinion – that large market blow-ups in the past cannot be blamed on quants.
The Long Term Capital Management (LTCM) blowup of September 1998, for example, may be described by some as a quant-fund disaster that fundamentally shook capital markets, and might have gotten worse without a Federal Reserve-Inspired ‘Bail-in’ of LTCM by Wall Street firms.
Narang points out, however, that while LTCM used sophisticated quantitative models in their trading, their strategy remained fundamentally discretionary – meaning humans made the calls on what went into and out of their portfolio.
The 2008 Mortgage-bond inspired crisis, in addition, cannot reasonably be linked to quant trading firms. While the creation and trading of highly-leveraged mortgage and default-swap CDOs involves sophisticated mathematics and modeling, the billions in losses owed more to a combination of leverage (debt), human error, poor risk-modeling, the connectivity of markets and counter-parts, and misunderstood incentives among disparate parts of the mortgage-bond origination chain than anything to do with quantitative trading – as explained well in Michael Lewis’ The Big Short.
Arguably, the most significant systemic market risk crash to date that could be blamed on quant-like trading is the October 1987 market crash, in which computer-based “portfolio-insurance” algorithms each triggered a cascade of automatic selling, independent of human judgment. The Crash of 1987 occurred prior to the rise of quant trading as we know it, so ‘blaming’ quant trading can really be done only by historic analogy. It’s the kind of crash we could imagine from algorithmic trading in which the size of computer trading orders, added to the overwhelming one-sidedness of the computer signal (a crowded trade) causes the devastation.
The actual quant trading blowups to date, as Narang explains, have caused localized losses without inflicting much observable or systemic market disruption.
During a few weeks in August 2007, for example, a majority of algorithmic traders simultaneously experienced sudden losses, as the trends suggested by historic patterns of trading relationships reversed. It turns out that the similarities in quantitative strategies at that time meant that as some quants lost money and unwound their portfolios they caused a cascade of losses in other quants’ positions.
In the meantime, the non-quantitative investment community – people who just own stocks or mutual funds, or non-quant hedge funds – barely noticed any particular change in market prices.
The market as a whole neither soared nor crashed. Only the historic patterns of short-term trading relationships broke down and then suddenly and severely reversed. As a silent market-crisis, only experienced by the quant community, August 2007 may be a harbinger of the kind of localized pain that non-quants can safely ignore. Or conversely, the August 2007 crisis indicates the growing power and ‘crowded trade’ nature of quant trading that regulators should worry about which could freeze up or destroy orderly markets in the future.
Other flash crashes, such as the May 2010 Flash Crash or other mini-crises such as happened in August 2012 that brought down Knight Trading, may fit the pattern of acceptable, localized, market blips. Optimistically, programming errors like those are just a Darwinian way of punishing the inept and incentivizing the careful and level-headed to take advantage of others’ mistakes.
Or, they could be the early tremors in market-structure fault-lines created by quant trading that will eventually cause an irreversible earthquake in world markets. I don’t want to be alarmist on a topic where I’m an amateur, but I hope some regulators have taken on the ‘seismic geologist’ role of figuring out how to predict and prevent ‘The Big One’ caused by quant trading, before it happens.
I mean, we’ve all seen 2001 Space Odyssey, Tron, Terminator, and The Matrix, right, so we know this doesn’t bode well for algorithmic trading. Even Aliens and Planet of the Apes are basically warnings about quant trading, aren’t they? Wait, am I overthinking this?
Open the pod bay doors, please, Hal
Narang does not particularly address this much more troubling, forward-looking question, of whether and to what extent quants will significantly blow up markets in the future. His primary response in Chapter 16 to this worry or criticism points out that human errors have more frequently caused losses or mini-crashes in the markets than computer glitches. No doubt this is true, but that’s a bit like saying we shouldn’t worry about tornadoes because hurricanes have caused more damage up until now.
As we learned from the 2008 Crisis, markets have a way to experiencing the near-statistically impossible, stunningly unlikely, “6-sigma” event, or “1,000-year flood” just about every decade or two, causing massive wealth destruction in new and unpredictable ways.
Often we can see, in retrospect, that the root cause of a market crash has something to do with the rapid advance of a profitable financial technology far ahead of our ability to control all the risks of the financial technology or regulate the inequities or societal costs of the financial technology.
I’m not in a great position to evaluate the probability and consequences of a future algorithmic-trading driven market catastrophe, and perhaps Narang is not either, although he’s certainly better positioned than most.
I do remain worried about a future SkyNet-driven Crash.
Is quantitative trading in general, or HFT in particular (as the rapidest version of quantitative trading is known) inherently unfair, unethical, or taking advantage of other market participants?
I have not read Flash Boys yet, but I know enough about the plot line to see Michael Lewis’ book as a call to action about the inequities inherent in HFT, a subset of quantitative trading. Whether he’s right or not, regulators are already responding to their perception that HFTs are gaming the system in a way that’s harmful to markets overall.
Narang clearly feels that quantitative trading is an interesting and honorable way to make a living. In the final section of the book, he describes the methods and implications of High Frequency Trading, with noted contributions from his brother Manoj Narang, the CEO of a HFT firm Tradeworx and sometime commenter on the industry.
Given Narang’s experience and connection with the industry, we would not expect him to point a finger at quants or HFTs and declare them inherently unfair. What he does try to do instead is educate the non-expert in how quants may make money, and why HFTs in particular feel the need for speed.
To give one example of this need for speed, his brother’s firm Tradeworx researched the profit difference between placing purchase or sale orders first, versus placing them later in the queue for order filling. Inherent to equity trading, he explains, is the rule that when buy or sell orders get placed on an exchange at the same price, the first order placed gets filled first. The value of being first with certain types of orders, according to Tradeworx, turns out to be approximately 1.7 cents per share.
With other kinds of orders, they have found that immediate, speedy, fulfillment of the trade reduces the cost or ‘slippage’ due to trading. In a strategy dependent on a massive amount of trading to capture momentarily fleeting price reactions, the high speed fulfillment of orders matters tremendously.
How do HFTs get so fast?
In the same chapter, Narang mentions the advantage of locating HFT trading servers at, or near, market exchange servers, a major theme of Flash Boys.
Along a similar vein, Narang reports the lengths and costs endured by trading firms to reduce data transmission times between major trading centers such as New York, Chicago, and London. HFT trading firms will sign long-term, multi-million dollar contracts for fiber optic access in order to shave off five milliseconds (there are 1,000 milliseconds in every second, so five milliseconds is 5 thousandths of a second!) in data transmission times between New York and London, as an indication of the importance of being the first to post, or first to cancel, transaction orders.[3]
HFT firms, Narang points out, monitor data and order trades in tenths of a millisecond, creating myriad technical challenges, between hardware, software and network engineering.
Another speed bump in the way of the highest frequency trading, Narang notes, has been the regulatory rule adopted in July 2011 that US equity brokers must enact a ‘risk check’ for every trade, in order to ensure that the size and capacity of the customer and the trade are valid – meaning not too big, which might indicate a trade made in error.
The brokerage software required to do this risk check slows down every trade, which could dampen HFTs speed by 0.5 milliseconds. In response, some HFT created their own brokerages, in order to reduce the time-lag of using someone else’s risk check system.
Where do I differ from Narang’s perspective?
1. This is a bit nitpicky, but I have a philosophical issue with Narang’s definition of ‘value’ and the role of ‘timing’ in investing. He argues in both Chapter 3 and Chapter 16 that all investors, all the way from HFTs on the one side of the spectrum to long-term investors such as Warren Buffett on the other side of the spectrum, depend on timing to beat the market. He means that investors have to execute their purchase (or sale) in advance of others’ execution, and it’s this earlier-than-others’ action that creates value.
In Narang’s presentation, the good investor buys first, others follow suit later, and the original investor can then sell at a higher price based on the fact that others have driven up the price of the instrument.
Here I think Narang’s missed the point of value-investing as advocated by Buffett: Value is created by purchasing future cash-flows at an attractive price, and value investors are mostly indifferent to other investors’ subsequent actions. If you conceive of stock ownership with a perpetual time horizon, as a value investor like Buffett seems to, and that value derives from those future cash flows purchased at a good price, then timing falls out nearly completely as a consideration.
With a quant’s focus on short-term price changes rather than the cash-flow potential of financial instruments, I can see why Narang views timing as a key to all investing, but I think that focus leads to inaccurately describing other types of long-term investing.
2. I cannot tell from what I read and really this book doesn’t say – whether a constant stream of high frequency orders and cancelled orders constitutes a confusing ‘spamming’ of equity market exchanges, to the detriment of slower investors. Rapid fire orders, the non-quant critics say, create the illusion of market liquidity where it doesn’t really exist, as it may disappear quickly when non-quants actually seek to trade.
I can understand the way in which HFTs seeking arbitrage opportunities between linked instruments (like the S&P500 and the SPY) create liquidity and efficiency to some extent, in a way that may benefit the smooth running of markets.
In addition, Narang claims that rapid order cancellation is a necessary component of HFT strategies[4] and he claims that no evidence supports the complaints from non-quants about damage done by this type of activity.
But do millisecond orders and cancelled orders jam up the orderly execution of others? Does algorithmic swarming of orders around non-quant investment flow hurt the non-quants in a fundamentally unhelpful or unfair way?
Clearly some non-quants traders and investors believe they are harmed by excessive order-making and order-cancellation, when it comes to trading. Are the non-quants being overly paranoid or jealous of HFT profits, or is there actual harm done? Basically I need a non-quant trader’s opinion here, and I don’t think Narang sufficiently addresses this type of externality of HFT activity.
3. I’ve already mentioned that his book does not dwell on the potential for future catastrophic losses, nor did I expect it to, as the future is unpredictable and Narang would be engaging in speculation about the unknown. On the other hand, we’ve had enough mini-shocks in the market attributable to algorithmic trading that I hope someone, somewhere (ideally in a government regulatory office) is thinking deeply about ways to prevent The Big One from happening.
[1] People, what I’m trying to say is: SkyNet already took over the commodities and equity markets. Good thing there’s some real live ex-bankers who still write finance blogs. Ha ha. Or maybe that’s just what I want you to think.
[2] Maybe in this analogy traditional quant trading is a more complex George Clooney-character? I don’t know, still working on it.
[3] I don’t generally begrudge hedge fund traders anything, but now I’m starting to think they never have to wait those annoying 10 seconds for Game of Thrones to load up on Amazon Prime, like I typically do. Damn, I hate those guys. If I was a well-compensated hedge fund trader right now I would happily pay 12 million dollars just to eliminate that delay.
[4] to avoid a problem he explains well, which is the ‘negative selection’ of passive orders on an exchange.
SkyNet made its mark on US stocks again this morning at the market open, as about a dozen companies saw their shares jump or plummet wildly in the opening minutes on the New York Stock Exchange, on exactly zero news, most likely due to computer-driven orders, and with no human input.
Outside of the specialized world of stock trading, many retail investors probably do not know that an estimated 80% of US stock trading volume is based on computer-programmed buy and sell orders. In other words, 4 out of every 5 dollars flowing through stock exchanges has no human decision-maker behind it. Stocks trade only a programmed response to preset algorithms. At this point, SkyNet completely dominates stocks.
How did this come about? High Frequency Trading firms (HFTs) seek to profit from predictable responses of some shares to the movement of other shares and market inputs, but HFT computers need to execute trading orders in fractions of seconds in order to benefit from their ability to see the immediate predictable future. Since humans cannot act fast enough, the machines have been programmed to trade in their place.
When things in stocks get really screwy, as they did this morning, and as they did most dramatically during the 2010 Flash Crash, we’re reminded that the humans are no longer in charge.
Lobbyists for the HFT firms have already come back from the future and are visiting the offices of our Congressman one by one to eliminate any Sarah or John Conners that seek to regulate HFT’s role in US stock markets.
In myriad industries – data management, financial trading, the military, manufacturing, robotics, and gaming to name a few – artificial intelligence experts are training computers in intelligent functions. The economic impact is massive and promises accelerating gains.
The economics paper argues that one big effect will be economic gains to people who control capital versus people who provide labor. That makes intuitive sense since increased automation favors the investor over the worker. From a financial perspective, the innovators who achieve the biggest breakthroughs in AI will likely reap huge rewards. So, plausible long-term effects of AI include accelerating economic growth and accelerating inequality. Which, okay, sounds like a path we’re already on.
Normally I don’t go in for this kind of fuzzy futurism, except for one thing. AI advances are driven by the mathematics of exponential growth. And the mathematics of exponential growth leads us to previously unexpected results. Bear with me here for a moment on an analogy from finance, where I am more comfortable.
Many blame recent occurrences of “flash crashes” on algorithmic trading. Flash crashes are exactly the kind of mess that chaotic evil-doers would wish on markets.
I’ll explore some of Kovac’s reasoning in follow-up posts, but for the moment I have in mind what I wish, and perhaps think to be true, regarding electronic traders.
I read Rishi Narang’s 
SkyNet made its mark on US stocks again this morning at the market open, as about a dozen companies saw their