High Speed Trading – And DandD Alignments

Apparantly this is me in D&D, except I think minus the pony tail

I’m trying to figure out where electronic and high frequency trading firms fit with respect to Alignment, and how I feel about that.


I mean “Alignment” the way a Dungeons and Dragons (D&D) player means it (full disclosure: I’m a Lawful Good Half-Elf Bard in real life. I mean, in the game. I mean, outside of my writing. Whatever, you know what I mean.) All D&D characters are either Good, Neutral, or Evil, and act either Lawfully, Chaotically (law-breaking), or Neutrally (in between).

So, on our quest to understand electronic trading, it is helpful to know which alignments electronic traders fall under, including high frequency traders (HFTs)?

But First: A Definition

Instead of making human judgments about when and how much to trade stocks (or bonds, or currencies, or commodities or derivatives) electronic traders program computers to make those decisions, usually based on some set of conditions that indicate a momentarily profitable opportunity. Electronic, or ‘algorithmic’ trading, is about 25 to 30 years old. High frequency trading is a subset of electronic trading, except done at a humanly unimaginable pace – like 1,000 to 10,000 buy or sell orders per second. High frequency trading is about 10 to 15 years old. In recent years electronic trading accounts for between 50-75 percent of all stock market trading. Like SkyNet or Hal 9000, this naturally makes the humans nervous. But are they good or evil or neutral? [1]

They slyly dropped “Don’t Be Evil” from their corporate motto

Are HFTs Chaotic Evil?

If you read Michael Lewis’ 2014 book Flash Boys the most widely read story about the high frequency trading industry to date – you would develop the strong impression that these firms hew to chaotic evil on the D&D alignment compass.

In Lewis’ story, HFTs operate as predatory sharks attracting unwitting investors inside broker-sponsored ‘dark pools,’ all the better to extract trading profits through quick-strike trading against slow-footed prey. These evil creatures also use ‘spoofing’ subterfuge and aggressive ‘front running’ tactics. I expand on these tactics below.

‘Spoofing’ – in which electronic trading firms send large numbers of false orders to market exchanges, only to cancel them immediately, is a ploy (I admit I can’t explain in plain English exactly how this would work) to manipulate markets, and is clearly chaotic. It’s also illegal, and would lead to enforcement action against any firm doing this and getting caught.[2]

‘Front-running’ – in which an electronic trading firm uses prior knowledge about a customer order to buy or sell ahead of a customer for its own profit is also evil, as well as clearly illegal.[3]

And Flash Crashes

Many blame recent occurrences of “flash crashes” on algorithmic trading. Flash crashes are exactly the kind of mess that chaotic evil-doers would wish on markets.

Increasing the frequency or severity of flash crashes is the most likely way in which electronic trading causes chaotic evil effects. I don’t mean intentionally, but rather as an unintended consequence of numerous market players pursuing their own strategy. Something like: All market signals indicate to the algorithms the need to sell – all at the same time – which becomes a self-fulfilling downward spiral for prices. That type of unintended effect, however, predates the rise of HFTs. The 1987 Crash, for example, stemmed from the rise of ‘portfolio insurance’ that caused many institutions to suddenly need to sell securities, all at the same time, to limit losses. In the absence of real news, prices drop on such rush-for-the-exits stampedes.

On the issue of crashes and market glitches, there’s the not-too-infrequent case of human traders – not only computer traders – doing a bad job of ensuring orderly markets. This happened in August in a high-profile case of the floor trader on the NYSE who halved the value of publically traded KKR, a company whose markets he was responsible for trading, for about 15 minutes, for no apparent reason. The right standard for comparing human trader to computer trader is probably not “error-free,” but rather frequency and severity of mistakes and glitches like this. My point here is that human traders can probably screw up markets just as badly as programmed computers.

Or Lawful Good?

My friend Peter Kovac wrote a book last year – Flash Boys: Not So Fast, as a response to Michael Lewis’ book, in which he argues not only that Lewis got many details of the industry wrong, but perhaps the HFTs should be regarded instead as something like Lawful Good (my words, not Kovac’s.)

I’ll explore some of Kovac’s reasoning in follow-up posts, but for the moment I have in mind what I wish, and perhaps think to be true, regarding electronic traders.

Lawful Neutral

As a Dungeons and Dragons player (as well as a greedy capitalist,) I would hope for Lawful Neutral alignment among high-speed electronic traders. I mean, I don’t expect a trader to be saving the whales or reducing carbon emissions when he or she programs a computer algorithm to buy and sell securities at light speed. Their goals, as for-profit companies, are to make a profit. But I do expect them to always follow the law.

What Lawful Neutral means to me is that as long as they follow the rules – avoid conscious or even unintended evil-doing – then I’m ok with extraordinary profits accruing to electronic traders. That’s because I believe the profits of an algorithmic trading firm will mostly come at the expense of legacy Wall Street trading firms (the “old guard”) which are slower, or which operate at less efficient (meaning, wider) margins. I’ll write more about this next week as well.


A version of this post ran in the San Antonio Express News.

Please see related posts:

Book Review: Flash Boys by Michael Lewis

Book Review: Flash Boys Not So Fast by Peter Kovac

Book Review: Inside The Black Box, by Rishi Narang



[1] Assigning corporate alignments in D&D fashion is not necessarily new. Google’s previous corporate motto “Don’t Be Evil” is a seemingly simple standard, for example, from which to begin to evaluate high frequency trading. I’m not sure Google founders Brin and Page ever played D&D, but let’s just say it’s not unlikely that they know their way around a 20-sided die, right? Also, did anybody else notice Google dropped “Don’t Be Evil” as a corporate motto? Do you think it means what I think it means?

[2] My sense is that while this has happened in the past, it’s not normal market practice among electronic trading firms, any more than spamming is normal market practice among marketing companies. Sort of like: there are spammers, and there are marketers, but these are different types of firms with different business models

[3] Incidentally, front running as a business practice is probably as old as any stock brokerage business, it just happens that speedy trading could make it more easily perpetrated, and more easily hidden, at least for a time. Any firm shown to front-run as a business practice, however, would be fined and regulated out of the trading business.

Post read (854) times.

Excerpt From Critique of Michael Lewis’ Flash Boys

Pete Kovac, a friend who worked for a quantitative trading firm, got in touch with me soon after Michael Lewis’s Flash Boys came out last Spring to let me know he thought the book had serious errors.

My friend became so alarmed at the Michael Lewis version of quant trading – which appears to have been quickly adopted by regulators and politicians – that he set out to write a response and correction to the flawed Lewis narrative.

high frequency trading volume

What’s so interesting about Kovac’s response is that quant traders have generally shunned describing publically how they make money. As a result, they are at a huge disadvantage in telling the quantitative trading version of things when competing with a writer like Lewis. Yet, one of the weaknesses of Lewis’ book is that he appears to have had very little access to quant traders themselves. So how does a quant get his version of the truth out?

Kovac releases his book this week, and I’ve included an excerpt below.

To set up the excerpt – and for those who haven’t read Flash Boys yet – here’s a quick primer.

The premise of Flash Boys (reviewed here) is that a new set of quantitative (or algorithmic) firms emerged in the past decade that engage in an unfair technology race that allows them to front-run other investors. According to Lewis, the quant firms use a combination of:

  1. Paying exchanges for trading order flow to gain information milliseconds before other traders.
  2. Locating trading machines physically closer to central exchanges to get millisecond information advantages over other traders.
  3. Engaging in untraceable trading activity within a broker-sponsored ‘dark pool’ exchange to front-run slower traders.
  4. Sending rapid-fire trade inquiries and cancellations to exchanges or dark pools to manipulate market liquidity.

In addition, Lewis describes a plunky band of misfits led by Brad Katsuyama who cobble together an alternative stock exchange – The IEX – using old newspapers, string, and wadded up chewing gum to launch a better, fairer, slower exchange to keep out the quant trading baddies.

Kovac wrote his critique – excerpted below – to correct what he sees as the biggest errors of Lewis’ book.


Excerpt of Kovac’s Flash Boys Critique


I’ll link to the full book, available on Amazon, as soon as it gets posted.


Please see related book reviews:

Inside the Black Box, by Rishi Narang


Flash Boys, by Michael Lewis

Please also see related posts:

The Rise of The Machines

The Katsuyama Revolution Continues


Post read (2451) times.

The Katsuyama Revolution Continues

Katsuyama & Team at IEX
Katsuyama & Team at IEX

The Wall Street Journal carries an update this morning on the main protagonist of Michael Lewis’ recent book Flash Boys, Brad Katsuyama and his newly launched (October 2013) exchange known as the Investor’s Exchange (IEX).

One of my main questions left after reading the book is the viability of this newly launched exchange. Katsuyama & his team seek nothing less than the irrelevance of both dark pool equity trading and high frequency trading on equity exchanges, a mighty set of targets. Because Lewis published Flash Boys just a few months after the exchange’s launch, we’re left wondering whether Katsuyama’s revolution will happen or not.

As of today’s article, Katsuyama carries on, applying to expand the IEX into a full-fledged stock exchange. Most importantly, he has set the rules of the IEX so that traditional broker-dealers (The Goldmans and Morgans of the world) trade for free – to encourage them to bring their trade flow in high volume to the IEX, while outside firms – most pointedly we assume high frequency trading firm – all pay fixed commissions per trade.

This second part is a key feature of IEX, which is built to counteract the conflicted cost and fee structures of other equity exchanges which pay for order flow in a convoluted – but probably investor-unfriendly – way.

The main thesis of Flash Boys is that the combination of dark pools – in which broker-dealers did not disclose who had access to deal flow and in what manner – and the complicated set of fees paid or received in different equity exchange – seems to have benefited high frequency traders at the expense of slower market participants.

From what we can glean from the article, the Katsuyama revolution rolls on.

Please see related book review for Michael Lewis’ Flash Boys

also see related book review for Rishi Narang’s Inside the Black Box


Post read (8125) times.

Book Review: Flash Boys by Michael Lewis

The Rise of the Machines

Michael Lewis wrote Flash Boys to alert the non-finance world about the scourge of high frequency traders front-running investors and fracturing traditional capital markets.

Lewis does not distinguish between quantitative (or algorithmic) trading strategies and high frequency trading firms (HFTs), although it’s helpful to define these terms first.

Quantitative strategies – of which HFTs form a subset – are computer-driven trading models, in which the human input all occurs prior to a market’s opening bell. The human instructions come from computer programmers who tell the model to look for certain signals in the way securities trade to prompt a buy or sell order. HFTs are a type of quantitative strategy that rely on speed, in milliseconds, to successfully execute trades. A good primer generally on quant trading and HFTs from someone inside that world is Rishi Narang’s Inside The Black Box, which I reviewed recently.

Lewis points to at least four simultaneous innovations that have led to the profitable opportunity for high frequency trading firms over the past decade.

  • First, an investor-protection law from 2005 called Reg NMS demanded that investors receive the ‘best’ price visible on a stock exchange, even though sophisticated investors know that large investment purchases or sales may get a best overall price if done quietly ‘off-market’ without alerting the rest of the investment community. Following Reg NMS, HFTs can play games with the ‘visible’ market price by posting, say, 100 shares for purchase or sale, only to cancel that price as soon as a real order hits the market. In Lewis’ telling, the 100 share order from the HFTs becomes an electronic trip-wire to signal certain types of large investors are making a move, and allowing the HFTs to front-run that investor through superior trading speed.
  • Second, the fracturing of the equity markets into more than a dozen major electronic exchanges and 40 (or so) broker-dealer created ‘dark pools’ for anonymous electronic trading has created multiple opportunities for risk-less arbitrage between exchanges, for those HFTs who execute trades in milliseconds.
  • Third, the privatization of US stock exchanges like the Nasdaq and New York Stock Exchange led the exchanges to seek their own profit through fee arrangements with HFTs at the expense of investor-oriented protections, which would have limited the access of HFTs.
  • Fourth, technology – between lightning-fast software and speed-of-light fiber optic cable – created a haves and haves-not unfair playing field between investors in many markets.

Lewis’ narrative follows the evolution of his protagonist Brad Katsuyama who figures out just enough of the HFT game to become inspired to shut it down – first because it interferes with his job trading equities for the Royal Bank of Canada, and later because he’s a self-appointed evangelist for protecting real investors from the HFTs.

Bill Murray

Katsuyama and his plucky rag-tag group of Wall Street castoffs – and here Flash Boys most closely resembles the plot of every early Bill Murray movie like Meatballs and Stripes – set out to build a better exchange known as the Investors Exchange (IEX),[1] which through slow trading will box out the HFTs and their nasty algorithms.


The moralistic tone, and why it matters

Flash Boys differs from Lewis’ earlier finance books in the introduction of his moralistic tone – he seems genuinely outraged by the activities of high frequency trading firms. This moral outrage differs from the way that he was previously mostly amused by disgusting mortgage traders, stupid Icelandic Viking financiers, or Sub-prime CDO structurers.

In Liar’s Poker, Boomerang, and The Big Short Lewis distinguished himself from other financial journalists by adopting a knowing attitude toward Wall Street’s greedy ways. Whereas other financial journos portray a fairy tale world of virtuous small-time investors and evil greedy bullies, Lewis worked on Wall Street for a few years and knew better than to fall into that trap.

Lewis usually celebrates – at least up to a certain extent – those who outwit the competition to earn themselves a big payout.

Lewis’ bad guys in those earlier tales typically would receive a kind of satirical treatment for their excessive attitudes. Lewis found ways to laugh at his antagonists because he spent time enough with them to understand their strengths, weaknesses, and the right distinguishing characteristic to turn their unattractiveness into humor.

Lewis does not seem to have spent any time getting to know high frequency traders for Flash Boys, however, and here his moral tone – rather than knowing satire -exposes a weakness.

I’m not saying Lewis shouldn’t be upset about high frequency trading. He makes a compelling case that we should all take a much harder look at whether all of their activity acts like a massive, hidden, tax on capital markets. What I am saying is that the moral tone – which resembles the style of weaker financial journalists – exposes the fact that he hasn’t spent enough time getting to know actual high frequency traders.


If he had spent time with some, we would have gotten some funny anecdotes and satirical send-ups – That Russian programmer with the bad breath and an unhealthy obsession with Miley Cyrus! Ha! The South African technologist who keeps twenty cats in his office and eats only vegetables that start with the letter T! You can’t believe how funny these guys are! That kind of thing.

The humor is amusing in its own right of course, but the humor also tells us that Lewis was there, and got to know these people. Unique among journalists he has a track record of actually going out and finding the stories rather than create fairy tales based on preconceived moral views. The Good Guys = Brad Katsuyama & Team versus Bad Guys = Faceless & Nameless HFTs formula makes me suspect we only got a portion of the full story.

I’m thinking about Lewis’ apparent failure to talk to HFT folks because a friend of mine from the HFT industry thinks Lewis totally blew it when describing his world.

I do not know HFTs myself well enough to judge, but I know my friend has a moral compass and wants the HFT story portrayed accurately.

(And you should see his 20 cats! Just kidding.)

I’m hoping in coming weeks to learn enough to judge better the accuracy of Flash Boys. More importantly than judging the book, I’d like to know to what extend HFTs really threaten the system, as Lewis argues.

More questions than answers

For my own future reference, but also perhaps other readers, here’s my beginning list of further questions to explore and answer after reading Flash Boys.

We have got to stop SkyNet
  1. Lewis leaves practically unanswered what I think is the much greater problem of quantitative and high frequency trading: As computer algorithms constitutes 50-80% of all trading volume on US exchanges, what are we doing to shore up the system against massive technical fails like the Flash Crash of 2010, or like the Crash of ’87, for that matter? We haven’t seen The Big One yet but I’m pretty worried about it, and I hope regulators have a plan in place to prevent it. In other words, WE MUST PREVENT SKYNET! WHERE IS OUR JOHN CONNOR?
  2. If Katsuyama’s IEX is a better mousetrap and a solid protection against HFT front-running, as Lewis believes, how has it fared in the subsequent months since opening in October 2013? I’ll be curious to know if it has begun to siphon off volume from other exchanges and the broker-created dark pools. If investors are self-interested, they should want to participate in the IEX far more than the shark-infested dark pools.
  3. Lewis mentions only two HFT strategies that I can see, in simplest form: Strategy #1: Set up 100 share trip-wires inside these exchanges. When those get tripped, quickly front-run the direction of the market ahead of a big order. Strategy #2: Gain arbitrage opportunities by seeing an order in one exchange and then quickly executing in another exchange based on that order. Strategy #1 is borderline illegal so it strikes me as something that regulators could address. Strategy #2 is theoretically (marginally) ‘creating efficiencies,’ although not if the HFTs are, as they seem to be doing, seeing order flow to some exchanges faster than everyone else. IEX could put that strategy #2 out of business. But something tells me there are many dozens to hundreds more HFT strategies not described in this book. What are they?
  4. Whatever happened to the high-speed line built by Spread Networks from New Jersey to Chicago mentioned in the early chapters? And was it made obsolete by the microwave towers mentioned in the Epilogue, or is that part of the same network?
  5. My friend from the HFT firm mentioned this one to me: Lewis relays a very fishy anecdote about a hedge fund trader typing a buy order into his computer, only to watch the market suddenly shift away from him before he hits enter to execute the trade. This is, basically, impossible – unless the HFTs have hacked into the hedge fund guy’s computer – to see his trades before he even sends them to the exchange. Even I’m not that paranoid about Skynet yet. So, Lewis, what’s up with that anecdote?
  6. Can we, and should we, distinguish between quantitative trading – relying on computer algorithms rather than human input to execute trades – and HFTs in a meaningful way when it comes to regulation and treatment in a market exchange?


That’s my short list of questions. More to come later.

Please see related posts:

Book Review of Pete Kovac’s Flash Boys: Not So Fast

Book Review of Rishi Narang’s Inside The Black Box

Book Reviews of Michael Lewis’ previous books on finance:

Liar’s Poker


The Big Short

Crashes happen when quants take over the markets, in Rise of The Machines



[1] Fun fact: They didn’t use the full URL of the exchange name because, you know, investorsexchange.com could be interpreted a variety of ways.



Post read (7846) times.

Book Review: Inside the Black Box

I read Rishi Narang’s Inside the Black Box as a kind of primer on quantitative trading, in advance of reading Michael Lewis’ Flash Boys.

Need for a primer on quantitative trading

I need a primer before reading Flash Boys for several reasons.

I was an ‘over-the-counter’ institutional bond guy, rather than a participant in the exchange-traded stock markets that form the core of many quantitative trading strategies.  In addition, my Wall Street experience is a decade old at this point, (I left Goldman in 2004) and the development of dark pool trading, high-frequency trading (HFT), and alternate electronic exchange trading has proliferated since then.

Whereas the traditional stock exchanges with real-live human specialists (like the New York Stock Exchange – NYSE) was already in decline by the time I left Wall Street, the vast majority of stock-trading volumes now happen electronically on alternate exchanges and dark pools away from the traditional NYSE.

As I noted a few years ago, most stock-trading volume is entirely devoid of human decision-making, which is to say, computer-programmed algorithms determine what trades happen on a second-by-second, moment-by moment basis.

Finally, I have a few friends in the quantitative trading world and one of them reached out to me shortly after the publication of Lewis’ book and said there are laughably wrong statements and implications in Flash Boys. In the interest of having an intelligent dialogue with him, to figure out if Lewis really screwed up, I needed some background.


Different Audience for the Books

Narang’s audience for his book differs from Michael Lewis’ audience for Flash Boys. We rely on Lewis for funny anecdotes and characters, and enough financial sense to give the general public an accurate, entertaining, and non-technical narrative.

By contrast Narang’s book targets non-expert practitioners who need more quantitative trading knowledge such as: new hires into the quantitative trading world – or current and prospective investors/asset allocators to quant-trading funds. Narang currently is an investor/allocator to quantitative strategies, who previously founded, and ran, a quantitative trading firm directly.

Narang provides a simplified ‘how to understand’ or ‘how to evaluate’ quantitative trading firms, for readers without specialized trading and computer programming background.

When I say ‘simplified’ I don’t mean the book will be ‘simple’ for people outside finance. Narang’s language is clear and precise, even full of useful analogies for explaining concepts, but it’s academic in style. He systematically reviews, engineer-like, the common components and varieties of quantitative strategies.

I learned a ton, but folks without a need to know about quant trading will not rip through this like they would a Michael Lewis book on finance.

I had a number of questions about quant trading before I read Inside the Black Box and Narang sheds light on my questions.


What is quantitative trading and what should we know about it?

Quant trading, in its simplest definition, can be distinguished from discretionary trading by the role of human decision-making. In traditional, discretionary, investing we decide to sell or buy a financial instrument for any number of good or bad reasons. With quant trading, by contrast all of the human inputs occur ahead of time, that is to say, through the programming of a trading algorithm to look for and respond to the optimal opportunities to trade. The actual order to sell or buy occurs in response to the signals, rather than any in-the-moment human decision.

How big is quant trading compared to discretionary trading in the markets?

Narang cites a study from 2009 that indicated 60% of trading volume in US equities came from quantitative trading, as well as 45% of trading volume in European exchanges. So if 6 billion US shares changed hands per day, then Hal9000 algorithm-based trades accounted for 3.6 Billion of those, leaving only 2.4 Billion shares traded by human discretion. In addition, over 90% of commodity trading firms (CTAs) are quant-trading firms, rather than discretionary traders.[1]

So how do programmers and traders set up the trading algorithm to order trades ahead of time?

Naturally, the next question is about how this programming gets done, and answering this question forms the bulk of Inside the Black Box.

Narang comprehensively reviews and explains both the common elements of building a quant trading strategy, as well as the wide variety of approaches used by the entire spectrum of quant trading firms.

Typically, programming a successful trading algorithm depends on reviewing extraordinary amounts of historical data on observed trading responses of financial instruments, and then seeking patterns among the noise. If a quant trader can find predictable enough responses, then it becomes possible to test if trading on an expectation that the pattern continues could be profitable over time. If his algorithm recognizes the pattern well enough, he can, in effect, ‘see the future’ enough to trade predictably and profitably.


So, it’s pattern-seeking?

A few simple examples of patterns may be helpful here to give an introductory idea. Some stocks – often from the same industry – predictably track each other’s direction, such that a directional move from one typically will be mirrored by a directional move in the other. A similar pattern could hold for a single stock and a ‘basket’ of similar enough stocks.

Volume data on stocks – how much is being traded over any given time period – may predictably be followed by moves in the stock or financial instrument.

Some directional trends in financial instruments could indicate a predictable pattern of a continuation of that trend, or just as likely (and somewhat paradoxically) the likely reversal of a trend.

The common denominator in this type of trading, however, posit that past trading data indicates somehow what will happen in the near future.

To the extent that markets behave in patterned ways – and in fact most of the time financial markets do behave in recognizable historical patterns, or at least recognizable to quants and their computers – quant traders can make money with trading algorithms.  This doesn’t have to happen every time, with every trade or every day. In fact, quant strategies depend not so much on winning on every trade, but winning on enough trades to overcome the losses on other trades, as well as the costs, or ‘trading friction,’ of executing many trades simultaneously.

Some of the time, of course, market regimes may shift and past patterns become poor indicators of future moves. At that point, the human-element decision becomes whether to lower the trading exposure based on the algorithm, to shut down the algorithm altogether, or to reprogram the algorithm in response to updated market conditions and patterns.

My sense from reading Narang’s book, as well as from talking to friends in the business, is that quant traders must engage in constant vigilance around their algorithms:  Algorithms may cease to generate positive returns as competition eliminates the opportunity, or competitors’ speed eliminates the previous advantage, or patterns simply stop working.

Research and development to find new ‘signals’ inside the noise of past trading data requires constant work.

Summer Glau, my favorite terminator ever

Besides pattern-seeking and constant R & D, what are other elements common to the wide variety of quant traders?

DATA – The acquisition of historical data, the scrubbing of data to extract useful information, and the testing of hypotheses (will following that pattern actually make money?) using the historical data are common to all quants.

COSTS – Some strategies require thousands or even millions of trades, so costs associated with trading matter a lot.  Quants have to include in their model expected trading costs – not only commissions paid to brokers or exchange – but the ‘slippage’ that occurs due to trading. The scientific method analogy to this ‘slippage’ here would be the “Hawthorne Effect” or “Observer Effect,” meaning it’s difficult-to-impossible to collect ‘profit’ from the market without affecting the market opportunity itself.

When a quant finds a financial instrument momentarily ‘cheap’ according to some historical pattern, the act of purchasing the instrument to take advantage of the cheapness may cause the price to rise.

The larger the trade that the quant algorithm demands to take advantage of that momentary cheapness, the larger the ‘slippage’ due to the trade itself will actually eliminate the opportunity. Any historical model that fails to take into account this cost element will likely fail to work.

SCIENTIFIC METHOD – Quant-trading, more than discretionary trading, depends on a ‘scientific method’ approach to investing. While a discretionary trader may depend on fuzzy thinking and ‘gut feeling’ to guide his buy and sell signals, quant trading eliminates this part of trading in favor of a rigorous Hypothesis à Test à Hypothesis à Test continuous feedback loop, with consistent profitability (in trading-speak, the generation of “Alpha,” or excess return over risk) the primary indicator of whether the hypothesis has any validity in real markets. Like scientific hypotheses, the trading algorithm can never become ‘proven’ or ‘permanent law.’ In fact, many current predictors of market moves may be shown to be poor predictors in the future when markets change. A continuous testing of hypotheses about patterns through the scientific method helps distinguish quants from discretionary traders.

How different is High Frequency Trading (HFT) from other types of quant trading, and what should we know about the subset of quant trading known as HFT?

Narang devotes the final chapters of this book to explaining HFT. Narang suggests that the trading ideas of HFTs can often be much simpler than other quant trading – but depend on faster execution of the simpler idea.

Narang provides a relatively easy illustration of a typical, simple, HFT idea.

The S&P500 index consists of shares from the 500 largest market-capitalization US companies, in known relative weightings. Naturally whenever the price of an individual stock in the index changes, the value of the S&P 500 index as a whole changes slightly as well. An exchange traded fund (ETF) like the SPY, or an S&P500 futures contract (ES), both of which are meant to track the index precisely, should exactly mirror the price changes in the underlying stocks.

If, however, one of the underlying stocks goes up in value and the SPY value does not change precisely, instantaneously, to a new value based on the moves in the underlying stocks, a super-fast arbitrage can be had by buying the SPY and shorting the stock. These arbitrage opportunities are either rare or exist for very brief instances – all sorts of traders would look for precisely this simple risk-less opportunity – but trading speed makes exploiting a price difference like this possible.

Keanu: “Whoa”

The best way to explain this is that only a computer programmed to move as fast as Neo in that scene dodging Mr. Smith’s bullets, could hope to profit from the S&P/ SPY arbitrages. Nobody else has a chance.

The basic trading idea here is super-simple. On the other hand the technology involved in calculating the precise price discrepancies, in the right weightings, based on the right data feeds, and doing it faster than anyone else in the marketplace, is not easy.

The S&P 500 trade described above is probably version 1.0 of HFT trading, and most HFT firms seek out similar price opportunities with fewer competitors and less efficient instruments in myriad markets around the world.

HFTs may seek to use more complicated algorithms than this example, but at a certain point the computing time required to sift through the market signal may slow down the process, so that they begin to resemble other quant traders.

By keeping the trading idea dumb and simple, the HFTs retain the advantage that comes from technology. HFT is the Keanu Reeves of quant trading, dumb and dependent on Speed to keep the bus above 88 mph to prevent everything from exploding.

Conventional quants may have harder-to-explain patterns that they recognize and exploit, but probably do not depend on execution speed to the same extent as HFTs in order to be profitable.[2]

How likely, and with what severity, could we experience a destructive flash-crash catastrophe driven by the present and future take-over of markets by quant trading? In other words, should we the public (and regulators) be much more actively regulating SkyNet?

Narang addresses a version of this question in Chapter 10, “Risks Inherent to Quant Strategies,” although his goal is limited to showing – persuasively in my opinion – that large market blow-ups in the past cannot be blamed on quants.

The Long Term Capital Management (LTCM) blowup of September 1998, for example, may be described by some as a quant-fund disaster that fundamentally shook capital markets, and might have gotten worse without a Federal Reserve-Inspired ‘Bail-in’ of LTCM by Wall Street firms.

Narang points out, however, that while LTCM used sophisticated quantitative models in their trading, their strategy remained fundamentally discretionary – meaning humans made the calls on what went into and out of their portfolio.

The 2008 Mortgage-bond inspired crisis, in addition, cannot reasonably be linked to quant trading firms. While the creation and trading of highly-leveraged mortgage and default-swap CDOs involves sophisticated mathematics and modeling, the billions in losses owed more to a combination of leverage (debt), human error, poor risk-modeling, the connectivity of markets and counter-parts, and misunderstood incentives among disparate parts of the mortgage-bond origination chain than anything to do with quantitative trading – as explained well in Michael Lewis’ The Big Short.

Arguably, the most significant systemic market risk crash to date that could be blamed on quant-like trading is the October 1987 market crash, in which computer-based “portfolio-insurance” algorithms each triggered a cascade of automatic selling, independent of human judgment. The Crash of 1987 occurred prior to the rise of quant trading as we know it, so ‘blaming’ quant trading can really be done only by historic analogy. It’s the kind of crash we could imagine from algorithmic trading in which the size of computer trading orders, added to the overwhelming one-sidedness of the computer signal (a crowded trade) causes the devastation.

The actual quant trading blowups to date, as Narang explains, have caused localized losses without inflicting much observable or systemic market disruption.

During a few weeks in August 2007, for example, a majority of algorithmic traders simultaneously experienced sudden losses, as the trends suggested by historic patterns of trading relationships reversed. It turns out that the similarities in quantitative strategies at that time meant that as some quants lost money and unwound their portfolios they caused a cascade of losses in other quants’ positions.

In the meantime, the non-quantitative investment community – people who just own stocks or mutual funds, or non-quant hedge funds – barely noticed any particular change in market prices.

The market as a whole neither soared nor crashed. Only the historic patterns of short-term trading relationships broke down and then suddenly and severely reversed. As a silent market-crisis, only experienced by the quant community, August 2007 may be a harbinger of the kind of localized pain that non-quants can safely ignore. Or conversely, the August 2007 crisis indicates the growing power and ‘crowded trade’ nature of quant trading that regulators should worry about which could freeze up or destroy orderly markets in the future.

Other flash crashes, such as the May 2010 Flash Crash or other mini-crises such as happened in August 2012 that brought down Knight Trading, may fit the pattern of acceptable, localized, market blips.  Optimistically, programming errors like those are just a Darwinian way of punishing the inept and incentivizing the careful and level-headed to take advantage of others’ mistakes.

Or, they could be the early tremors in market-structure fault-lines created by quant trading that will eventually cause an irreversible earthquake in world markets. I don’t want to be alarmist on a topic where I’m an amateur, but I hope some regulators have taken on the ‘seismic geologist’ role of figuring out how to predict and prevent ‘The Big One’ caused by quant trading, before it happens.

I mean, we’ve all seen 2001 Space Odyssey, Tron, Terminator, and The Matrix, right, so we know this doesn’t bode well for algorithmic trading. Even Aliens and Planet of the Apes are basically warnings about quant trading, aren’t they? Wait, am I overthinking this?

Open the pod bay doors, please, Hal

Narang does not particularly address this much more troubling, forward-looking question, of whether and to what extent quants will significantly blow up markets in the future. His primary response in Chapter 16 to this worry or criticism points out that human errors have more frequently caused losses or mini-crashes in the markets than computer glitches. No doubt this is true, but that’s a bit like saying we shouldn’t worry about tornadoes because hurricanes have caused more damage up until now.

As we learned from the 2008 Crisis, markets have a way to experiencing the near-statistically impossible, stunningly unlikely, “6-sigma” event, or “1,000-year flood” just about every decade or two, causing massive wealth destruction in new and unpredictable ways.

Often we can see, in retrospect, that the root cause of a market crash has something to do with the rapid advance of a profitable financial technology far ahead of our ability to control all the risks of the financial technology or regulate the inequities or societal costs of the financial technology.

I’m not in a great position to evaluate the probability and consequences of a future algorithmic-trading driven market catastrophe, and perhaps Narang is not either, although he’s certainly better positioned than most.

I do remain worried about a future SkyNet-driven Crash.


Is quantitative trading in general, or HFT in particular (as the rapidest version of quantitative trading is known) inherently unfair, unethical, or taking advantage of other market participants?

I have not read Flash Boys yet, but I know enough about the plot line to see Michael Lewis’ book as a call to action about the inequities inherent in HFT, a subset of quantitative trading. Whether he’s right or not, regulators are already responding to their perception that HFTs are gaming the system in a way that’s harmful to markets overall.

Narang clearly feels that quantitative trading is an interesting and honorable way to make a living. In the final section of the book, he describes the methods and implications of High Frequency Trading, with noted contributions from his brother Manoj Narang, the CEO of a HFT firm Tradeworx and sometime commenter on the industry.

Given Narang’s experience and connection with the industry, we would not expect him to point a finger at quants or HFTs and declare them inherently unfair. What he does try to do instead is educate the non-expert in how quants may make money, and why HFTs in particular feel the need for speed.

To give one example of this need for speed, his brother’s firm Tradeworx researched the profit difference between placing purchase or sale orders first, versus placing them later in the queue for order filling. Inherent to equity trading, he explains, is the rule that when buy or sell orders get placed on an exchange at the same price, the first order placed gets filled first. The value of being first with certain types of orders, according to Tradeworx, turns out to be approximately 1.7 cents per share.

With other kinds of orders, they have found that immediate, speedy, fulfillment of the trade reduces the cost or ‘slippage’ due to trading. In a strategy dependent on a massive amount of trading to capture momentarily fleeting price reactions, the high speed fulfillment of orders matters tremendously.

How do HFTs get so fast?

In the same chapter, Narang mentions the advantage of locating HFT trading servers at, or near, market exchange servers, a major theme of Flash Boys.

Along a similar vein, Narang reports the lengths and costs endured by trading firms to reduce data transmission times between major trading centers such as New York, Chicago, and London. HFT trading firms will sign long-term, multi-million dollar contracts for fiber optic access in order to shave off five milliseconds (there are 1,000 milliseconds in every second, so five milliseconds is 5 thousandths of a second!) in data transmission times between New York and London, as an indication of the importance of being the first to post, or first to cancel, transaction orders.[3]

HFT firms, Narang points out, monitor data and order trades in tenths of a millisecond, creating myriad technical challenges, between hardware, software and network engineering.

Another speed bump in the way of the highest frequency trading, Narang notes, has been the regulatory rule adopted in July 2011 that US equity brokers must enact a ‘risk check’ for every trade, in order to ensure that the size and capacity of the customer and the trade are valid – meaning not too big, which might indicate a trade made in error.

The brokerage software required to do this risk check slows down every trade, which could dampen HFTs speed by 0.5 milliseconds. In response, some HFT created their own brokerages, in order to reduce the time-lag of using someone else’s risk check system.


Where do I differ from Narang’s perspective?

1. This is a bit nitpicky, but I have a philosophical issue with Narang’s definition of ‘value’ and the role of ‘timing’ in investing.  He argues in both Chapter 3 and Chapter 16 that all investors, all the way from HFTs on the one side of the spectrum to long-term investors such as Warren Buffett on the other side of the spectrum, depend on timing to beat the market. He means that investors have to execute their purchase (or sale) in advance of others’ execution, and it’s this earlier-than-others’ action that creates value.

In Narang’s presentation, the good investor buys first, others follow suit later, and the original investor can then sell at a higher price based on the fact that others have driven up the price of the instrument.

Here I think Narang’s missed the point of value-investing as advocated by Buffett: Value is created by purchasing future cash-flows at an attractive price, and value investors are mostly indifferent to other investors’ subsequent actions. If you conceive of stock ownership with a perpetual time horizon, as a value investor like Buffett seems to, and that value derives from those future cash flows purchased at a  good price, then timing falls out nearly completely as a consideration.

With a quant’s focus on short-term price changes rather than the cash-flow potential of financial instruments, I can see why Narang views timing as a key to all investing, but I think that focus leads to inaccurately describing other types of long-term investing.

2. I cannot tell from what I read and really this book doesn’t say – whether a constant stream of high frequency orders and cancelled orders constitutes a confusing ‘spamming’ of equity market exchanges, to the detriment of slower investors.  Rapid fire orders, the non-quant critics say, create the illusion of market liquidity where it doesn’t really exist, as it may disappear quickly when non-quants actually seek to trade.

I can understand the way in which HFTs seeking arbitrage opportunities between linked instruments (like the S&P500 and the SPY) create liquidity and efficiency to some extent, in a way that may benefit the smooth running of markets.

In addition, Narang claims that rapid order cancellation is a necessary component of HFT strategies[4] and he claims that no evidence supports the complaints from non-quants about damage done by this type of activity.

But do millisecond orders and cancelled orders jam up the orderly execution of others? Does algorithmic swarming of orders around non-quant investment flow hurt the non-quants in a fundamentally unhelpful or unfair way?

Clearly some non-quants traders and investors believe they are harmed by excessive order-making and order-cancellation, when it comes to trading. Are the non-quants being overly paranoid or jealous of HFT profits, or is there actual harm done? Basically I need a non-quant trader’s opinion here, and I don’t think Narang sufficiently addresses this type of externality of HFT activity.

3. I’ve already mentioned that his book does not dwell on the potential for future catastrophic losses, nor did I expect it to, as the future is unpredictable and Narang would be engaging in speculation about the unknown.  On the other hand, we’ve had enough mini-shocks in the market attributable to algorithmic trading that I hope someone, somewhere (ideally in a government regulatory office) is thinking deeply about ways to prevent The Big One from happening.

Please see related book review: Flash Boys, by Michael Lewis

as well as Flash Boys – Not So Fast by Pete Kovac

Please see related post:  The Rise of The Machines


[1] People, what I’m trying to say is: SkyNet already took over the commodities and equity markets. Good thing there’s some real live ex-bankers who still write finance blogs. Ha ha. Or maybe that’s just what I want you to think.

[2] Maybe in this analogy traditional quant trading is a more complex George Clooney-character? I don’t know, still working on it.

[3] I don’t generally begrudge hedge fund traders anything, but now I’m starting to think they never have to wait those annoying 10 seconds for Game of Thrones to load up on Amazon Prime, like I typically do. Damn, I hate those guys. If I was a well-compensated hedge fund trader right now I would happily pay 12 million dollars just to eliminate that delay.

[4] to avoid a problem he explains well, which is the ‘negative selection’ of passive orders on an exchange.

Post read (12731) times.

Book Review, Review: Michael Lewis on John Lanchester’s “Capital”

I learned from Michael Lewis’ article in this month’s New York Review of Books that the English have a particularly literary strength compared to us Yanks.

After quoting a passage from John Lanchester’s Capital featuring the inner quotidian dialogue of a Londoner named Ahmed, Lewis observes:

You can find this sort of thing on every other page—a fresh and interesting description of a sensation you might have experienced a hundred times without ever having bothered to attach words to it. The talent for these sorts of small-bore social observations is peculiarly English—it kept Kingsley Amis in business for years, and still makes Alan Bennett’s diaries feel like required reading. Maybe it’s the bad weather. (All those hours trapped indoors, watching one another.) Or maybe it’s the literary side effect of a middle-class culture in which people are expected to be painfully self-conscious, clammy in their own skin, and alert to their own folly and deceptions, lest they be spotted first by others. Whatever the reason, the English really are just better at this sort of thing than anyone on the planet.

This strikes me as likely and true, so now I will be on the lookout for it ever after.

I also learned from Lewis that the English had a particular historic weakness, relative to us Yanks.

Lewis uses the article to point out the English historical evolution with respect to American Wall Street culture.  In the early 1980s, he argues, London exhibited a curiously uncommercial attitude toward business and finance.[1]

Sometime in the last 30 years, however, an über-capitalist mentality seized the City[2] and its new, brash, Masters of the Universe.  Lewis reviews John Lanchester’s new novel in part in order to describe the evolution of London’s new elite.

I write book reviews for this site

1. To organize my own thoughts about something I’ve read;

2. To save readers the risk of reading something they’re not sure they’ll enjoy, and;

3. Most often for the largest part, because the book review allows me to make a point about something that I already wanted to say.

The New York Review of Books is better at these latter two functions than any other publication I know of, and of course Michael Lewis is better at writing about finance than anyone else.[3]  So you can imagine my pleasure on opening up this month’s NYRB to see Lewis’ review of an author I’d never heard of, who recently published a novel Capital about a diverse collection of characters in the City, in London.

Lewis clearly admires this hitherto unknown-to-me John Lanchester, even anointing him ‘one of the greatest explainers of the financial crisis and its aftermath.’  Further praise by Lewis about Lanchester:

He has a gift for taking a reader who knows nothing about a complicated topic and leaving him with the feeling that he knows all about it, or at least everything worth knowing.  He makes you feel smarter than you are.

Which, of course, is what I would have said about Michael Lewis.  Bottom-line: I’ve got to get John Lanchester’s book, Capital: A Novel.

Please also see my reviews of Michael Lewis’ books:

Liar’s Poker

The Big Short



Please see related post: All Bankers Anonymous Book Reviews in one place.


[1] Lewis related a hilarious (to me) anecdote from the early 1980s London, illustrating this curious lack of commercial attitude.  He writes: “There was a small grocery store around the corner from my flat, which carried a rare enjoyable British foodstuff, McVitties’ biscuits.  One morning the biscuits were gone.  ‘Oh, we used to sell those,’ said the very sweet woman who ran the place, ‘but we kept running out, so we don’t bother anymore.’”

[2] “The City” is the London equivalent of “Wall Street.”

[3] I’ve previously reviewed three of his finance books, Liar’s Poker, The Big Short, and Boomerang.

Post read (6222) times.