Thomas Frey //December 10, 2012//
(Editor’s note: This is the first of two parts.)
When you buy a stock, you place a bet on how that stock will perform in the future. In a perfect world, where market insiders and manipulators are removed from the equation, the market is a terrific tool for determining the true value of companies being invested in.
But what happens when the volume of data used to make decisions increases 100 million times, and trading volumes increase 100 million times, and trades can be transacted over 100 million times a second?
The proliferation of mobile phones, social media, machine data, and web logs has led to massive amounts of data being generated, stored and processed, and this volume is increasing exponentially with the digital shift from offline to online. Inside these rapidly expanding data pools are millions of tiny little “tells” that can be extracted and combined with the emerging science of anticipatory computing into very predictable movement indicators.
At the same time, these “tells” can be fabricated, manipulated, and auto-generated. In less time than it takes for a human to blink an eye, the entire value of the markets can fluctuate over $100 trillion and back again without leaving any humanly understandable trace of what just happened.
How long will it be before a series of trading algorithms go terribly wrong, causing the markets to simply implode? Here are a couple thoughts.
Moving into the Big Data Era
First a few jaw-dropping stats on the growth of structured and unstructured data. Here are a few examples:
Big Data is indeed …big! And getting bigger.
The Future Stock Market
When it comes to the stock market, humans are already the outsiders. The number of actual trades transacted by real humans is only a small fraction of a day’s total trading volume.
Yes, there are still people who buy stocks because they “like” the company, and yes, companies who are attempting to recruit key employees will still give stock options as part of their compensation package.
But for those who are serious traders, buying and selling stock on a daily basis, the game is constantly amping up to new levels where automated systems do most of the grunt work.
In a trash-talking society with quants boasting “my algorithm can beat your algorithm,” the entire stock exchange has deteriorated into a kind of bot vs. bot warfare that has lost all semblance of being a marketplace of human values.
One Possible Scenario
As a way to help you understand how this may unfold, here is a short scenario.
With the stock market, every trade creates a butterfly effect, a tiny ripple of influence, with the size of the ripple being directly proportional to the size of the trade. When trades happen at lightening speed, the size of the ripples can be manipulated through a series of external distortion signals designed to mislead all of the other bots, leading to ever-greater buy-sell patterns until they reach a point of maximum distortion for profit taking.
If, as example, a quant were to set up a specific trading pattern starting with 2 shares, followed over a set period of time by a series of doublings, 4 shares, then 8, 16, 32, 64, 128, 256, 512, and so on, the exponentially growing signal produces a “resonancing effect.”
Whenever a pattern like this is repeated multiple times, other bots will pick up on it, and start formulating strategies around it. For the puppet masters this becomes a relatively easy way to manipulate the market by first establishing a pattern, and then skipping or altering the final trade.
Much like the cracking of a whip, where all of the attention is focused on the final snap, the market positions itself for the grand finale. Predictability creates manipulability. It’s games like these that polymaths live for.