We are at a crucial juncture in the history of the financial markets: Technology and deregulation have wiped out the floor-based monopolies of the past and replaced them with an incredible new system for transacting securities. In some ways this transformation has been unbelievably beneficial, as it has opened up markets, brought down trading costs and increased efficiency across global markets.
In other ways, this transformation has been very bad, resulting in a complex system with far more rules, linkages and interconnections than the previous system, all operating at speeds that are difficult for humans to comprehend. This has frightened some people, and understandably so.
While supporters of the current system like to brush off criticism of our new electronic stock markets as the dinosaurs' last gasp, they fail to recognize that the public itself has started to share these concerns. Capitalism survives because of the confidence of its participants that it is functioning properly and fairly, to a certain extent, and this confidence is waning for many reasons. The U.S. political system aside, I continue to believe market structure plays a significant role in the confidence that the public has in the economy and the financial markets, whether they even have heard of the term.
It often has been said that "complex systems fail," and that this means that the repeated technology problems we've witnessed in equity markets over the past several years are excusable. In other words, this is what we should expect as a "new normal." As any student of complexity theory will tell you, this is misguided. Complexity theory tells us that while systems will fail repeatedly, that does not mean they need to crash.
While airplanes undoubtedly have systems fail all the time, they do not typically crash. They fail gracefully, even transparently to their users/customers. It may not feel that way when you're waiting at the airport because of another delayed flight, but the safety record in the airline industry is incredible, and exceeds a five 9s (99.999 percent) success rate in terms of flights that do not crash. Other complex industries able to achieve five 9s include spaceflight, telecommunications, payment processing, manufacturing, oil/gas exploration and delivery, and many others. Why should we expect anything less from the backbone of capitalism?
Financial services is a complex system and a complex industry, one that has struggled in the past to find the right balance between regulation and free markets. As a technologist, I am dismayed at the constant technology failures that plague our industry. The list has become all-too-familiar, even to the layman: The Flash Crash, the BATS and Facebook IPOs, the Knightmare on Wall Street, and now Goldman's Options Trades and the Nasdaq Outage last week.
How have we addressed these problems? Have we taken a step back, and looked at the system as a whole, and honestly acknowledged its shortcomings? Have we spent time truly deliberating over the nature of our new, complex markets, and the conflicts of interest and skewed incentives that have gotten us to where we are today?
No. We have failed to have an honest conversation about these problems, instead trying to be political or to preserve the status quo (read: our P&L). We have failed to separate the interests of those who are making money and in power from those whom the markets serve. Instead, we mistakenly search for the "broken part" that causes each of these technology failures and try to fix it.
We talk about kill switches, circuit breakers, limit-up/limit-down, risk checks and software-development methodologies, while ignoring the broader issues of complexity, fragmentation, conflicts of interest and misaligned incentives. It is far easier to "fix" the "cause" of the previous problem, only to have a new one pop up, as if we're playing technological whack-a-mole. For some reason we convince ourselves that the failures of the past are a guide to future problems, and if we can just keep fixing these problems with band-aids, eventually the market will operate smoothly.
This attitude shows an incredible misunderstanding of complex systems. Without an understanding of complexity theory, we will never be able to have the difficult, honest dialogue needed to help shepherd our industry through this transformation. In order to prevent future problems and reduce unnecessary complexity, we must shift the incentives in our industry away from those that incentivize competition and speed over system stability. This means examining skewed incentive structures such as maker-taker pricing and self-regulatory organizations, among others.
[Hear More: Cris Sheridan on HFT, the Surveillance Society, and Mathematical Governance]
Anyone who has an interest in these issues should read Drift Into Failure, by Sidney Dekker. I have not read a finer explanation of complex industries and the interplay between firms and regulators in a complex, dynamic environment. It means changing the language that you use to describe the system, the means of managing incentives and conflicts of interest, and the role of the regulator from rule-maker to co-evolver. It offers insights on high-reliability research and fault-tolerant system design. Most important, it forces us to acknowledge and embrace our limitations, rather than deny them.
I'll close with a quote from the book:
"The growth of complexity in society has got ahead of our understanding of how complex systems work and fail. Our technologies have got ahead of our theories. Our theories are still fundamentally reductionist, componential and linear. Our technologies, however, are increasingly complex, emergent and non-linear. Or they get released into environments that make them complex, emergent and non-linear. If we keep seeing complex systems as simple systems - because of the dominant logic and inherited scientific-engineering language of Newton and Descartes - we will keep missing opportunities for better understanding of failure. We will keep missing opportunities to develop fairer responses to failure, and more effective interventions before failure."
Follow Dave Lauer at twitter.com/dlauer
