If you have an ACS member number, please enter it here so we can link this account to your membership. (optional)

ACS values your privacy. By submitting your information, you are gaining access to C&EN and subscribing to our weekly newsletter. We use the information you provide to make your reading experience better, and we will never sell your data to third party members.



Scientists Tackle Finance

Physicists and other scientists apply mathematical models to financial markets, with mixed results

by Frederick M. Peterson
June 3, 2013 | A version of this story appeared in Volume 91, Issue 22

Credit: Shutterstock/C&EN
Metal sculpture of Wall Street bull symbol in street with physics formulas overlaid.
Credit: Shutterstock/C&EN

I find most business books to be boring and was reluctant to read another one about financial market forecasting. I was mistaken. “The Physics of Wall Street,” by James Owen Weatherall, provides a lively and intellectually stimulating account of how physicists and those with similar training have advanced the sciences of economics, finance, and forecasting.

The book is about more than just forecasting. It is an excellent history, skillfully weaving the personal lives and professional contributions of scientists who worked in economics or forecasting. The book even does a reasonable job of explaining the broader contributions of these scientists. It delves, for instance, into relativity and the statistical mechanics of molecules. In the epilogue, we learn that physicists are even touching one of the “third rails” of politics—price indexing of the U.S. Social Security entitlements. Physicists should learn what happened to a chemist named Lavoisier.

However, as a defense of using mathematical models to forecast and trade in financial markets, the book leaves me less than convinced. Having done similar work during my entire career, I have achieved a degree of humility when it comes to forecasting the future. We need the mathematical elegance and rigor that physicists are bringing to the field, but someday they may have to acknowledge that they are up against something equivalent to the Heisenberg uncertainty principle—some things are unknowable.

The Physics of Wall Street: A Brief History of Predicting the Unpredictable, by James Owen Weatherall, Houghton Mifflin Harcourt, 2013, 304 pages, $27 hardcover (ISBN: 978-0547317274)
A red book cover entitled "The Physics of Wall Street" with origami U.S. dollar bills representing the "Y" and "A" in the words Physics and Wall.
The Physics of Wall Street: A Brief History of Predicting the Unpredictable, by James Owen Weatherall, Houghton Mifflin Harcourt, 2013, 304 pages, $27 hardcover (ISBN: 978-0547317274)

Because Chemical & Engineering News is a magazine devoted to chemistry, a little professional rivalry is appropriate. Although many of the people Weatherall cites are, or were, truly physicists, many come from other fields as well. For instance, he puts J. Willard Gibbs into his club. In 1863, Yale University awarded Gibbs a Ph.D. in engineering, the first such degree awarded in the U.S. Some people consider Gibbs the greatest “chemist” of the 19th century—he’s ours! Gibbs influenced Yale students like Irving Fisher, the first American economist, and mathematical physicist E. B. Wilson. Wilson had a big influence on Paul Samuelson, who advanced the science of forecasting and was the first American to win a Nobel Prize in Economics. I grant that physicists are pretty smart. We just don’t want them stealing our people.

Weatherall, an assistant professor of logic and philosophy of science at the University of California, Irvine, focuses on forecasting, particularly the forecasting of financial markets with mathematical models. This is a controversial subject. There are many criticisms of this kind of forecasting, including, “It’s a waste of resources,” “It’s dangerous,” “It’s ineffective,” and what about “The Hedgehog and the Fox”?

As to the criticism about resources, one hears, especially from Europeans, that market forecasting and trading is a “zero-sum game” providing no net social benefit. “Why should some of the country’s finest mathematicians and physicists be employed in this manner?” such critics ask. “What a waste!” they proclaim. And, of course, “They are paid way too much!”

Without a doubt, mathematical trading models can be dangerous. The U.S. hedge fund Long-Term Capital Management (LTCM) was run by some of the brightest minds in mathematical finance, including Nobel Prize winners Robert C. Merton and Myron Scholes. The company did very well for a while, but then the Russian ruble collapsed in 1998—something the model couldn’t handle—and LTCM nearly took Wall Street down with it. (See Roger Lowenstein, “When Genius Failed: The Rise and Fall of Long-Term Capital Management,” Random House, 2000. Merton and Scholes were two of the creators of the highly acclaimed Black-Scholes-Merton option pricing model.)

With its high-frequency trading algorithms, Knight Capital Group was the largest trader in U.S. equities. Last August, the algorithms suddenly ran amok and wiped out the company’s capital in just 45 minutes. “Where was the off switch?” people asked.

Weatherall suggests that such episodes are the result of firms not really listening to their physicists. He says the firms built “black box” models rather than really modeling and understanding the market processes. He writes that “models in finance are best thought of as tools for certain kinds of purposes, and ... these tools make sense only in the context of an iterative process of developing models and then figuring out when, why, and how they fail—so that the next generation of models are robust in ways that the older models were not.” Weatherall expresses some welcome humility here, but my guess is that there will always be another iteration, another level of sophistication, or another “Russian ruble” type of event that wasn’t expected and now needs to be added to the model.

Master investor Warren Buffett thinks that trading models add market volatility and risk—what is called “systemic risk.” Could these models bring down the entire world financial system? There have been various efforts to restrict computerized trading—“shock absorbers,” etcetera—but it is fair to say we haven’t gotten our heads around this problem yet. It is a little scary.

As to the effectiveness of models, the argument, best articulated by Burton Malkiel (“A Random Walk Down Wall Street,” W.W. Norton & Co., 1973), has long been that financial markets are random and nobody can beat them consistently (without insider information). As the argument goes, out of pure randomness with everyone having the same probabilities of success, there will always be people with far superior investment records over the past 10, 20, or even 30 years. Despite their superior record in the past, these investors have the same probability of success this year as everyone else—or so say the “random walkers.” One wonders whether the successful mathematical trading operations that Weatherall cites might not be examples of such statistical flukes. Mutual fund companies are able to claim that their funds “beat the averages” mostly by dumping or reconstituting funds that have bad records.

Then there’s the question, not mentioned by Weatherall in “The Physics of Wall Street,” of “The Hedgehog and the Fox.” In “Expert Political Judgment: How Good Is It? How Can We Know?” (Princeton University Press, 2006), Philip E. Tetlock, professor of political psychology at the University of California, Berkeley, Haas School of Business, refers to a 1953 essay by philosopher Isaiah Berlin, who in turn reached back to the Greek philosopher Archilochus (“the fox knows many things, but the hedgehog knows one big thing”). According to Tetlock, most experts are hedgehogs by nature. They know some things very well and try to interpret everything deductively within the framework of the principles that they have learned so well. They tend to be confident, to be mediagenic, and to make concise, black or white, predictions.

Foxes, on the other hand, lack confidence in their own or anybody’s ability to forecast complex events. They don’t have a single, or anchoring, doctrine, and they change their minds quite readily in the face of new information—or just a bad night’s sleep. They can sound wishy-washy—“on the one hand, and then on the other hand”—and don’t tend to get much media attention. Tetlock collected many years’ worth of predictions by people whose personal styles fit into one or the other of these animal categories—hedgehog or fox. Guess which one made the most accurate forecasts? The foxes!

This doesn’t bode well for physicists, or economists or chemists for that matter. We are all highly trained, think we know how the world works, and would have to classify ourselves, for the most part, as hedgehogs—the ones who sound confident but get it wrong. (Don’t tell anybody, though. We still like the media attention.)

That said, one still has to commend the contributions that physicists have made to the field of forecasting. They have good answers to many of the criticisms I’ve cited. First of all, who is to say what is a waste of talent? In this day and age, people are free to study whatever they find profitable, or just interesting. One can also argue that improved forecasting and trading increases market efficiency and thereby improves the allocation of capital in the economy.

Computerized trading models may pose dangers, but who better to figure out these dangers and mitigate them than the physicists? They have the ability to understand complex systems, and the financial markets are truly complex systems. If it hasn’t happened already, some of them need to be working for the Securities & Exchange Commission—albeit at lower pay.

The “random walk” theory sounds appealing, but, as Weatherall describes in the book, physicists have shown that not everything is random. While hiding from the Nazis in Lyon, France, during World War II, the gifted mathematician Benoit Mandelbrot developed the idea of fractal geometry, which led to chaos theory and the study of complex systems. Many things that appear to be random are in fact the result of highly complex, deterministic processes. Physicists have used this approach to better predict such diverse phenomena as earthquakes, pressure vessel failures, and financial market dislocations. They actually figured out the processes—at least well enough to predict them. We now know, for instance, that pressure vessels issue certain kinds of vibrations before they fail, and market dislocations are preceded by what are called “log-periodic” movements. Some people have used this kind of information to spot market opportunities and make a lot of money. The aforementioned Samuelson started a commodities trading firm that made good money for decades.

Eventually, of course, other people get access to the information and spot the market opportunities. Once enough people spot them, the markets take away the opportunities before they can be realized by anyone. The market simply becomes more efficient—a social benefit. This is analogous to the patent system, which awards a benefit to the inventor for a period of time, before the invention becomes a benefit for all.

Physicists should keep up the good work, but they should acknowledge that they are mostly hedgehogs like the rest of us professionals. Our foxy friends, or spouses, can probably do a better job of forecasting.

Frederick M. Peterson is president of Probe Economics, which provides planning, forecasting, and consulting services to the chemical industry. He holds a B.S. in chemical engineering from the University of California, Berkeley, and a Ph.D. in economics from Princeton University.


This article has been sent to the following recipient:

Chemistry matters. Join us to get the news you need.