By Robert H. Frank

Why do hardworking people with similar talents and training often earn such dramatically different incomes? And why, too, have these earnings gaps grown so much larger in recent decades? Almost no other questions have proved more enduringly fascinating to economists.

The traditional approach to these questions views labor markets as perfectly competitive meritocracies in which people are paid in accordance with the value of what they produce. In this view, earnings differences result largely from individual differences in “human capital”—an amalgam of intelligence, training, experience, social skills, and other personal characteristics known to affect productivity. Human capital commands a rate of return in the marketplace, just like any other asset, suggesting that individual pay differences should be proportional to the corresponding differences in human capital. So, for example, if Sue has twice as much human capital as James, her earnings should be roughly twice as large.

But not even the most sophisticated measures of human capital can explain more than a tiny fraction of individual earnings differences during any year. And since the distributions of intelligence, experience, and other traits across individuals don’t seem to have changed much during the past few decades, the human capital approach has little to say about growing pay disparities over time.

The human capital approach is also completely silent about the role of chance events in the labor market. It assumes that the more human capital you have, the more you get paid, which obviously isn’t always the case. Of course, most people in the top 1 percent didn’t get there just by being lucky. Almost all of them work extremely hard and are unusually good at what they do. They have lots of human capital. But what the human capital approach misses is that certain skills are far more valuable in some settings than in others. In our 1995 book, The Winner- Take-All Society, Philip Cook and I argued that a gifted salesperson, for example, will be far more productive if her assignment is to sell financial securities to sovereign wealth funds than if she’s selling children’s shoes.

If markets have been growing more competitive over time, why are the earnings gaps unaccounted for by the human capital approach larger than ever? Cook and I argued that what’s been changing is that new technologies and market institutions have been providing growing leverage for the talents of the ablest individuals. The best option available to patients suffering from a rare illness was once to consult with the most knowledgeable local practitioner. But now that medical records can be sent anywhere with a single mouse click, today’s patients can receive advice from the world’s leading authority on that illness.

Such changes didn’t begin yesterday. Alfred Marshall, the great nineteenth-century British economist, described how advances in transportation enabled the best producers in almost every domain to extend their reach. Piano manufacturing, for instance, was once widely dispersed, simply because pianos were so costly to transport. Unless they were produced close to where buyers lived, shipping costs quickly became prohibitive.

But with each extension of the highway, rail, and canal systems, shipping costs fell sharply, and at each step production became more concentrated. Worldwide, only a handful of the best piano producers now survive. It’s of course a good thing that their superior offerings are now available to more people. But an inevitable side effect has been that producers with even a slight edge over their rivals went on to capture most of the industry’s income.

Therein lies a hint about why chance events have grown more important even as markets have become more competitive. When shipping costs fell dramatically, producers who were once local monopolists serving geographically isolated markets found themselves battling one another for survival. In those battles, even a tiny cost advantage or quality edge could be decisive. Minor random events can easily tip the balance in such competitions— and in the process spell the difference between great wealth and economic failure. So luck is becoming more important in part because the stakes have increased sharply in contests whose outcomes have always hinged partly on chance events.

Cook and I argued that these changes help explain both the growing income differences between ostensibly similar individuals and the surge in income inequality that began in the late 1960s. In domain after domain, we wrote, technology has been enabling the most gifted performers to extend their reach.

Winner-take-all markets generally display two characteristic features.

One is that rewards depend less on absolute performance than on relative performance. Steffi Graf, one of the best female tennis players of all time, played at a consistently high level throughout the mid-1990s, yet she earned considerably more during the twelve months after April 1993 than during the preceding twelve months. One reason was the absence during the latter period of her rival Monica Seles, who had been forced to leave the tour after being stabbed in the back that April by a deranged fan at a tournament in Germany. Although the absolute quality of Graf’s play didn’t change much during Seles’s absence, her relative quality improved substantially.

A second important feature of winner-take-all markets is that rewards tend to be highly concentrated in the hands of a few top performers. That can occur for many reasons, but most often it’s a consequence of production technologies that extend a given performer’s reach. That’s true, for example, in the music industry, which exhibits both features of winner-take-all markets. As the economist Sherwin Rosen wrote,

“The market for classical music has never been larger than it is now, yet the number of full-time soloists on any given instrument is on the order of only a few hundred (and much smaller for instruments other than voice, violin, and piano). Performers of the first rank comprise a limited handful out of these small totals and have very large in- comes. There are also known to be substantial differences between [their incomes and the incomes of] those in the second rank, even though most consumers would have difficulty detecting more than minor differences in a “blind” hearing.”

One hundred years ago, the only way to listen to music was to attend a live performance. Then as now, opera buffs wanted to hear the most renowned singers perform, but there were only so many live events those musicians could stage during any given year. And so there was a robust market for thousands of sopranos and tenors on the worldwide tour. The lesser-ranked performers earned less than their higher-ranked colleagues, but not that much less. Now, lifelike recording technologies enable fans to hear their favorite operas reproduced faithfully at home. And those who demand the entire stage spectacle can now watch HD broadcasts of performances of the New York Metropolitan Opera Company in theaters around the globe. All the while, local opera companies have been closing their doors.

Explosion in CEO pay

The forces driving recent trends in CEO pay shed additional light on how small differences in performance can translate into enormous differences in earnings. Consider a company with $10 billion in annual earnings that has narrowed its CEO search to two finalists, one slightly more talented than the other—by enough, say, to cause a 3 percent swing in the company’s bottom line. Even that minuscule talent difference would translate into an additional $300 million in earnings. Even if the better performer were paid $100 million, that person would still be a bargain.

CEO leverage has been growing quickly as firms have expanded in size. As the New York University economists Xavier Gabaix and Augustin Landier argued in a 2008 paper, executive pay in a competitive market should vary in direct proportion to the market capitalization of the company. They found that CEO compensation at large companies grew sixfold between 1980 and 2003, roughly the same as the market-cap growth of these businesses.But growth in executive leverage alone cannot explain the explosive increase in executive salaries.

A second factor necessary to explain explosive CEO pay growth—an open market for CEOs—simply didn’t exist in earlier decades. Until recently, most corporate boards shared an implicit belief that the only credible candidates for top executive positions were employees who had spent all or most of their careers with the company. There was usually a leading internal candidate to succeed a retiring CEO, and seldom more than a few others who were even credible. Under the circumstances, CEO pay was a matter of bilateral negotiation between the board and the anointed successor.

That focus on insiders has softened in recent decades, a change driven in no small part by one particularly visible outside hire. That would be Louis J. Gerstner, who was hired away from RJR Nabisco by IBM in 1993. At the time, outside observers were extremely skeptical that a former tobacco CEO would be able to turn the struggling computer giant around. But IBM’s board felt that Gerstner’s motivational and managerial talents were just what the company needed and that subordinates could compensate for any technical gaps in Gerstner’s knowledge. The company’s bet paid off spectacularly, of course, and in the ensuing years the trend toward outside CEO hires has accelerated across most industries.

Most companies still promote CEOs from within, but even in those cases, the more open market for executive talent has completely transformed the climate in which salary negotiations take place. Internal candidates can now threaten credibly to move if they’re not paid in accordance with the market’s estimate of their economic value.

The more open market conditions have affected executive salaries in much the same way that free agency affected the salaries of professional athletes. CEOs of the largest American corporations, who were paid forty-two times as much as the average worker as recently as 1980, are now paid more than four hundred times as much. So once more we see the growing importance of the seemingly minor random events that produce small differences in absolute performance.

The winner-take-all account of rising inequality has not persuaded everyone. Some critics complain, for example, that the explosive growth of CEO pay proves that executive labor markets are not really competitive—that CEOs appoint cronies to their boards who approve unjustifiably large pay packages. We’re also told that industrial behemoths conspire to drive out their rivals, thereby extorting higher prices from captive customers. To be sure, such abuses occur. But they’re no worse now than they’ve always been. As Adam Smith wrote in The Wealth of Nations, “People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public, or in some contrivance to raise prices.” CEOs have always appointed people they know to their boards, so that’s not enough to explain recent trends.

Events of the past two decades have provided little reason to doubt that runaway growth in top incomes has resulted in large part from increasing leverage in the “winners” positions, in tandem with growing competition to fill those positions. By every measure, markets have grown more competitive, and the most productive players have gained additional leverage since The Winner-Take-All Society’s publication in 1995.

What’s also clear is that the economic forces that have been causing the spread and intensification of winner-take-all markets have by no means run their course. We can expect continued growth in the intensity of competition on the buyers’ side for the best talent, and on the sellers’ side for the top positions.

In his widely discussed 2013 book, Capital in the Twenty-First Century, Thomas Piketty suggested yet another reason for rising inequality, which is the historical tendency for the rate of return on invested capital to exceed the overall growth rate for the economy. When that happens, he argues, wealth continues to concentrate in the hands of those who own the most capital. All things considered, then, it appears prudent to envision a future characterized by continued growth in income and wealth inequality—which is to say, a future in which chance events will become still more important.

Because the enormous prizes at stake in many arenas attract so many contestants, the winners will almost without exception be enormously talented and hardworking. But they will rarely be the most talented and hardworking people in the contestant pool. Even in contests in which luck plays only a minuscule role, winners will almost always be among the luckiest of all contestants.

Excerpted from Success and Luck: Good Fortune and the Myth of Meritocracy by Robert H. Frank. © 2016 Robert H. Frank. Published by Princeton University Press. Reprinted by permission.

2016 May 7