By Geoff Mulgan

The organization of thought as a series of nested loops, each encompassing the others, is a general phenomenon. It can be found in the ways in which intelligence is organized in our bodies and within the groups we’re part of. The efficient deployment of energy to intelligence depends on a similar logical hierarchy that takes us from the automatic and mindless (which require little energy) to the intensely mindful (which require a lot). With each step up from raw data, through information to knowledge, judgment, and wisdom, quantity is more integrated with qualitative judgment, intelligence becomes less routine and harder to automate, and crucially, the nature of thought becomes less universal and more context bound. With each step up the ladder, more energy and labor are required.

What Is the Organization in Self-Organization?

Seeing large-scale thought through this lens provides useful insights into the idea of self-organization. The popularity of this idea reflects the twentieth- century experience of the limits of centralized, hierarchical organizations— even if the world is still dominated by them, from Walmart and Google to the People’s Liberation Army and Indian Railways. We know that a central intelligence simply can’t know enough, or respond enough, to plan and manage large, complex systems.

Widely distributed networks offer an alternative. As with the Internet, each link or node can act autonomously, and each part of the network can be a fractal, self-similar on multiple scales.

There are obvious parallels in human systems. The term stigmergy has been coined to describe the ways in which communities—such as Wikipedia editors or open-software programmers—pass tasks around in the form of challenges until they find a volunteer, a clear example of a community organizing itself without the need for hierarchy.

Friedrich Hayek gave eloquent descriptions of the virtues of self- organization, and counterposed the distributed wisdom of the network to the centralized and hierarchical wisdom of science or the state: “It is almost heresy to suggest that scientific knowledge is not the sum of all knowledge. But there is a body of very important but unorganized knowledge: the knowledge of the particular circumstances of time and place. Practically everyone has some advantage over all others because he possesses unique information of which beneficial use might be made, but of which use can be made only if the decisions depending on it are left to him or are made with his . . . cooperation.” More recently, Frederick Laloux wrote the following lines, capturing a widely held conventional wisdom: “Life in all its evolutionary wisdom, manages ecosystems of unfathomable beauty, ever evolving towards wholeness, complexity and consciousness. Change in nature happens everywhere, all the time, in a self-organizing urge that comes from every cell and every organism, with no need for central command and control to give orders or pull levers.” Here we find the twenty-first- century version of the late nineteenth-century notion of the élan vital, a mystical property to be found in all things.

It’s an appealing view. But self-organization is not an altogether-coherent concept and has often turned out to be misleading as a guide to collective intelligence. It obscures the work involved in organization and in particular the hard work involved in high-dimensional choices. If you look in detail at any real example—from the family camping trip to the operation of the Internet, open-source software to everyday markets, these are only self-organizing if you look from far away. Look more closely and different patterns emerge. You quickly find some key shapers—like the designers of underlying protocols, or the people setting the rules for trading. There are certainly some patterns of emergence. Many ideas may be tried and tested before only a few successful ones survive and spread. To put it in the terms of network science, the most useful links survive and are reinforced; the less useful ones wither. The community decides collectively which ones are useful. Yet on closer inspection, there turn out to be concentrations of power and influence even in the most decentralized communities, and when there’s a crisis, networks tend to create temporary hierarchies—or at least the successful ones do—to speed up decision making. As I will show, almost all lasting examples of social coordination combine some elements of hierarchy, solidarity, and individualism.

From a sufficient distance, almost anything can appear self-organizing, as variations blur into bigger patterns. But from close-up, what is apparent is the degree of labor, choice, and chance that determines the difference between success and failure. The self-organization in any network turns out to be more precisely a distribution of degrees of organization.

Cognitive Economics

The more detailed study of apparently self-organizing groups points toward what could be called a cognitive economics: the view of thought as involving inputs and outputs, costs and trade-offs. This perspective is now familiar in the evolutionary analysis of the human brain that has studied how the advantages of an energy-hungry brain, which uses a quarter of all energy compared to a tenth in most other species, outweighed the costs (including the costs of a prolonged childhood, as children are born long before they’re ready to survive on their own, partly an effect of their large head size).

Within a group or organization, similar economic considerations play their part. Too much thought, or too much of the wrong kind of thought, can be costly. A tribe that sits around dreaming up ever more elaborate myths may be easy pickings for a neighboring one more focused on making spears. A city made up only of monks and theologians will be too. A company transfixed by endless strategy reviews will be beaten in the marketplace by another business focused on making a better product.

Every thought means another thought is unthought. So we need to understand intelligence as bounded by constraints. Cognition, memory, and imagination depend on scarce resources. They can be grown through use and exercise, and amplified by technologies. But they are never limitless.

This is apparent in chaotic or impoverished lives, where people simply have little spare mental energy beyond what’s needed for survival. As a result, they often make worse choices (with IQ falling by well over ten points during periods of intense stress—one of the less obvious costs of poverty). But all of us in daily life also have to decide how much effort to devote to different tasks—more for shopping or your job; more time finding the ideal spouse, career or holiday, frequently with options disappearing the longer you take.

So we benefit from some types of decision becoming automatic and energy free, and using what Kahneman called System 1 and 2. Walking, eating, and driving are examples that over time become automatic. With the passage of time, we pass many more skills from the difficult to the easy by internalizing them. We think without thinking—how and when to breathe, instinctive responses to danger, or actions learned in childhood like how to swim. We become more automatically good at playing a tune on the piano, kicking a football, or riding a bicycle. Learning is hard work, but once we’ve learned the skill, we can do these things without much thought. There are parallels for organizations that struggle to develop new norms and heuristics that then become almost automatic—or literally so when supported by algorithms. This is why so much effort is put into induction, training, and inculcating a standardized method.

Life feels manageable when there is a rough balance between cognitive capacity and cognitive tasks. We can cope if both grow in tandem. But if the tasks outgrow the capacity, we feel incapable. Similarly, we’re in balance if the resources we devote to thinking are proportionate to the environment we’re in. The brain takes energy that would otherwise be used for physical tasks like moving around. In some cases, evolution must have gone too far and produced highly intelligent people who were too weak to cope with the threats they faced. What counts as proportionate depends on the nature of the tasks and especially how much time is a constraint. Some kinds of thought require a lot of time, while others can be instantaneous. Flying aircraft, fighting battles and responding to attack, and flash trading with automated algorithmic responses are all examples of quick thought. They work because they have relatively few variables or dimensions, and some simple principles can govern responses.

Compare personal therapy to work out how to change your life, a multistakeholder strategy around a new mine being built in an area lived in by aboriginal nomads, or the creation of a new genre of music. All these require by their nature a lot of time; they are complex and multilayered. They call out for many options to be explored, before people can feel as well as logically determine which one should be chosen.

These are much more costly exercises in intelligence. But they happen because of their value and because the costs of not doing them are higher.

Here we see a more common pattern. The more dimensional any choice is, the more work is needed to think it through. If it is cognitively multidimensional, we may need many people and more disciplines to help us toward a viable solution. If it is socially dimensional, then there is no avoiding a good deal of talk, debate, and argument on the way to a solution that will be supported. And if the choice involves long feedback loops, where results come long after actions have been taken, there is the hard labor of observing what actually happens and distilling conclusions. The more dimensional the choice in these senses, the greater the investment of time and cognitive energy needed to make successful decisions.

Again, it is possible to overshoot: to analyze a problem too much or from too many angles, bring too many people into the conversation, or wait too long for perfect data and feedback rather than relying on rough-and-ready quicker proxies. All organizations struggle to find a good enough balance between their allocation of cognitive resources and the pressures of the environment they’re in. But the long-term trend of more complex societies is to require ever more mediation and intellectual labor of this kind.

This variety in types of intelligence, the costs they incur, and the value they generate (or preserve) gives some pointers to what a more developed cognitive economics might look like. It would have to go far beyond the simple frames of transaction costs, or traditional comparisons of hierarchies, markets, and networks. It would analyze the resources devoted to different components of intelligence and different ways of managing them—showing some of the trade-offs (for example, between algorithmic and human decision making) and how these might vary according to the environment. More complex and fast-changing environments would tend to require more investment in cognition. It would also analyze how organizations change shape in moments of crisis—for instance, moving to more explicit hierarchy, with less time to consult or discuss, or investing more in creativity in response to a fast-changing environment.

Economics has made significant progress in understanding the costs of finding information, such as in Herbert Simon’s theories of “satisficing,” which describe how we seek enough information to make a good enough decision. But it has surprisingly thin theories for understanding the costs of thought. Decision making is treated largely as an informational activity, not a cognitive one (though greater attention to concepts such as “organization capital” is a move in the right direction).

A more developed cognitive economics would also have to map the ways in which intelligence is embodied in things—the design of objects, cars, and planes—and in systems—water, telecommunications, and transport systems—in ways that save us the trouble of having to think.

It would need to address some of the surprising patterns of collective intelligence in the present, too, many of which run directly counter to conventional wisdom. For example, organizations and individuals appear to be investing a higher, not lower, proportion of their wealth and income in the management of intelligence in all its forms, particularly those operating in competitive environments. Digital technologies disguise this effect because they have dramatically lowered the costs of processing and memory. But this rising proportion of spending appears close to an iron law, and may be a hallmark of more advanced societies and economies. Much of the spending helps to orchestrate the three dimensions of collective intelligence: the social (handling multiple relationships), cognitive (handling multiple types of information and knowledge), and temporal (tracking the links between actions and results).

A related tendency is toward a more complex division of labor to organize advanced forms of collective intelligence. More specialized roles are emerging around memory, observation, analysis, creativity, or judgment, some with new names like SEO management or data mining. Again, this effect has been disguised by trends that appear to make it easier for anyone to be a pioneer and for teenagers to succeed at creating hugely wealthy new companies. Linked to this is a continuing growth in the numbers of intermediaries helping to find meaning in data or link useful knowledge to potential users. This trend has been disguised by the much-vaunted trends toward disintermediation that have cut out a traditional group of middlepersons, from travel agencies to bookshops. But another near iron law of recent decades—the rising share in employment of intermediary roles, and the related rise of megacompanies based on intermediary platforms such as Amazon or Airbnb—shows no signs of stopping. In each case, there appear to be higher returns to investment in tools for intelligence.

A cognitive economics might also illuminate some of the debates under way in education, as education systems grapple with how to prepare young people for a world and labor market full of smart machines able to perform many more mundane jobs. Schools have not yet adopted Jerome Bruner’s argument that the primary role of education is to “prepare students for the unforeseeable future.” Most prefer the transmission of knowledge—and in some cases rightly so, because many jobs do require deep pools of knowledge. But some education systems are concentrating more on generic abilities to learn, collaborate, and create alongside the transmission of knowledge, in part because the costs of acquiring these traits later on are much higher than the costs of accessing knowledge. The traits generally associated with innovation—high cognitive ability, high levels of task commitment, and high creativity—which were once thought to be the preserve of a small minority, may also be the ones needed in much higher proportions in groups seeking to be collectively intelligent.

An even more ambitious goal for cognitive economics would be to unravel one of the paradoxes that strikes anyone looking at creativity and the advance of knowledge. On the one hand, all ideas, information, and thoughts can be seen as expressions of a collective culture that finds vehicles—people or places that are ready to provide fertile soil for thoughts to ripen. This is why such similar ideas or inventions flower in many places at the same time. It is why, too, every genius who, seen from afar, appears wholly unique looks less exceptional when seen in the dense context of their time, surrounded by others with parallel ideas and methods. Viewed in this way, it is as odd to call the individual the sole author of their ideas as it is to credit the seed for the wonders of the flowers it produces. That some upbringings, places, and institutions make people far more creative and intelligent than others proves the absurdity of ascribing intelligence solely to genes or individual attributes.

But to stop there is also untenable. All thought requires work—a commitment of energy and time that might otherwise have been spent growing crops, raising children, or having a drink with friends. Anyone can choose whether to do that work or not, where to strike the balance between activity and inertia, engagement and indolence. So thought is always both collective and individual, both a manifestation of a wider network and something unique, both an emergent property of groups and a conscious choice by some individuals to devote their scarce time and resources. The interesting questions then center on how to understand the conditions for thought. How does any society or organization make it easier for individuals to be effective vehicles for thought, to reduce the costs and increase the benefits? Or to put it in noneconomic language, how can the collective sing through the individual, and vice versa?

The current state of understanding these dynamics is limited. We know something about clusters and milieus for innovation and thought. It’s clearly possible for the creative and intellectual capability of a place to grow quickly, and using a combination of geography, sociology, and economics, it is easy to describe the transformation of, say, Silicon Valley, Estonia, or Taiwan. Yet there are few reliable hypotheses that can make predictions, and many of the claims made in this area—for example, about what causes creativity—have not stood up to rigorous analysis. For now, this is a field with many interesting claims but not much solid knowledge.

Read an interview with the author at This View of life.

Excerpted from BIG MIND: How Collective Intelligence Can Change Our World by Geoff Mulgan. Copyright © 2018 by Princeton University Press. Reprinted by permission.

2017 December 22