Bar-Yam: Balancing Scale with Complexity

Yaneer Bar-Yam, President of the New England Complex Systems Institute and author of Making Things Work: Solving Complex Problems in a Complex World (Knowledge Press, 2004)

You are a pioneer in the study of ‘Complex Systems’. Please describe this emerging field.
Complex Systems studies how relationships between the parts give rise to the collective behaviours of a system, and how the system interacts and forms relationships with its environment. Examples of complex systems include social systems, the human brain and weather patterns. Social systems arise out of relationships between people; the brain’s behaviours result from relationships between neurons; and weather patterns are formed by relationships between air flows. One of the exciting aspects of the study of Complex Systems is how relevant it is to understanding the world around us, and how it changes the way we think about the world.

You believe that some methods of thinking about complex systems are often, if not always, useful. What are they?
The approach that my colleagues and I have developed involves focusing on a few fundamental things: patterns in collective behaviour; a multi-scale perspective (i.e. the way different observers describe a system); the evolutionary process that creates complex systems; and the system’s goal-directed behaviours. Whether we are talking about biological molecules or corporations, these interwoven elements help us classify complex systems, recognize their functional capabilities and develop a context in which their strengths and weaknesses can be evaluated.

You believe that the more complex a system is, the more individuals matter. Please discuss.
The fact that people are working together more and interacting on a global scale is one of the main reasons that the world is so complex. The variety of ways people are interacting is increasing the diversity of behaviour taking place, and at the same time, the diversity of interactions is causing more of the details of what people do to matter to many other people.

When you are trying to solve complex problems, it is essential to recognize that individuals matter. When a problem is more complex than a single individual, the only way to solve it is to have people solve it together. However, they must be organized in a way that lets each individual matter. The traditional experience with organizing people is for large-scale problems that are not very complex---the need for many people arises because many individuals must do the same thing to achieve a large impact. A hierarchy works in these instances because it is designed to amplify what a single person knows and wants to achieve. However, hierarchical structures cannot perform complex tasks or solve complex problems.

Unfortunately, the way we as a society have been trying to solve most problems contributes greatly to their existence. We continue to respond to societal problems by centralizing authority and imposing the will of one person. To be successful, we need to employ complex networks of people to solve complex problems.

Describe the concepts of ‘emergence’ and ‘interdependence’.
These concepts are fundamental to Complex Systems. Emergence refers to the relationship between the details of a system and the larger view. When you focus on the small-scale details of a system or situation, you run the risk of missing the larger picture; but focusing solely on the large-scale view is not adequate either. In conventional views, the observer considers either the trees or the forest; but when you shift back and forth between seeing the trees and the forest, you can see which aspects of the trees are relevant to the forest, and vice versa. Emergence seeks to discover which details are important for the larger view, and which are not; and how collective properties arise from the properties of the parts. Emergence demands that we move between different perspectives.

Studying complex systems also helps us recognize and understand indirect effects that manifest interdependence. Many problems are difficult to solve because the causes and effects are not obviously related. Pushing on a complex system here often has effects over there because the parts are interdependent. Of course, this has become increasingly apparent in our efforts to solve societal problems and to avoid ecological disasters.

You have said that “Higher-complexity organisms have more behavioural options, which in turn enables them to make more right choices.” So complexity is good, right?
Put it this way: the only way to succeed in a complex environment is to be complex. The key is to match the complexity of the organism to the complexity of the environment. If the environment has many different conditions or situations, then the organism has to have many different responses that are appropriate under each of those conditions. If the environment is simple, the organism doesn’t need as many responses.

If the environment in which a corporation operates is very complex, many decisions must be made correctly for success to be achieved. These might include product choices, price decisions, investment choices, resource allocations, hiring policies and so on. If one company makes better choices than another, it will succeed and the other might go out of business. The difficulty occurs when the environment is much more complex than the organism; when this happens, complexity is a very negative thing for that organism and it is destined to fail.

Can you give an example of a company that has successfully matched its complexity to its environment?
Most people don’t know that Visa International is the largest corporation in the world, measured by transaction revenue. It has intentionally designed itself as a distributed organization: it is run and owned by its 21,000 member organizations -- banks and other credit card issuers. The effectiveness of the organization is evident in the pervasiveness of the ability to use VISA cards everywhere in the world, for so many different purposes. That has to do with the wide diversity of activities that can be undertaken by an organization that is only weakly centrally-coordinated; it centralizes only the functions that have to be centralized and allows as much flexibility as possible in the response of the system to localized needs.

How can one go about relating the nature of the problem to the nature of the solution?
This involves understanding the patterns that exist in the environment around you. By recognizing patterns, we can identify the structure of the problem, and then we can tackle the structure of the solution. Within the context of very highly-interacting systems, patterns allow us to recognize the key collective behaviours that are important to look at, and by identifying these, we can understand how to act.

You have said that there is a trade-off between complexity and scale and that the success of any organization depends on both complexity and scale. Please explain.
In general, larger-scale challenges should be met with larger-scale responses. Doing things on a large scale means doing the same thing many times over. The tradeoff between complexity and scale has to do with the understanding that if we want to do many different things then we cannot also do the same thing many times. In a manufacturing context, this is a very natural idea: if you want to mass produce the same item many times, you cannot at the same time produce many different items. In meeting the demands of the environment, an organization has to match the scale that is needed with the complexity that is needed. If what is needed is to make 100,000 copies of a product, the organization should be capable of producing that number of products; if what is needed is to make 1,000 different, customized products, then it should be capable of producing diversity. Basically, the organizational structure has to match the demands of the environment. This also applies to the effectiveness of a healthcare system or the military in achieving their goals.

What are ‘positive feedback loops’ and what is their significance to the economy?
Much of the economy works through positive feedback: if people buy more, more workers are hired, who in turn earn more money to buy more goods. The problem with positive feedback loops is that they can run in the opposite direction: if people buy less, fewer workers are needed, which means lower earnings and less purchasing. What stabilizes these loops has to do with the many different ways and places people can work, buy and sell, and interact with the market; more variety creates greater stability in the system as a whole. When there are many different ways for people to work, some will be going up while the others are going down, so the system as a whole doesn’t collapse.

The danger arises when too many people are doing the same thing, as when too many investors buy certain stocks, creating a bubble, or when too many people invest in mortgage-backed securities, as we recently saw. Having too much of a good thing is a common economic problem, but regulators often fail to recognize it. While free markets are a good thing, when they are too free, they self-destruct. For example, the SEC supports policies enabling short selling because amongst other things, it can help weed out the weak corporations and avoid market bubbles. This is fine as long as there aren’t too many short sellers. Even a strong corporation can’t survive if too many short sellers gang up on it. Some people call short sellers predators, and the comparison may not be too far off: having a few predators to eliminate weak and sick animals may be good for the herd, but too many can lead to extinction.

If too much of anything is not good, how do we prevent it?
Moderation is usually considered a personal virtue, but it is also an essential economic principle. Strong companies are a good thing; monopolies are bad. What is bad for markets is when one player, or a coordinated group, controls the movement of prices. Regulations can restrict the extent of short selling or rapid buying, rather than forbid or allow it.

You believe that the best way to coordinate people to perform complex tasks is to create an environment “where evolution can take place.” What does this entail?
The competition between firms is similar to the competition and selection process that exists in evolutionary processes in nature. Within organizations, traditionally, competition and reward systems have rewarded individual performance. In order to create a more effective process, we have to understand the dynamics of evolution, which enable us to work on team-based performance rather than just individual-based performance. All of this requires a fairly elaborate understanding of the evolutionary process in order to create planned environments for continuous improvement. Evolution is not easy to understand, so grasping this will be a major challenge for today’s leaders; but I believe it will become more and more integral to activities in the management context. This is a key area in our ongoing research.

What is the first step to tackling a complex problem?
The first step is to begin thinking about how the parts of a system affect each other. If you take one part away, how will the parts be affected and how will the rest of the system be affected? Consider the family or organization you are part of and ask questions like, ‘How strong are the dependencies between the parts?’ ‘What would happen if one part were taken away?’ and ‘Does it matter which part I take away?’ By asking these sorts of questions, we take an important first step towards understanding relationships and relatedness.

Looking ahead, are you optimistic about our collective ability to face complexity?
While the complexity of our society is often overwhelming, it also has the potential to create an increasingly protective and productive environment for each of us, linked to increasingly effective collective behaviour. The difficulties we face in providing the essential aspects of well being – education, health care, infrastructure and economic development – are happening largely because we don’t recognize the power of our complex collective. Once we gain this insight, these difficulties can surely be resolved.

Yaneer Bar-Yam is the president of the New England Complex Systems Institute and author of Making Things Work: Solving Complex Problems in a Complex World (Knowledge Press, 2004). He holds a PhD in Physics from MIT. Complex Systems is a new field of science studying how parts of a system give rise to the collective behaviours of the system, and how the system interacts with its environment. For more, visit necsi.edu.

[This article has been reprinted, with permission, from Rotman Management, the magazine of the University of Toronto's Rotman School of Management]