Nonlinear Systems Overview
Nonlinear systems theory is a modeling framework for describing nonlinear phenomena. In math and science, a nonlinear system, in contrast to a linear system, is a system whose output is not directly proportional to its input due to the interconnections and interdependencies within the system.
When we look around us in the natural world, we see very few straight lines. But the straight line is ubiquitous in the systems we engineer, from buildings to circuit boards, it appears to be the default position. One way to understand this is that the systems we engineer are based upon our scientific and mathematical understanding of the world, which have inevitably started by describing the simplest and most orderly type of systems, that is to say, those composed of linear forms and relations. From Euclid to Newton and on, science has been focused on the orderly systems of perfect squares, triangles and linear relations of cause and effect that can be encoded into beautifully compact equations. And thus, we describe the real world as a kind approximation to these perfect linear models. Linear basically means line, and a line is often understood as the shortest or most direct path from one point to another. To say there is a linear relationship between two things is to say there is a direct relationship of cause and effect between them. Let’s take an example of this. Say I am playing baseball and I hit a ball with a bat. If we make a model of this system, we see the energy inputted by my swinging the bat, will be directly proportional to the output of the ball’s momentum as it travels off in the opposite direction. This is a simplified model, but it illustrates the direct, or linear relationship between cause and effect, my swing the bat and the ball’s motion in response. There are of course many more examples of linear systems, particularly in physics.
The underlying logic to linear systems, in general, can be captured with a model that has only two simple rules, called the superposition principles, which states firstly that the output to the system will always be directly proportional to the input.1 So if there is a linear relationship between the amount of fuel I put in my car and how far it will go, well then if I put twice as much fuel in, it will go twice as far. And secondly, that if we add the inputs from two or more different systems together, then the output to this combined system will simply be the sum of the two outputs of the original systems. So say we have two tractor factories, each producing a million tractors a year. Well, if we merge them then we will get a factory that will produce 2 million tractors a year. Linear systems are deterministic, meaning that if we know their present state, we can then fully determine their past and future states. This can be seen by plotting a linear system on a graph where it will always be depicted as a straight line.
Although linear systems modeling has proven highly successful in many areas and is often a very good approximation, the reality is that we live in a world with ecosystems, economies, societies and many physical phenomena that are not governed by the superposition principles, and thus are what we call nonlinear. An example of non-linearity might be listening to two of your favorite pieces of music at the same time. Because there is a relationship of interference between them, the result of this experience will not be a simple equation of adding the enjoyment from listening to each independently. This illustrates how nonlinearity arises whenever there is some relationship between elements within the system that can be either synergistic, making the output to the system greater than the sum of its parts, or one of interference, making the output less than the sum of its individual components.
To illustrate this further, we could take an example of four workers producing clothing. In isolation, each seamstress can sew a given amount of clothes within a day. Now if we put them together, we might get one of three results; Firstly, they might not interact with each other very much, meaning we would simply have a linear system where the whole would be the sum of its parts. But equally likely, they might form some cooperative relationship that lets them each specialize in a particular function, making them more efficient as a whole. And thus, the output to the system would be greater than the sum of its parts due to these synergistic relations. Or inversely, they might start all talking with each other, getting little work done, and thus the total output would be less than the sum of the individual outputs due to these relations of interference. Nonlinearity can also arise from feedback loops whereby the same process is iterated with the output fed back as the input to the next cycle. A classical example is compound interest where at the end of each period the balance plus interest is fed back into a formula to compute the next cycle of interest accumulation.
Iterative functions are an important concept within nonlinear science and have been used to create a whole new type of geometry called fractal geometry, whereby iterating a simple function generates irregular organic looking patterns that can model many of geometric forms we see in nature, from the structure of seashells to the rugged formation of mountains. In these nonlinear systems, superposition fails, meaning one cannot break the system down into small sub-problems and add their solutions. We must consider a nonlinear problem in total. It is this need to approach nonlinear systems as a whole that is giving rise to new more holistic approaches to science that are developing under the canopy of complex systems.
Non-linearity in all its shapes and forms is at the heart of many of the 21-century challenges to science, as we try to extend the scientific framework beyond its dependency upon linear systems theory to finding new ways to embrace the complex world we live in on its own irregular and imperfect terms. There are many qualitative differences between how linear and nonlinear systems develop over time. Linear systems typically show smooth, regular motion in space and time that can be described in terms of well-behaved functions. Nonlinear systems, however, often show transitions from smooth motion to erratic and chaotic behavior. Chaos theory is the area of math that studies these bifurcations or phase transitions as systems goes from ordered equilibrium to periodic cycles and on to chaotic behavior.