Emergence forces us to abandon simplistic reductionism as complex systems reveal behaviors that transcend their parts. From ant colonies to neural networks, understanding how novel properties arise from collective interactions reshapes our approach to problem-solving across disciplines.
Photo Credit: Rob GrzywinskiOriginally posted on March 8, 2023 on LinkedIn. Edited from original version.
The Reductionist Mindset
I remember thinking "I got this!" when studying physics in high school and early in my college days as a shiny new physics major: All matter breaks down into atoms which are made up of smaller and smaller components (protons, electrons, quarks, etc). End of story. Easy! I was a card carrying reductionist. I believed that any complex system could always be understood by simply reducing it down to its fundamental parts. If the engine on my car broke then I could just reduce it down to its set of components, figure out which one was broken and then fix it. If this process worked for my car engine then it should work for everything else!Then we got into electricity and magnetism and quantum mechanics and my reductionist understanding of the world came to a screeching halt. I'll admit that initially I didn't buy the fact that the universe could operate in a non-reducible way. I held my reductionist beliefs as long as I could, until they collapsed under the weight of their own inconsistencies. (So maybe not a screeching halt but it was jarring nevertheless!)So what is this other thing that exists if reductionism doesn't cover everything? And is it only available in the quantum realm? (Damn you Marvel for making me shudder as I write this!) Emergence!
Understanding Emergence
Emergence is where a system exhibits new properties or behaviors that cannot be predicted or explained by analyzing its individual components in isolation. There are systems that cannot simply be reduced down to their individual parts. Emergent systems can be understood only by studying their properties as a whole — the whole is greater (different?) than the sum of its parts. (We could dabble a bit in the related concepts of holism and separability as well but I'll leave that for the interested reader.)Emergence is all around us. From a colony of ants which can exhibit complex behaviors such as foraging and building nests (which are not directly predictable from the behavior of a single ant) to traffic jams on your favorite expressway (where traffic patterns cannot be explained by the individual drivers but rather from the traffic as a whole). Flocks of birds, social systems and even the stock market all exhibit emergent properties. (I've even experienced emergent behavior in complex software systems. Debugging these is a hoot!)
Two Flavors of Emergence
While not essential to understanding emergence, it's interesting to note that there are two types of emergence: weak and strong. Weak emergence occurs when the emergent property of the system is reducible to, but not predictable from, the properties and behavior of its individual components. The emergent property is a result of the interactions between the components, but it can be explained by examining the lower-level components. For example, the behavior of a flock of birds can be explained by examining the behavior of each individual bird and the interactions between them, but predicting the exact flight pattern of the flock is difficult.Strong emergence occurs when the emergent property of the system is completely unpredictable and cannot be explained by the properties and behavior of its individual parts. In this case, the emergent property is not reducible to the properties of the lower-level pieces. Life is an example of strong emergence. We can study the chemical and physical processes that make up living organisms, but the properties and behaviors of life such as reproduction, growth, and adaptation cannot be reduced down to or explained by these individual processes alone.
Experiencing Emergence
You can experience weak emergence for yourself! I recommend playing with John Conway's Game of Life or watching some of the fascinating videos of ant simulations by "Pezzza's Work". You can see how very simple rules can give rise to very complex and unexpected behaviors.Quick aside: I don't want to leave you with the notion that emergent properties are mathematically unsolvable like the "Halting Problem" is. They are simply difficult to predict or explain based solely on the properties of individual components. For example, emergent properties in cellular automata (e.g. the Game of Life) can be represented as graphs and then graph theory can be used to analyze some of the properties.
Why This Matters
I'm taking a moment to talk about emergence because neural networks exhibit weak emergence. There's going to be a moment when I'm describing how neural nets work and I simply say "it exhibits emergent properties". We can talk about and analyze those emergent properties but I'm not going to be able to break it down into a set of rules that describe why they exist. On the flip side, I also can't just start with a bunch of neurons and explain to you how the resulting emergent behavior comes to be.If you remember nothing else about this post, remember this: even if you can break a complex system down into its individual components, you may still not be able to predict or explain the emergent behavior of that system. I hope this helps provide some context when we get into neural nets (and other areas) where emergence is a key concept. As always, please leave a comment below if you have any thoughts or questions!(1,036 tokens)
The human experience of reality unfolds across multiple sensory channels, creating a rich tapestry of understanding that no single mode can capture alone. As AI systems evolve beyond text-only interactions, the integration of vision, sound, and other modalities mirrors our own multisensory world, promising deeper and more nuanced machine comprehension that enhances rather than diminishes each individual channel's power.17 January 2025
Mental models and their transmission form the foundation of human cognition and communication. By examining how we construct, manipulate, and share these internal representations, we gain crucial insights into both human intelligence and the emerging challenges of artificial minds.15 January 2025
