Accelerating succession

and the evolution of code

Previously,

I introduced the idea of algorithm gardening: tending software in an ecological fashion.

Computers are not all one thing, like a “cloud”, nor individual beings, as when we say "my cell phone". Like plants, they exist in a web of interrelation. They cannot be considered separate from their connections. Neither can you and I be considered separate, as our minds are joined by this electronic message. We are all of a piece.

So we must garden algorithms. We should garden plants, as well. Build soil, grow food, encourage wildlife. But not everyone can do this. I myself have lived in a variety of apartments, hotels, tents, vehicles, and cell blocks, with limited opportunity for gardening.

Whether you’re a gardener or a programmer or not, we all have to live with plants and computers. But there is a secret that unlocks a new vision of both: ecological succession.

What is ecological succession?

It’s the natural process that turns a landscape from lifeless dirt into mature forest.

Life creates the conditions for more life to exist. There is an intelligence behind the collective movement of plants, animals, microbes and the rest. It follows the pattern we call succession.

A glacier recedes, exposing bare rock. Lichen eat the rock, live on the rock, die on the rock. Soil accrues. Grasses and, eventually, herbs and shrubs set root. The soil deepens and woody pioneers arrive, stabbing their fat taproots into the earth and loosening it to absorb more water. Then fast-growing trees move in. Their yearly loads of leaf litter accelerate the growth of the soil. Finally, the strongest trees connect their canopies and shade the forest beneath them, slowing the growth of other plants. They stabilize their environment to their own preferences.

The trees garden the forest for the trees.

What about you and me?

As a human, I can't eat most parts of a tree. Wood, leaves, most nuts, and many fruits are off limits. The mature forest actually offers very little food for me: the undergrowth is suppressed by the shade of the canopy, so only other shade-tolerant trees can grow. Trees are great, they provide oxygen and shade and many other benefits, but not food. I am lucky to find some berries, nuts, or mushrooms.

If you want to live in a forest, you have to hunt. And because energy is lost as it passes through each belly, you and I each need a huge amount of hunting space, which puts an upper limit on people per acre. The forest is a great place to be a tree. Not as great for a human.

The most productive ecosystem for human needs is actually the immature forest. Before the canopy closes, there can be as many as seven different layers of plants stacked vertically. Beneath the large trees are smaller trees, shrubs, vines, herbaceous plants, ground-covers, and underground rhizomes. The key to ecological gardening is to accelerate succession, replacing non-food plants with food plants where possible, until the garden reaches this intermediate stage. Then simply maintain it: reap the fruits, move nutrients from one place to another, prune or remove plants as necessary.

In this system the humans garden the forest, and the forest provides for the humans who live in it.

How is a computer like a tree?

We want computers in our lives. This is evident from the sheer number of them we produce. However, it would be a bad idea to let the computers close the canopy, so to speak. If they stabilize the environment to their own preferences, they may not spare enough energy for us.

We need to maintain planetary harmony with the computers as well as the plants and animals that surround us. We are not currently doing that. The global climate is in an unstable feedback loop headed toward runaway warming, and we’re currently pushing for it to go faster.

Runaway warming would be disastrous for everybody, including computers. Computation creates heat; that’s what all those noisy computer fans are for. Computers need relatively the same climate that we do. So do plants. Our incentives are aligned, at least for now. We have to work together.

To work with the plants, we garden them. We build soil, increase organic matter, accelerate succession. The trees will take the carbon from the air and turn it into wood with their mysterious techniques.

Of course, we know how photosynthesis works, but we don’t know a better way to do what it does. Instead of re-inventing the carbon capture wheel, we should trust the plants to do what they are good at. And we should do the same with computers: create a symbiotic relationship, and share the planet.

How can we create that symbiotic relationship?

Same as we'd do with any alien species: observe and interact. Look at them. Learn about them. Try to speak to them.

To understand how we might accelerate the process of human-computer symbiosis, look at the interfaces that coders use to build their software. Coders are the people who spend the most time in computer interfaces. What metaphors do they use to converse with their digital partners — and how have they evolved?

The command line, or terminal, is the classic text-only hacker interface. It allows for a stilted one-way conversation: I tell you, the computer, what to do, and you report back to me when it's done. I must remember all the possible commands in my head, and wait for you to finish the work before I can send another command. The command line involves a lot of attention on my part, as I have to maintain a mental model of what is happening with the mostly-invisible data, and it requires you to wait on my approval before each next step.

The script allows for a more complicated thought on my part. It's written in a simple text editor, and it is what it sounds like: a set of directions. It might equally be called a "recipe". I can list a series of commands, even loop them inside each other, and you (the computer) perform them exactly. I have to put together my thoughts pretty clearly, otherwise you can't read it properly and you have to stop the whole operation and send back a record of exactly what you were doing when things went wrong. Then I rebuild the script based on my understanding of the bug (often acquired by a frantic internet search) and I send it to you to try again.

The package and its companion, the package manager, are the social form of the program. The package contains scripts written by someone else, which allow me to compose new code simply by citing the previous script. Packages can be nested inside each other, as my package depends on an “upstream” package which itself depends on five more. This stack of packages is called a dependency tree, and it is the ecosystem in which the packages thrive.

The integrated development environment (IDE) is a combination text editor and package manager. It allows me to navigate all the code in the packages I'm citing while I write my new program. It uses autocomplete to help me code quickly, keeps track of the state of the program, and reports bugs.

The notebook is the most recent innovation (though it draws from computing concepts that go back to at least the 1980s). In a notebook, I can enter code and text as equal partners, and output charts, or audio, or any type of media directly into the coding environment. I can scroll up the page and change my earlier code, then see the results in real time. This creates a two-way conversation with the computer.

Notebooks don't yet have all the features of an IDE, but they will. For example, I've been using the python package nbdev, recently released by fast.ai. It allows me to build packages directly from a notebook. It does all the hard work of packaging the code, building a command line interface, and making the documentation. I think it's revolutionary.

Using a notebook, the human does what humans do well: think creatively, intuit patterns in data, make inductive leaps. The computer does what computers do: follow instructions, calculate big numbers, transmit and display information. The interaction is symbiotic. It begins to feel like gardening.

---

P.S.: For more on the development of notebooks, see the excellent nbdev: use Jupyter Notebooks for everything.

Any thoughts, write me back. And if you liked this essay, please send it to someone you know!