Progress is a lake, not a line

When people describe progress, they often describe it in terms of a linear progression taking us from primitive to advanced — an idea or invention occurs as a singular event, and somewhere further down the line of time a new idea or invention completely replaces it, relegating the old to the annals of history. This viewpoint is exemplified by traditional worldviews that organize all beings according to a chain of evolution, sometimes called the “great chain of being” (or scala naturae). [Read More]

Why Systems Work So Well

In the book “Thinking in Systems”, Donella Meadows dedicates an entire chapter to explaining why functioning systems seem to work so well. In it, she recognizes three characteristics: resilience, self-organization, and hierarchy. Resilience We can use the standard definition from the Oxford English dictionary to describe resilience: re·sil·ience /rəˈzilyəns/ noun the capacity to recover quickly from difficulties; toughness. “the often remarkable resilience of so many British institutions” the ability of a substance or object to spring back into shape; elasticity. [Read More]

What complex systems can teach us about building software

As a software system scales it becomes sufficiently large that the number of working parts, coupled with the number of working programmers making changes on it, makes the behaviour of the system extremely difficult to reason about. This complexity is exacerbated by the transition of many organizations towards a microservice architecture, as exemplified by the so-called “death star” architecture, where each point in the circumference of the circle represents a microservice and the lines between services represent their interactions. [Read More]

Connecting Technology to the Needs of the Business

All healthy technology discussions should begin with business goals and use those goals as a reasonable set of guidelines to focus technology investment decisions. These business goals are best articulated from a deep understanding of what the company, the product team, or the marketing team want to accomplish. In a business-first model, technology is forced to balance the desires for technical effectiveness and efficiency with the operational needs of the business. [Read More]

The Most Important Criteria for Choosing a Programming Language

Use a Language Your Company Can Support

One of the recurring themes of any technology discussion is programming language. It doesn’t take much effort to find blog posts with dramatic headlines (and even more dramatic comments) about how shipping a new project with Haskell or Clojure or Elm improved someones job, marriage, and life. These success stories are posted by raving fans that have nothing but the best to say about their language of choice. A common thread running through these posts is that they are typically tied to building out new, greenfield projects. [Read More]

Making Modular Monoliths Work

Think Slices, not Layers

Microservices have become part of the software engineering cultural zeitgeist to the extent that alternative approaches to architecture and development are treated as somehow inferior. Given the challenges that running microservices present, I usually recommend beginning development of new projects and systems as a single deployable unit — the monolith. Sam Newman, in the book “Building Microservices”, agrees with this approach. He recommends leveraging microservices only if you can become convinced of the benefits for your system, not as a default for every project. [Read More]

Handling the rudder as an organization grows

In a shopping cart, the swivel wheels of the cart are set in the front, and the fixed wheels are set in the back. Now picture yourself pushing a shopping cart backwards. Almost naturally, you swivel the cart to move the front end to one side or the other before beginning to push the cart. Now picture yourself pushing a shopping cart backwards, on an ice rink. Here, the cart keeps sliding around even after you’ve stopped pushing it. [Read More]

Above-the-line and below-the-line

Making sense of the complex world of software

Engineering, for much of the twentieth century, was mainly about artifacts and inventions. Now, it’s increasingly about complex systems. As the airplane taxis to the gate, you access the Internet and check email with your PDA, linking the communication and transportation systems. At home, you recharge your plug-in hybrid vehicle, linking transportation to the electricity grid. At work, you develop code, commit it to a repository, run test cases, deploy to production, and monitor the result. [Read More]

Building Learning Communities

Leading software companies have discovered that developing capable technology is not enough to guarantee long-term success. To stay relevant, software leaders need to develop and support the repeatable systems necessary to develop and sustain knowledge and expertise. Many organizations have taken inspiration from Spotify’s culture and adopted the concept of a guild or community of practice to connect engineers throughout the organization and steer them towards common goals. As organizations adopt this model, what is often missing is a clear understanding of the purpose of communities of practice and a repeatable process for developing the communities to their fullest potential. [Read More]

Optimizing Processes Using a Design Structure Matrix

Complex processes may require collaboration and coordination of many components. We can model such processes using a Design Structure Matrix to represent information flow among the components, and then optimize the process to avoid rework and downtime. As a simplified example, consider a project with only two tasks: “A” and “B”, and a directed graph representing this system where a vertex represents a task and an edge represents the flow of information between tasks. [Read More]