Architecture

One of the many paradoxes of human creativity is that it seems to benefit from constraints. Although we [perceive] the imagination as requiring total freedom, the reality of the creative process is that it’s often entangled with strict conventions and formal requirements. Pop songs have choruses and refrains; symphonies have four movements; plays have five acts; painters still rely on the tropes of portraiture. Jonah Lehrer Constraints can seem like the last thing you’d want for a creative project, but they’re actually beneficial when it comes to doing good work. Anyone who has written knows the hurdle of the blank page, and the feeling that comes from being paralysed by innumerable choices. By restricting the choices available to us, paralysis disappears and we can focus on the task at hand. Don’t believe me? Consider the task of writing an entire story in less than six words. It seems impossible! How can you develop characters, expand a plot, and tell a tale in six words? Here’s how Hemingway did it: ...

The Research and Development department would appear to give equal weight to both research and development by virtue of naming, yet there are few engineers who would claim that research is a primary responsibility of their job. Between working on product features, planning sprints, fixing bugs, and attending meetings there is little time left for research. But what is the cost of foregoing research in favour of development? In this article, I consider the economic cost of ignoring research and argue that focusing solely on development is harmful to both product and engineering. ...

While microservices have become the defacto architectural pattern for building modern systems, they come with their own set of challenges. Key among them is ensuring application data consistency when data is spread over multiple databases. This article discusses one solution to this problem, distributed sagas. Distributed sagas provide consistency guarantees for systems spanning multiple databases in the face of failure using a clear and manageable implementation pattern. A saga is highly related to a finite state machines that moves consistently through a set of states, ensuring at each step that some conditions hold. ...

Complex systems exhibit unexpected behavior. — John Gall, The Systems Bible One reaction to the myriad failure cases we encounter with distributed systems is to add more testing. Unfortunately, testing is a best-effort verification of system correctness — we simply cannot predict the failure cases that will happen in production. What’s more, any environment that we use to verify system behaviour is — at best — a pale imitation of our production environment. An alternative to adding more tests is to add more monitoring and alerting so we know as soon as possible that a system shows signs of degradation. Unfortunately, monitoring and alerting suffers from the same faults as testing — we monitor for behaviour that is predictable in nature or that we have experienced in the past. Together, monitoring and testing try to enumerate all possible permutations of partial and total system failure. Unfortunately, they only provide a simulation of how you expect a system to actually function in production. ...

Early computers were described using mathematical notation and theoretical constructions, which were then translated by enterprising machinists into custom built calculation engines. The first computer programmers using these machines built applications directly in hardware by plugging together wires and sockets in varying configurations. Even with these short-comings, the value of general purpose computing machines was evident. As companies began to realize the utility of computing, they saw computing as a means for differentiating themselves from their competitors. ...

A principle is a concept or value that is a guide for behaviour or evaluation. — Wikipedia This post presents a principled approach to architecture. These principles specify what I believe is important about architecture, without diving into any details about how an architect should work. No matter how an architect works day-to-day, by following principles, you can be sure you are providing value in the right areas. As usual, this post is personal opinion, and I’m interested in hearing any differing or similar opinions in the comments. ...

One proposed benefit of following a microservice architecture is that each service can be developed, released, and supported independently. In theory this allows development teams to work with less coordination and less overhead, leading to faster development times. In practice, this is difficult to achieve without some guidelines that make it work. The book The Tao of Microservices provides two such guidelines — transport independence, and pattern matching — that create an environment allowing you to compose services. Service composition is the holy grail of microservices, allowing you to build progressively more complex functionality safely and efficiently. This post discusses how transport independence and pattern matching allow you to compose services, what service composition means in terms of functions, and how you can use this composition to build complex services additively. ...

In most organizations, software development is split between two groups — product management and engineering. Product management is focused on building the right system. This requires product managers to meet with users, try and understand their needs, and develop solutions that meet those needs. To actually develop the solution, product managers must involve engineers who are focused on building the system right — writing the code, making it stable, and supporting it in production. Unfortunately, there exists an uncomfortable gap between product managers and engineers that forces the product manager to explain features in terms that an engineer can implement, and that forces the engineers to explain technological constraints to product managers in terms that a product manager can understand. To bridge this gap requires careful and considerate communication. It requires empathy from both product managers and engineers. ...

We all know that a page that loads in three seconds will provide a negative user experience. But does having every page load in less than one second make a meaningful difference? How can you tell? When is enough … enough? Non-functional requirements such as performance are often not precise enough to be specified as discrete numbers. Rather, they work on a sliding scale of acceptability. Would slightly better performance be significantly more valuable from a market perspective? Would significantly better performance be just slightly more expensive to implement? When dealing with performance, usability, reliability and so forth, we often end up in a difficult trade-off analysis. ...

Bertrand Meyer first introduces the principle of Command Query Separation in his book Object-Oriented Software Construction. The principle states that a well designed object should have methods that are either commands or queries. A command changes the state of an object, but does not return any data, while a query returns data and does not change any state. By dividing methods into these two categories, you will have a better understanding of what does, and what does not, change the state of your system. ...