Current software systems, like the ones powering any large website or app like Facebook, Amazon, Netflix, Google, Booking.com, etc are huge. They have become just like Operating Systems have always been. Large and complex.
Most websites and apps began development as small monoliths. But after growing, evolving and operating for 20+ years, the software powering most of our internet applications has become huge and complex. Much like our OS’s like Unix, Linux, MacOS or Windows.
The Challenges of Scale
Today, the software world is busy moving monolithic code and deployments to microservices to manage the whole system better.
In the e-commerce world, Netflix and Amazon were the first to pioneer microservices. Since then, the microservice architecture has been adopted by all large organisations as they face challenges of scale such as the speed of development, reliability, availability, and robustness of the system.
In this article, I want to compare this recent trend of microservices (as an architecture) to the traditional philosophy behind Unix, which has powered many of the world’s OS’s.
Microservices break down complex operations into smaller independent services. Unix has been doing the same for decades, with small command-line programs that do one thing well.
The Unix Philosophy
Ken Thompson pioneered the Unix Philosophy and it has lived on for 40 years. The thing is that it is not a formal way of doing things but has evolved naturally from experience. Just like microservices have evolved from monolithic applications.
As described by Wikipedia: “The Unix philosophy emphasizes building short, simple, clear, modular, and extendable code that can be easily maintained and repurposed by developers other than its creators. The philosophy is based on composable (as opposed to contextual) design.”.
The main ideas behind the Unix philosophy can be summarised as:-
- Write programs that do only one thing well
- Write programs in a way that the output of one can be the input of another, allowing them to be chained together.
- Write programs to handle text streams, as that is a universal interface
Microservices are supposed to be designed to be small and do one thing well. They are chained together in a complex system, and it is not uncommon for big organisations like Amazon and Google to have thousands of microservices chained together in calls. And the inputs they take are mostly all JSON these days. Sounds familiar? If this isn’t a perfect analogy, I don’t know what is?
Looking at four decades of Unix, here are 10 key takeaways that are equally applicable to the microservices world:-
- Modularity – Build simple solutions that do one task well, and expose them via a simple interface (command-line arguments, JSON, XML, etc). These separate modules should be replaceable and upgradable independently, without breaking the entire system around it.
- Connectedness – When building a large system, build many small systems connected to each other. The interconnections should be flexible and not constrained.
- Keep things simple – Only add complexity if you really need to. Resist bloat and always value simple solutions over complex ones. Write programs not just efficient in execution but also in maintenance.
- Do not surprise – When designing interfaces, do not surprise its users. If there are accepted defaults in the organization or industry, stick to them.
- Value programmer time more than machine time – Programmer time is more expensive than machine time. When it comes to optimisation and design of systems, keep that in mind.
- Automate Everything – Anytime you see a task being done more than once, it is a candidate for automation. Keep human involvement to a minimum, as it is expensive and error-prone. Automate generating client libraries and other wrapper code.
- Ship Fast – Build prototypes or proof of concept solutions. Ship them fast. And then make them better and robust. Build in small increments.
- Warn and Log Errors during failure – It is best if your program can handle failure without failing itself. But if it must fail, proper error logging and warnings should be triggered so that responsible people can do something about it.
- Diversity of Tools and Technologies – There is no one best way of doing something. Embrace different tools and technologies, as that will make your system flexible and not rigid. Embrace different languages and technologies.
- Grow and Evolve – With time, allow your systems to grow and extend to include new functionality, technologies or architecture.
All of the above lessons are equally applicable to the microservices world as they have been to Unix over the last four decades. If we keep the above in mind while making design decisions, we can avoid making the same mistakes we have always made while developing software.
As you have seen, there are strong parallels in the philosophy behind microservices and Unix, and we must not ignore the lessons of the last four decades as we build and deploy software in the decades to come.
