Lessons From a Year of TDD

Hello everyone, my name is Charanjit. I am working as a technical lead at Trantor.

I joined Trantor a year ago, in one of Trantor's projects where I was fortunate enough to see people willing to apply test driven development on what was going to become a rather complex project. I am leading one part of this project under a team named "The cloud team". This talk is the story of things we learned while practicing test driven development to build a micro-services based cloud with several third-party integrations.

Our cloud is built primarily with Node.js (express.js), uses GraphQL to communicate with clients as well as for inter-service communication, backed by Azure's CosmosDB (with its Mongo API), has Kafka for messaging queue, and Elasticsearch for complex search solutions. We have about 9 services till today, and 3 third party integrations. We are building an IOT-ish solution, and our cloud primarily serves the android application running on our custom hardware, along with some companion applications (for support and sales).

Before we get into the details of how we did TDD in our project, or of TDD itself, let's take a step back and work out some abstract thoughts. Abstract thoughts about what we are trying to accomplish with techniques like TDD. Actually, let's take all the steps back and go to the very beginning.

In the beginning, there were myths. Humans weren't always recording history. The myths were used to fill the plot holes in stories told by one generation to next. A common and very fascinating theme among the myths of different cultures is that of magic.

Allegedly there used to be some very special people who could use the mysterious powers. With flick of a wand they could create things out of thin air, they could put life into the inanimate, they could bend the reality itself to their will.

But those were of course myths. What we have now are developers.

Well, I argue that developers are in fact the wizards of the information age. We certainly meet the criteria:

• [X] Create things (business/products/tools) out of thin air (without any raw materials)
• [X] Put life into the inanimate (software is practically the soul of computer)
• [X] Bend reality to our will (/r/insidethesoftstone reality is what you want it to be)

Raw materials for the products a software developer creates are the thoughts. The third point here is the focus of this talk.

I am not kidding. I believe it is fair to say that every software is a universe of its own. It has its own rules it operates within, its own reality, which is at your disposal to do whatever with.

It is a living and breathing thing which must change and evolve and die as well.

Do anyone here watches Rick and Morty? It is a good show about a rockstar and a junior software developer. Try it sometime.

One of the episodes are about the microverse battery. The rockstar developer creates a miniature universe whose sole purpose is to provide power to his car. It is essentially a very complex car battery.

Funnier thing in that episode, the universe within the universe in the battery does the exact same thing. Creates a universe whose sole purpose is to power its host.

Software we write are not very different. We create entire universe to serve our needs, and it is rarely created in isolation. Usually our universe has a bunch of other, more complex universes wrapped around it.

We often start with a base universe of Operating system. The platform, programming language, the frameworks, they all make up for our initial "empty" worlds. They set up their layers of rules, on which then we write our own. Layers and layers of worlds.

Now, how do we populate this empty universe with the inhabitants which would do our bidding? Well, with our thoughts and prayers of course.

I am not kidding. But you get it, you aren't laughing. We literally send our thoughts, often prayers as well, formulated in some special form governed by some programming language to a compiler (or interpreter), which then inhabit and basically forms the universe of our software.

Asgard isn't a place. Asgard is its people.

That brings us to the corollary which takes us in the general direction of practices like Test Driven Development.

Correctness of software is inversely proportional to the client side product team.

Thoughts go wrong primarily in two ways:

1. They are wrong to begin with. Mostly caused by misunderstanding
2. Thoughts that go wrong during translation

We can create correct software if we can validate the correctness of thoughts that get into the system, and those that reside it subsequently.

This is what TDD essentially do.

The first point points to the red-green-refactor cycle. When doing TDD, you write down the "what" aspect of the unit/module/function you are going to write. It essentially allow you to pause, sit back, and think about what you are going to implement. How it should behave.

The second point points to the help we get from our well-written tests when we have to then make changes in our.

It was an interesting observation. Team seem to have a much higher appreciation for design patterns like dependency injection. We also observed a shift in PR review comments toward a focus on software design and architecture. Discussions we used to have to make our TDD approach more fruitful brought a cultural shift in the team.

Biggest resistance we faced was convincing developers that it's worth their while to write tests before they write code for their code. We ended up introducing an extra step in our PR review checklist: ensure there is a "test:" commit which only has test descriptions. This helped somewhat, but it took some time (and a lot of pair programming) till developers realized the benefits of this process.