Don’t rely on memory: knowledge management for engineering teams

Good teams are always improving their processes and the way they code. In this constant pursuit of betterment, we often forget how and why we’ve got to where we are. This text talks about some of the most usual expressions of this forgetfulness and gives you some ideas as to how to solve them.

In our paths as developers, we learn many new things every day - not only about tools, languages, and frameworks. We learn which patterns work better in each scenario, we learn how to research, how to investigate, how to see through layers. Who hasn’t looked at a terribly written line deeply buried in the code only to find that the maker of such abomination was no other than yourself?

We also forget a lot. And one of the most important things we forget is context. For every bad decision we make on software, there is some context behind it that could justify the reason we did it - or maybe it doesn’t justify it at all. It's very common for startups or young companies to turn down code quality in exchange for faster deliveries, for instance, be it to prove a concept or to understand their clients better. And this is not a bad thing. What's bad is to lose track of why you chose to walk down these paths and take those risks by making certain decisions.

Recording documentation on solutions without context is insipid and may endanger the team. Engineers feel uneasy and riddled with questions: is this overengineered or under-engineered? Are the estimates too tight or too loose? Why was this the chosen route rather than the alternative? By losing context you cannot judge correctly how far away from the right path you were when you made that decision.

If it's hard enough to maintain documentation of what you have implemented, adding context turns it into a nightmare. Your docs are going to be even more volatile - why is it bad, though?

‍

Having a healthy relationship with documentation

Documentation can be used to tackle many problems, from explaining complex business logic to stating the dos when extending functionality. Very often good documentation can be very helpful to distribute knowledge in your team. Registering takes time and plenty of processes to rightly maintain documents up-to-date.

Great mature projects invest much of their time to document every routine and logic, and rightfully so. Now take early products, for example. In the early stages, business rules and architecture changes frequently, as no one knows for sure how the product will evolve. In this case, it's even harder to maintain documentation. Once the job is done, it needs to be written over again.

It’s important to keep in mind that there is a sparse range of documentation types, each one with their advantages and disadvantages depending on the context. We won’t dwell on the multiple types of documentation here, this video does it much better: watch this talk from Daniele Procida about What nobody tells about documentation.

Numerous types of documentation are strongly tied to the code base - this makes them high in upkeep, because they need to grow with the code. As documentation is hard to maintain, be smart. Prioritize what to document and the level of detail you want to track.

To deter constant updates, register the context of its development. With clear sight of technical decisions’ the backdrop, our documentation tells us a far more vivid story and can help the readers to really understand why things look the way they do.

‍

Looking back may help to figure out the path forward

Software is a living entity. The context behind its features and technical decisions is mutable. Context mutability is one of the biggest reasons for a team to forget why a technical decision was taken, and question the judgment of the people who took it. Trust your fellow workers: with the existing software architecture and business requirements at the time, they most likely made the best decision possible with the knowledge they had by that time. By examining the past context, one may even discover that part of it remains true, so the technical decision taken is still valid or optimal.

Tracking context is very challenging, nonetheless. With no method, documentation becomes easily outdated, a change for the worse.

But what if the documentation was some sort of changelog instead of a hard map of your application? With context in place, understanding isolated decisions are supported by an explanation of the scenario they were made in. Resorting to this strategy will lead to concise and high-level documentation that conveys the main pillars of your software. In this changelog-inspired doc, any team member can find details by simply navigating to a decision in the changelog, and understand not only how it works but also why it works.

For that, we suggest Architecture Decision Records (ADRs): a simple, dated, and immutable record that states both context and description of one atomic technical decision, highlighting the then assumptions and projected risks. With such accentuations, previous decision-making becomes more transparent, clarifying past solutions.

ADRs are very effective tools to help people understanding the story behind your software, and with more visibility of code’s structure we can:

• Understand the value of logic pieces that may seem faulty;
• Spot past mistakes patterns so as to make better decisions;
• Enable cleaner improvements with fewer reworks.

If you want to get more details on how to create and manage ADRs, you can see how companies like GitHub and Spotify do it. There’s also this open-source project that aggregates a lot of knowledge around ADRs.

Of course, they are no silver bullet. Context tracking in the long term leads to a hefty list of ADRs, so new team members may have a lengthy work to mow down their full list and understand the project more holistically. This brings us to our next topic.

‍

ADRs are valuable reminders, but awful teachers

The farthest you go with registering architectural decisions the harder it will be for newcomers to the project to consume them in full. That’s why it’s still recommended to maintain general business-oriented and processes documentation. High-level and stable forms focused on broad-stroke technical registration will allow for faster adaptations.

ADRs come to help you understand the deepest levels with a full picture of the scenario at the time then. That said, they might not even reflect the current scenario anymore, and decisions there documented may be outdated when compared to the codebase, but you will be able to see the path that took you to where you are now.

Understanding big codebases takes time. Hands-on work may be the best way to get familiarized, even without visibility of every detail.

A handy shortcut for someone to really understand what they’re looking at is through training sessions. Digging into more complex parts of the code can be a real ordeal. Training should be much more than isolated events, covering how something works. Code walkthroughs, code dojos, and presentations work wonders when part of the team’s routine. Progressive repetition builds technical muscle and reminds your team of how your application works.

‍

Repeat yourself: repetition creates muscle memory

One could argue that having very detailed documentation could even be prejudicial, especially for beginners. If they’re too extensive or complex, they could make people hesitant to handle some part of the code solely because the documentation made the code look scary.

Repetition helps us understand things better. We build the confidence to go to infinity and beyond and back to improve your software.

By making training sessions lighter and periodic you open space for people to ask questions and even suggest improvements, making people feel engaged and relevant to the project faster.

Use your own codebase as training material. Open your code during a  session and explain what every part does. Seeing the code will help people overcome the fear of touching it.

Although training is an incredibly valuable tool, ponder the number of hours it requires from the presenters on preparation. If you’re a Tech Lead (TL) or a more senior engineer you may be asking yourself if you'll find time to prepare for all this. That leads to our next topic: knowledge generation must not be centralized.

‍

Distributed knowledge requires distributed content generation

Creating content is also a terrific way to learn. Building presentations, writing documentation, pair-programming; all these practices make the content creator consolidate their knowledge. This is true especially because teaching requires you to rephrase or even rethink some of the concepts you already know.

If content generation is distributed, the Tech Lead’s work will be a lot more strategic by pointing team members in the right direction while not centralizing the execution.

Trusting people in your team to train each other also builds confidence and pushes team building further by creating an environment where it's part of the daily work to share knowledge creates healthier and self-improving teams.

Establishing such an environment isn’t a walk in the park. Product roadmaps are constant reminders of our priorities. It's very challenging to ensure everyone's learning while development speed is as swift as the product requires.

The best strategy I found so far is to include knowledge generation in the new things we deliver. Making training and documentation part of the shipping process is key for not losing focus and keep learning.

‍

Processualize learning: day-to-day must bring enlightenment to all

Writing documentation should be part of your development process. Registering and sharing knowledge is fundamental to keep everyone on the same page.

To assure proper documentation, write it before implementing code. RFCs are good examples of it. That will not only make you think more before getting attached to a design but will also expose the whole team to big technical decisions.

Sharing with the team additionally helps everyone know where the new features are going before they’re implemented.

This can be a lightweight process with minimal time investment. The main objective is less to make everybody understand how features are designed, and more to give a general idea of how and why the architecture is going in a specific direction.

By sharing, you not only make everybody on the team more aware of how new things are designed but also make them have contact with big architectural decisions and learn how to make them by comparison.

New complex architectures need more documentation. Guarantee people have the sources to consult when they need it. But that may not be enough: take the next step and do code walkthroughs so they can have the confidence to put their hands on code.

When engineers are sidelined and don’t work on code they will always be a little bit scared. It will surely impact positively on their confidence and also their estimates.

‍

Conclusion

We believe knowledge management is of extreme importance when building world-class teams, and great teams make great products. When managed expertly, the strategic unfoldings of better documenting and frequent training are of the greatest importance. Here’s a compilation of some of our advice to you:

• Memory is not reliable so you have to document decisions and repeat them periodically so the team doesn’t forget;
• Documentation gets outdated, but that’s ok if it tells a story instead of being a hard source of the current state of your application;
• Having documentation that actually states how your app works is necessary in many cases but be careful on how detailed it is and how hard it is to maintain it;
• Sharing knowledge must be a shared responsibility, everyone learns by teaching as well; and
• Knowledge sharing must be done every day and must be part of your development process.

‍