Organising, decluttering, Dan-Sha-Ri - order trends for repositories

Whether it's "The Home Edit" or "Tidying up with Marie Kondo" - these popular Netflix series are all about professional and sustainable tidying. The organisation pros promise to turn chaos into a system in which we can keep order forever.

But what exactly is a system? A system is an organised and comprehensive compilation of individual parts in a specific area of knowledge or thought. It is any collection or set of individual parts that are related to each other. Professional organisers define what makes a good system: form and function. A good system saves time. You can find individual parts quickly, know where to store individual parts and can quickly adapt the system to your own needs. And you save money. You have an overview of all the individual parts in the system. Because you can find them again, you don't have to buy new ones. A good system also takes up less space. And last but not least, you might also want to make a good impression on visitors with your system.

In a system, various quality features can be more or less pronounced depending on requirements. In a workshop, more emphasis is placed on expandability, as new tools and materials are constantly being added. In the arc42 quality model there are many more such quality features specifically for software systems.

Of course, a system is much more than a simple file repository. From the outside, however, we simply place individual parts in a box designed for this purpose. People do this in two ways:

• 1. Simply throw everything in. In this case, the users of the system are later dependent on search tools - such as the search in our IDE.
• 2. You think about a structure and organise everything into this structure again and again. This also enables us to navigate logically in order to find something again.

Even if we don't consciously think about it, we tend to structure our repos in some way. Structures that I have encountered in repos are folders structured by file types, by components of a particular framework - in the case of Angular, for example, this would be pipes and directives - or folders structured by features. The latter are often advertised with metaphors such as "Screaming Architecture". The term "screaming architecture" is used when you can recognise at first glance what the core of a new project does and what it is about.

All of the above structures have their advantages and disadvantages. However, science has shown that our brain tends to break down blocks of knowledge into packages and create objective schemas that represent the real world. It has also been shown that we activate the same areas of the brain when navigating virtually through a file system as we do when navigating in the real world, for example when we are once again looking for the peeler in the kitchen. When searching by entering a search parameter, on the other hand, it is mainly left lateralised, mostly frontal activations that are observed, which are associated with linguistic processes (Broca's area). The navigation process still works very well even in people with severe language problems following brain damage. In order to be able to navigate through a system in a meaningful way, a good and understandable structure is required.

But unfortunately, we often fail to maintain an appropriate structure, neither in the home nor in our digital systems. Do you have a box at home that you don't know exactly what's in it, but it could actually be anything? Not just one? We often have boxes like this in our repos. Developers spend more than 50 per cent of their time trying to understand programmes. But a real system with form and function should be easy to understand.

So what to do when the chaos is already there? Here the tidying professionals give us a few valuable steps. Tidy up and let's go!

Step 1: Define a goal

Tidying up is annoying, takes time and there are always better things to do. That's why we need to make time for it and find good reasons to motivate everyone involved - including ourselves. In IT projects, this often includes project managers, customers and the rest of the development team. But still ask yourself: why am I actually tidying up here? If I have managed my personal chaos well so far and have not disturbed anyone, then I may even be investing time and money unnecessarily. Concrete goals of refactoring could be to save time and money in the future, to extract components for reuse or to avoid errors.

Step 2: Sort out

You have decided to take the next step. This is about getting rid of anything you don't need or that isn't functional. A popular Japanese method here is Dan-Sha-Ri (断 (dan): cut off, 捨 (sha): throw away, 離 (ri): let go), which stands for minimalism and simplicity. Marie Kondo would say to every item: ask yourself whether it makes you happy. Anything you answer no to will be discarded. If you now go through your repo with your team and do this, there might not be any repo left. So that doesn't quite work. Nevertheless, there are definitely unused files, commented-out functions, unused classes, unnecessary comments, etc. in our repos that can only increase their value by being deleted. Sometimes these are recognised as code smells during development, but sometimes they have been lying around for longer. That's why we need to clean up regularly instead of just continuing to develop.

But there is another very special category: sentimentalities. Of course, there are individual pieces or entire boxes that are hard to part with. You've spent a lot of time with them. You have special memories of the time you spent with them. You simply can't part with them. That's okay. Put them in their own box and put them somewhere in the basement.

Step 3: Categorise

What remains, we want to categorise in order to pack it into boxes. In our software systems, these categories are often referred to as modules. An architecture is modular if it has been broken down in such a way that coherent units are created at all levels of abstraction, from which knowledge units - chunks - can be formed. It is an advantage if the proportions of our categories are balanced. Let's say I want to categorise my bathroom utensils. What would I categorise then?

Most architects can think of many basic principles here. Certainly the Single Responsibility Principle, Separation of Concerns and the Component Cohesion Principles. In addition, measures for determining connections and dependencies within and between individual categories:

• 1. Cohesion: a measure of how well the individual parts of a box fit together. Are they individual parts that are primarily focussed on the respective purpose?
• 2. Coupling: a measure of how strongly two categories (and by categories we mean: methods, functions, classes, modules, etc.) are dependent on each other.

In the following, I will therefore only discuss two other concepts. An important measure for determining the optimal coupling is instability. The value of instability is made up of the following two values:

• 1. Afferent coupling: how many other boxes require the contents of a particular box to function?
• 2. Efferent coupling: how many other boxes does this box need in order to function?

The second measure is abstraction. Abstractions provide interfaces that allow the code to be tested and modified by adding elements rather than making major changes.

• 1. Abstract: How many abstract (non-concrete) interfaces and abstract keyword-labelled items or boxes do I have in a package?
• 2. Concrete: How many concrete ones do I have?

It is important to maintain a balance between sufficient abstraction and instability. If I abstract little, but my boxes are as stable as possible or each one stands on its own, I am in the "pain zone". In the code, for example, I would not abstract basic table functions, but would have to implement them anew each time. However, if I abstract too much, the code becomes unreadable and unusable, and I find myself in the "zone of uselessness".

Another measure is connascence, a generalisation of coupling and cohesion. Connascence is made up of three properties:

• 1. Strength: there are different forms here, the strength of which varies.
• 2. Degree: How many dependents are there?
• 3. Locality: How far apart are they from each other?

How easily a constraint can be found and refactorised depends on its form. Static forms of constraints are easy to find using static code analysis, while dynamic forms are much more difficult to refactor. Race conditions or unfavourable loading sequences lead to errors that often cannot be assigned directly. Such temporal dependencies are not recognisable to the naked eye. To eliminate them, I could, for example, categorise my bathroom utensils according to when they are used (such as a box of "morning routine"). Categorisation generally depends heavily on individual needs and is not constant.

Step 4: Pack into boxes

Now that you have categorised everything as desired, what type of boxes do you pack the categories into and how do you stack the boxes to make the best use of the available space? The type of box you put something in depends on how visible or easily accessible you want the content to be for all users of the repo

For example, a library with all user interface components should be easy to find and easily accessible for everyone.

However, we don't want to have all the boxes in the room, but rather stack them as sensibly as possible on our shelves. The order and stacking height are decisive for the complexity. On the one hand, the complexity of the system depends on the number of boxes and the parts they contain. This can often not be influenced. However, the stacking height and the width of the structure are also decisive. These become larger and larger over time. Dan-Sha-Ri has a simple "one-touch rule" for this: everything in the system should be accessible with two hand movements.

We can not only stack boxes or place them next to each other so that each box is part of the whole. We can also nest crates inside each other and pack small crates into large crates as categorical subordination.

Step 5: Labelling

Now that we've sorted everything neatly, we still need to label it so that we know what's in the boxes later.

Good names are pronounceable, searchable and consistent in the domain. It is important to choose and keep only one word per specialised or technical concept. Further good tips on naming can be found in chapter 4 of the book Clean Code by Robert C. Martin.

Step 6: Retain and maintain

This brings us to the last step. How can we now maintain our new order?

In 1969, Meir M. Lehman conducted an empirical study at IBM to improve the efficiency of the company's programming. The study received little attention within the company and had no impact on development practice. Subsequently, however, a number of researchers have attempted to fathom the laws of programme development dynamics. Not all laws could be clearly confirmed and some laws could only be observed after the commercialisation of the systems studied. The scientist Pirzada came to the conclusion that commercial pressure limits the growth of the software and promotes the deterioration of the system. Furthermore, the laws only apply to E-type systems. E-type (evolutionary) programmes mirror human processes or a part of the real world. This type of programme attempts to solve a task that involves humans or the real world. Here is an extract:

• I. Law of constant change: A type E system must be constantly adapted, otherwise it will become less and less satisfactory in practice.
• II. law of increasing complexity: When a type E system is changed, its complexity increases and it becomes more difficult to develop it further unless work is done to maintain or reduce the complexity.
• III Law of Familiarity Maintenance: In general, the incremental growth (growth rate trend) of Type E systems is slowed by the need to maintain familiarity.
• VI. law of feedback system: E-type evolutionary processes are multi-stage, multi-loop, feedback systems with multiple participants.

Conclusion

It can be deduced from this that a system cannot simply be built or tidied up once and then left alone. A system must be constantly tidied up and important design decisions must be made again and again. If these decisions are delayed and individual parts are thrown in at random, the result is disorganisation. On the one hand, we need permanently installed "clutter detectors" in our local environment and in our pipelines: formatter, linter, tests. On the other hand, we need to schedule time to clean up.

Dan-Sha-Ri has a few more tips for you:

• Make tidying up a habit.
• If you build something new, think about whether you already have it, and if so, replace the old one completely.
• Think carefully about whether you really need the new solution or the new part.

Nobody expects us to build systems that tell stories. But only form and function turn chaos into a system. What's more, a good order that works for all users of the system should not be determined by a single person. Good decisions are only made as a team.

Would you like to find out more about exciting topics from the world of adesso? Then take a look at our previous blog posts.