A really interesting study done by Facebook shows how bugs reported to code review got 70% fix rate! [Published in Communicattions of the ACM, August 2019]
"For classes of bugs intended for all or a wide variety of engineers on a given platform, we have gravitated toward a “diff time” deployment, where analyzers participate as bots in code review, making automatic comments when an engineer submits a code modification. Later, we recount a striking situation where the diff time deployment saw a 70% fix rate, where a more traditional “offline” or “batch” deployment (where bug lists are presented to engineers, outside their workflow) saw a 0% fix rate."
This study was writen based on their 2 years of experence by Dino Distefano, Manuel Fahndrich, Francesco Logozzo and Peter O'Hearn
Architecture violations fixed before the merge
Our customer got similar results. They used Softagram to detect dependency violations automaticallly. Softagram allows to create a rules to inform developers before merge. Report tells if they are to introduce any violation for the preferred software architecture. Read more how to do that from the help. The Softagram bot created "Impact Report" appears automatically to code review discussion field. It would tell clearly the violation location, and what rule is violated. See an example from our dogfood environment:
Softagram case study
In a large (1 million Lines of Code) embedded environment, our customer created set of rules to keep the software architecture from deviating. Complex codebase and mutli-site development team of over 100 developers requires automation. You need to keep the code maintainable, and protect parts of the code not used unnesessary to secure the long future of the code. During the first months after the rules had been set, Softagram-bot reported tens of violations to pull requests. Over 40 of the violations were fixed right away! Without the checks and automation to provide the information to the author before the merges, most of the violations would have been raising the technical debt in the code base. Fixes done later with larger refactoring would have used multiple times more resources.
See an example of how the Softagram-bot works e.g. in GitHub environment:
Benefits of informing the issues directly in the code review phase
Facebook study clearly shows that issuesfound by any static analysis are fixed with over 70% rate when reported in code review. The bug fix rate alone clearly a great benefit. The affect is however multiplied by other benefits. If you need to fix a bug, it is easily up to 10 to 100 faster to do right after it's found, before the merge. You don't have to follow any new process for the fix. There is no need for planning, no splitting of your mind and time between multipe tasks. You don't have to stop your existing work to go back on the case your might not even remember any more. No studying the area of code due it's the latest thing you did, and remember it well.
We all know the hussle if the bug, un-noticed, or not fixed, is found by the customer. All the escalations and planning and studying. It all would have been avoided would the finding been seen already before the code was even merged to main branch!
Dependency violations could be very dangerous. They are causing situations where they are no more fixable without a large refactoring. Detecting them early enough, might be only way to keep the software maintainable. Clear arcthitecture allows to put efforts on new development and decreasing heavily the bug fix times.
If you would like to see how that automation could improve the quality of your codebase, try it for free at https://softagram.com/products
