There’s Only One Valid Metric For Developer Productivity And Quality

Depending on who you ask, software developers have been programming for over 40 to 50 years. And we still have no truly objective way to measure a developer's productivity and quality level. People can think or say that a developer is fast or productive, but they can't truly quantify it. People can think or say that a developer is great, good or bad, but again, they can't truly quantify it with numbers. We often look at other metrics to give us an idea on quality or productivity such as code metrics, or velocity but those numbers often reflect the efforts of a group of people. After all, a codebase is more often the result of the efforts of more than one person, and a team's velocity is typically the result of the productivity of, well, a group of people. Those numbers can't really be used to gain insight to the quality or productivity of a specific developer.

Not only is the current way of trying to determine the productivity or quality of a developer rather subjective, it's quite often based on things that are only one part of the equation: what they did, and the time in which they did it. I've always felt that that only really tells you half the story. I don't hold much value on knowing how long a developer took to write a certain piece of code or to complete a specific feature. I also don't really care a lot about the code metrics for that piece of code or that specific feature. All i know is that it took some effort. The quality or productivity of that effort can't truly be measured without knowing how much extra effort will be introduced later on because of that effort.

Suppose you have 2 developers, John and Sally. You give them both the same task and you tell them that the estimated workload for that task is 12 hours. John completes the task in 10 hours, and Sally needs 14 hours to complete it. It would be easy to think that John is more productive than Sally based on this. Especially if results like those were to repeat quite often. But what if John's solution requires an extra 6 hours of effort down the line to fix some bugs, or because his code was such a mess to follow that it slowed down future tasks which required another developer to comprehend the code of John's task? That would put John's eventual total to 16 hours of effort, instead of the originally reported 10 hours. And what if Sally's solution didn't introduce any future effort whatsoever down the line? Hell, maybe Sally's solution saved someone else an hour here or there because her solution was so easy to comprehend or perhaps even to reuse. Her total for that task remains at 14 while at the same time maybe even reducing future numbers. In this scenario, Sally is clearly a more productive and better programmer than John though we wouldn't know about it if we'd based ourselves on the initial numbers.

Consider another scenario though. John again spends 10 hours on the same task, and Sally again spends 14 hours. John kept it as simple and as straightforward as he could. Hell, he actually took a shortcut or two to finish faster. Sally is an outspoken member of the Software Craftsmanship movement and stays up to date with all of the latest and greatest patterns and approaches that are making their rounds on the internet. Sally spent a little bit of extra effort to make sure the code is as great as it possibly can be. It's incredibly readable and uses a great new pattern that all the cool kids on the internet rave about. The code metrics clearly show that Sally's solution is of a much higher quality. Unfortunately, it turns out that the other team members are having trouble understanding that code and they often lose time trying to figure it out whenever they need to do something which involves that part of the codebase. While Sally may claim that she took more time because she did it 'better', the extra effort wasn't worth it because no matter how well her intentions with that code, it ended up costing other people time. And what if John's shortcuts didn't introduce any future effort?

There are countless variations on this scenario which makes it virtually impossible to come up with a way to measure productivity or quality based on hours spent or code metrics. The only way to objectively measure this is to define a new metric which holds into account the extra effort that will be introduced later on. I'd call this metric Eventual Efficiency. Quality or productivity by themselves don't really mean anything if one influences the other negatively. Efficiency however can be seen as a combination of the two. Did you write truly fantastic code which ended up not mattering at all? Not efficient. Did you write mediocre code that got the job done and didn't (or hardly) introduced future effort? Sounds pretty efficient to me. Did you write great code that reduced future effort? Very efficient! Did you write bad code quickly that ended up introducing a lot of effort? Not efficient at all.

Unfortunately, we'll probably never get to the point where we can actually measure the Eventual Efficiency of developers. Until we do, it's worth keeping in mind that whatever numbers we might be looking at don't really mean anything without the link to the related future effort.