I suspect this sort of thing gets promulgated because it kind of massages our ego, like yes, they can measure other sorts of productivity, but not ours, oh no, we're too complex and intelligent, there's no way to measure the deep sorts of work that we do! Which, yes, OK, we're not exactly bricklayers, but surely, if you had to, you could do better.
Productivity is not a cut-n-dry stat for any kind of knowledge work.
For an admin, we could say they were more productive if they handled more cases than last month. But this does not account for tricky cases or ambiguous cases, which might have taken 2x time to sort out but on rote numbers it looks they were less productive.
For us, there are no specific goals(except arbitrary imaginary deadlines by clueless management and sprint points) to correctly measure productivity. A junior engineer may spend 4h each day blasting out many lines of codes, a mid/senior may spend more hours in reviews or meetings. A 5 line PR with test coverage is faster to review than a 800+ LOC PR with additional tests and have significant risk of breaking something, so review number is also not a good indicator. I need to coordinate with min. 12 people to get any new credentials or access to specific env owned by another team, which is not countable as productive but unavoidable. How about that garbage meeting where I was just spending time yawning because some new tech lead who believes that a modular monolith system is better if it was rewritten as event based. Productivity measurement is hard.
To take your envy(I sense it on the quoted remark) to another place, think about how you go about measuring productivity of a CEO, PO, PM, Scrum Master, Agilist/Agile-Specialist etc?
For individual, productivity is measured on multiple axis - new long-term features, throw-away and prototypes, code maintenance, fire-fighting, code reviews, high-level design, outside-of-team communications, intra-team communications (for example helping teammates), archeology, specific parts of your project, probably others. In ideal team, everyone is strong on at least one or two axes, so no person is worse than others. In real teams, I've seen people who were weak on every single axis and did not seem to improve with time. I've also seen people who are great at everything, and it's extremely nice when such people exist - but they are very rare.
(Btw, re your specific example: if you weren't productive because you were waiting for others to approve your access then you were not productive. There is nothing complex about this, there were plenty of times when I've said, "I did nothing for project S last week because they still could not give me access")
I have no envy; I'm a software engineer as well. I just don't seem to struggle, as others claim to, in measuring productivity. I find it fairly straightforward to see that some people are more productive than others, at least in my workplace where we can hold most things constant (e.g. meeting count, managerial ability, etc, etc). Yes, there may be external factors, and that's unfortunate. Yes, it's not fair that some people are more productive than others despite putting in half the effort. But I'm not going to stick my head in the sand and pretend it's invisible.
Software developers should look at anyone claiming they can measure software productivity as a snake oil salesman.
We have seen hundreds of attempts over the years and they have all "failed". More accurately they all have large error bars and biases.
Researchers can and should continue looking into how to measure software development productivity. It is likely over the next few decades we will start to understand how to measure it appropriately.
Wherein the work isn't repeating existing work no such measure should be expected to exist.
Any creative work is going to suffer from the same problem.
If you asked a brick layer to develop a better workflow for his fellows to work more effectively in a particular sitiation it would be the same. Are you going to measure words per minute?
Also imagine every other week as the bricklayer had half a wall built a PM comes out and says, "tear this all down and move it 3 feet to the north", and then the week after, "now move it east 1 foot, and why is it taking you so long?"
Calling this phenomenon a toy example is a bit out of touch. I've seen this every single day of my 25 year career. Value is produced by solving problems, not writing code. The solution with fewer lines of code, fewer new abstractions, lower complexity (yes, complexity IS objective and quantifiable) is invariably the best solution. Solutions that involve no code at all are the gold standard.
Adding/subtracting the right code produces value. But adding the wrong code decreases value. The logic of "productivity as code" contains a fatal flaw - it ignores this and assumes all code is right. Reality: everything hinges on the right vs wrong distinction which is inherently subjective. Code is too often a net liability - you must account for the risk!
By contrast, the "productivity as solving problems" approach is entirely objective. You can observe the problem, test hypotheses about the cause, and measure the situation after the intervention. A well stated problem has no ambiguity.
I don't see any support whatsover for the "productivity as code" idea. It's empirically false and lacks logical consistency.
The text gives an example to the core problem, and to argue differently requires thinking around it.
In practice. I’ve seen many attempts at measuring productivity, but once you dig into them, you see they are just abstraction mechanisms above something that is similar to lines of code.
I have yet to see an idea that sidesteps the core issue described in this post. Also, it applies to many types of work, and software is not unique in any way.
All other measures are a proxy for happy customers.
Actually, happy customers is also a proxy (the real measure is profits) but measuring profits directly (in the short term) can lead to decisions that have adverse long term effects. It's too easy to increase profits in the short term by avoiding long-term expenses.
So, if you're in the business of software, the goal is happy customers. (And I use the word Customers carefully here. Not just Users who pay nothing, but Customers who spend money.)
In a business context, it's really the only thing that matters. But, of course, it can be hard to measure (are they Happy?) and relies on multiple disciplines. Production (coding), Marketing, Sales, Support, Documentation, Training- all need to be working well to make it work.
Ultimately if the big picture doesn't lead to Happy Customers (again, I stress, in a businesses context) then no-one is "being productive."
Spoiler: they can't do that either. How do you measure the productivity of a bridge engineer designing a new suspension bridge? Number of struts placed on the blueprint? Even with more manual labor, it's very hard. Who's the most productive out of these three:
1. A carpenter who builds a house from a blueprint in 6 weeks. It collapses after 3 years.
2. A carpenter who builds a house from the same blueprint in 12 weeks. It stays up, but has visible defects that affect its value.
3. A carpenter who builds a house from the same blueprint in 20 weeks. It stays up for over a century and eventually becomes a historical landmark due to its lasting beauty and careful construction.
Which one was more productive?
https://doi.ieeecomputersociety.org/10.1109/32.44364
So you can measure productivity with a reasonable level of accuracy and consistency, but then what? For most organizations it's not actionable so calculating that metric ends up as another form of waste.
Well enough for what? And how do you measure quality?
Measuring developer productivity should, in my opinion, have one dimension for speed, one for quality, and one for user impact. LOC can be fine as a measurement for speed, you just don’t want to look at it in isolation. You would want to also measure, for example, escape rate and usage for the features the developer worked on, and be willing to change or refine these if circumstances require it.
You also need to look for different profiles based on the developer’s level of seniority. A senior dev probably shouldn’t be writing as much code as a contributor, but their user impact should be high, and escape rate low. Analyzing differences between teams is important, as well. A team that has a lot of escapes or little user impact probably has issues that need management attention, and may not have anything at all to do with individual developer productivity or ability.
In brief, the numbers are there to help you make better management decisions, not to relieve you of having to make them.
Developer productivity exists and I am totally comfortable describing Alice as being more productive than Bob. The fact that there isn’t a good, generic way to distill this into particular values does not mean that the phenomenon doesn’t exist.
We’ve all worked with Alice and Bobs. Claiming there’s no such thing as developer productivity is denying what we’ve all experienced. We aren’t special snowflakes whose work is beyond the ken of mortal men. Our work has value and sometimes we produce a lot of value in a particular time period and sometimes we product a little. This is productivity.
A tire factory has a distinct, singular goal: produce tires. It does this continually. Productivity is meaningful, but only in relation to a target that is typically specified by externalities (e.g. amount of demand)
A software company is usually not in the business of producing consumable commodities so this kind of measurement does not make sense. It can make sense to measure productivity during a period for delivering a particular piece of software within a given time bound, but once it's delivered, productivity becomes meaningless. You always need to understand productivity in relation to some purpose and I don't know how these knuckleheads who think this abstract idea is basically like a concrete measurable essence, like mass, or liquid, got leadership positions.
Imagine two people tasked with producing the same software, or software satisfying the same requirements or test suite. What would you call the person who produces it faster? More productive?
Measuring software in financial terms or lines of code might not be the right measurement of software in all situations, but surely we can measure time and cost to produce software or software that satisfies equivalent requirements.
By the way, why suddenly the money=bad sentiment is so popular? I thought USSR example loudly showed us what happens when people think that money is evil.
But something is being produced - it is version 2.0 of the software. This is an artifact that is then shipped to users or deployed to a server. Peter’s solution fixed the issue and did not (seemingly) create further maintenance burden, which would have taken attention away from other tasks, i.e. reduced future productivity.
I agree that metrics for programmer productivity are often useless (e.g. using lines of code is a bad idea for obvious reasons), but it seems silly to claim that the entire concept of productivity does not apply to the production of software.
The rest is just usual software guy hubris and lack of awareness of the discipline.
At least three people do, and they are mentioned in the article: the author, his colleague, and Martin Fowler.
I suspect that software productivity (like the productivity of scientists, artists, composers, etc.) on any non-trivial project may be measurable on a scale of years, but not necessarily months or calendar quarters. A larger issue is that productivity is often due to external factors as much as it is to individual effort, and overnight success is often the result of extended periods of limited progress or even repeated failure.
There is such a thing as productivity in programming. If you could measure it it would likely be some combination of peer review and an analysis of the impact implemented features and fixes had. Some companies actually have programmers rate each other. I don't know how well it works and I think it can lead to perverse scenarios. But you can come up with metrics that are positively correlated with productivity.
The only thing this article gets at is that engineers may not know how to calculate their own productivity; but it doesn't means it's not calculable.
But reality is never that clear cut. How’s the ratio look when:
- Peter goes to the park and the breakthrough doesn’t come?
- Or it comes 3 weeks later?
- Or he deletes 100 lines of code and introduces a new bug?
So more lines of code is better!
Um, we know this doesn't work that way as a good measure.
This is like comparing algorithms that do the same thing to algorithms that do different things. You're not going to get good valid comparisons. Metrics for one thing may not work at all for another.
It's a perfectly cromulent measure so long as we understand the limitations of the measure. For example, trying to measure the productivity of a day or a sprint? That's silly. Measure the output of a team which does not produce an entire product? Won't work because you'd have to figure out how to apportion the productivity.
Because it also means:
"cause (a particular result or situation) to happen or exist."
> You seem to be playing a definition word game. [...] Obviously productivity does approximate business value in a very meaningful way when it is defined in terms of delivered business value.
Then the author responds
> Martin asserts "any true measure of software development productivity must be based on delivered business value". I agree, and I propose there is no such thing. We're better off dropping metaphors altogether and just talking about what programmers do: Solve problems.
So I guess the author wants to replace the term "software productivity" with something like "problem-solving productivity, measured in terms of business value"...? Kind of silly IMO, but to each his own.
It complains, but doesn't offer a solution. It simply criticizes and says "all engineers cannot and thusly should not be measured".
The ironic thing is, the blog post is implicitly measuring by not explicitly measuring. The measurement is the bug ticket itself and whatever value attached to it.
But to this end, I generally agree. There are qualitative and quantitative measurements. Quantitative is the value of the ticket commonly ascribed by the team (scrum? agile? whatever). Qualitative should come up in review.
Qualitative is SO HARD. Top down? Team 360? Mixed? But it must be undertaken and refined by the team at each level of the org. Otherwise you will run into the exact situation described by the blog post and you won't know how to judge left from right, good from bad. Maybe the blog post's example isn't that great, too much information is missing to make a solid judgement, but you need to decide who to reward via promotion, annual raises and who to reprimand and who to not change.
But still, all systems are terrible, but you must pick one less it be picked for you.
It's how we sent rockets to the moon, it seemed to work ok.
Your company and boss is sued because someone says their firing was based on discrimination because they can't prove it was for performance.
The truth is that managers can suss out 90% of the problems without a number. However, we are asked to document the hell out of it if we want to do something about it. And twice in my career I've been wrong: I mistook someone who was quiet for not doing much until I dove into the work. I trust my gut, but I confirm with numbers.
There are so many problems rooted in this statement. Government program, presidential mandate (i.e. unlimited budget), no competition.
SpaceX is clearly better than NASA except maybe they don't push hard enough nor evaluate their engineers so you have a nice stable job if you do nothing.
This is simply never going to end if carried on along the lines of this article, or along the lines of most of the comments here.
There's no way to objectively and reasonably put a value on something.
All we have is a theory of subjective value, which does a bit of handwaving about utility and works out some ways where we can come to a price, regardless of the fact that the values in peoples' heads are subjective.
Thus the distinction between knowledge work and "tangible work" like bricklaying is actually a moot point. Yes, you can measure "productivity" of bricklaying in metres per day, but ultimately you care about value, not amounts of wall.
The arguments about one guy definitely being more productive than another are similar. One person values speed, another values maintenance costs downstream. It is subjective what ought to be more important.
But this doesn't account for the more realistic examples of Prakash, who completely fails to deliver a working solution, or delivers half a solution, and Percy, who gets it done two weeks late. I'm pretty sure you can define a shit_done/time_elapsed productivity metric for those two guys that is worse than that of Peter and Paul.
Maybe I am a cynic but I suspect that some people are upvoting this because the framing makes them feel OK about getting paid six figures to work three hours a day...
My first rule of software programming is: Work Hard to Avoid Writing Code
— Code is habitat for bugs; more code equals more bugs, and interactions between separate parts of code can harbor even more "interesting" varieties.
— Code takes time to run. Any NoOp is faster than even your best hand-tuned assembly module.
Obviously, this is not absolute in any sense — it breaks down as soon as you need the system to actually do something, at which point you must write some code. But it should be the minimal amount to get the job done, and nothing more.
Obligatory car analogy: While doing some amateur sportscar racing, a coach asked me
"What are the things you do that slow the car down?".
I thought for a moment and started saying "when I start into a corner, if I do a bit too much...",
he interrupted saying "NO, no, what are the BIG things you do that slow the car down?".
"Oh, like braking and turning and lifting off the throttle?".
"Right. So what that means is that you should always avoid doing those things. Obviously, you will certainly have to so some of them as soon as you approach the end of the front straight after the start, but make sure you understand your car, the track, and your skills to the point where you do only the absolute minimum."
Both the software and sportscar versions are deceptively simple — they take a LOT more thinking than it seems at first glance. And that thinking is totally worth it.
It is staggeringly hard to measure.
Output is a weak proxy for impact. But it’s the one that makes intuitive sense to people. Doesn’t make it right or useful. I’m sure you all can envision a parable about your subfield of expertise that showcases how a seemingly light touch has a huge positive impact.
It's straight up impossible. Best we can do is observe and attempt to measure proxies.
Take hypothetical situation: VPs A & B debate a business decision. Let's say A wins the argument and their solution leads to a revenue increase of $10M, and let's say we can confidently state that this is the outcome driven primarily by them winning the argument. Is the impact a net growth in business of $10M? In a sense, yes; but in some other sense, perhaps if they went with B's solution, the revenue growth would have been $15M? There's probably no way to know for sure unless you try both approaches, which is often impossible...
If you can solve a problem quickly, at the right time, with buyin from your org, that’s positive impact.
That general case differs wildly when you descend to the particulars.
It becomes clear that simple quantity of code and tickets is not enough- but it's also not nothing. Part of what is missing is quality and assessments of task complexity. Part of what is missing is the other parts of the jobs, like design and code reviews.
I don't think it's hopeless, and can at least be used to look into why some people don't seem to make much at all.
From one point of view, the users of the software are supposed to enjoy so much of a productivity increase that it's not supposed to matter if the coders are as productive as they could be or not.
Give or take a few hundred percent at least.
I realize that most people who've been to business school still aren't going to develop the needed acumen to handle a situation like this.
Too many times the only training retained is a knee-jerk over-reaction to a fraction of a percent :\
Ever see one of these "leaders" have a cow and it was as stupid as it could have possibly been?
I'm confident there are still some natural leaders that can thrive without worrying about every ounce of nose to the grindstone for their staff.
Some things you just can't fix.
Using productivity as a metric leads to the same nonsequitur stuff in many many fields.
It works somewhat for industrial output when you're working with commodities. Or it can work in some more fields as an ancillary measure if you pair it with some other quality, customer satisfaction, outcomes etc measure. But usually you don't want to maximize work while getting good outcomes, you just want the good outcomes.
It's still true that measuring lines of code, time spent coding, commits or anything else is at best a proxy of productivity. It's also true that without any code changes problems more often than not don't get solved or we at least can't call the activity software development.
Easy metric to understand, easy metric to teach, just remember that it applies to teams and not individuals.
See: Out of the Tar Pit.
Using lines of code to track productivity is absurd (do people really believe it or is it just a strawman at this point?). I'm reminded of that midwit meme where a junior has very few lines of code written because they don't know the code base well enough, the midwit writes up an whole framework, and the senior engineer has a net negative lines of code contribution.
Theres also a fundamental difference between creating and maintaining code. Something like 10 guys wrote visicalc. Does that mean they were contributing millions of dollars in profit per hours? What about the maintainance to keep it going? Bug fixes? Patches? On call infra guys? What about opportunity cost of putting engineers on deadend projects?
My point is tracking productivity in software dev - maybe all knowledge work for that matter - is complicated. Maybe that's why there's so much "busywork" (emails, slack, tickets, meetings, etc). Everyone wants to look productive but no one knows what it means
"Put another way, productivity has no applicability as a metric in software.
"How much did we create today?" is not a relevant question to ask. Even if it
could be measured, productivity in software does not approximate business value
in any meaningful way. This is because software development is not an activity
that necessarily produces anything.
Which of these two developers was more "productive" today? The answer is: It doesn't
matter. What matters the that Peter solved the problem, while simultaneously reducing
long term maintenance costs for the team. Frank also solved the problem, but he
increased maintenance costs by producing code, and so (all other things being equal)
his solution is inferior. To call Peter more "productive" is to torture the metaphor
beyond any possible point of utility.
Productivity does not mean "increases business value while decreasing maintenance costs [and having no net negative impact in any way]". It doesn't even mean "solving a problem".
Productivity just means "to make something", or more specifically the rate at which something is made. That's all. You can make 10x more of something, and it can be garbage quality, but you did make it, and you did make more of it, so your productivity increased.
If you produce 10x more grain than you did yesterday, you are more productive. The grain might now be full of heavy metals, pesticides and toxins. But you did in fact produce more grain. If you were trying to measure productivity of usable, healthy, high-quality grain, that is a different measurement than just productivity of grain. You may assume everybody knows what you mean when you say "productive", but you'd be wrong.