by alexfarran on 8/29/13, 4:33 PM with 74 comments
by lifeisstillgood on 8/29/13, 5:43 PM
Software is a form of literacy - and we measure literacy completely differently. In fact we measure it like we measure science - you are not a scientist unless other scientists agree you are, and you are not a coder unless other coders say you are.
What Fowler wants to measure is not the top echelons of productivity but the lower bounds - presumably to winnow out the unproductive ones.
But that is not how we conduct ourselves in literacy or science. We educate and train people for a very long time, so that the lower bound of productivity is still going to add value to human society - and the upper bounds are limitless.
What Fowler is asking for is a profession.
by ChuckMcM on 8/29/13, 5:23 PM
Feynman had some interesting thoughts on minimal computation that sort of paralleled Shannon's information complexity. As you know Shannon was interested in absolute limits to the amount of information in a channel and Feynman was more about the amount of computation per joule of energy. But the essence is the same, programs are a process that use energy to either transform information or to comprehend & act on information so 'efficiency' at one level is the amount of transformation/action you get per kW and "productivity" is the first derivative of figuring out how long it takes to go from need to production.
It has been clear for years that you can produce inefficient code quickly, and conversely efficient code more slowly, so from a business value perspective it there is another factor which is the cost of running your process versus the value of running your process. Sort of the 'business efficiency' of the result.
Consider a goods economy comparison of the assembly line versus the craftsman. An assembly line uses more people but produced goods faster, that was orthogonal to the quality of the good produced. So the variables are quantity of goods over time (this gives a cost of goods), the quality of the good (which has some influence on the retail price), and the ability to change what sort of goods you make (which deals with the 'fashion' aspect of goods).
So what is productivity? Is it goods produced per capita? Or goods produced per $-GDP? Or $-GDP per goods produced? Its a bit of all three. Programmer productivity is just as intermixed.
by integraton on 8/29/13, 6:32 PM
While I could be accused of excessive cynicism, I do believe this is common enough that it should be addressed. There's a pervasive delusion that decisions are made by rational, informed actors, when that is rarely the case.
by mathattack on 8/29/13, 6:15 PM
1) Are you doing the right things?
2) Are you doing things right?
They can be imprecisely measured, but every metric has problems and can be gamed. Combining the measurements is extremely difficult.
Let's start with 1 - doing the right things. Someone who chooses to have their team work on 3 high value tasks, and stops their early on 6 low value tasks is by one definition more productive than someone who forces their team to do all 9 things. Or at the very least they are more effective. This is what Fowler is getting at.
On point 2... Let's assume that the appropriateness of what you are doing is immaterial. How fast are you doing it? This can be somewhat approximated. You can say "Speed versus function points" or "Speed versus budget" or "Speed versus other teams achieving the same output" and then bake in rework into the speed. All of these metrics are doable. Lines of code isn't a good base though.
The real question is, "What are you going to do with all of this productivity data?" If the answer is systemic improvement, you're on the right track. If you try to turn it into personal performance (or salary) then people wind up gaming the metrics.
by seiji on 8/29/13, 5:53 PM
Everybody says there's a "shortage of developers," but I know good developers who keep getting shitcanned after a few interviews where nothing seemingly went wrong.
We can't tell who's going to be productive. Since we can't tell, we come up with ten foot high marble walls to scale. Our sterile interview problems make us feel "well, at least the candidate can do our Arbitrary Task, and since we decided what Arbitrary Task would be, they must be good, because they did what we wanted them to do."
Productivity is pretty much the same. There's "just get it done" versus "solving the entire class of problems." Is it being productive if you do 50 copies of "just get it done" when it's really one case of a general problem? I'm sure doing 50 copies of nearly the same thing make you look very busy and generates great results, but solving the general problem could take 1/20th the time, but leave you sitting less fully utilized after (see: automating yourself out of a job).
by RogerL on 8/29/13, 5:38 PM
You know how I figure out if something can be improved? I dig in, understand it, and then look for ways to improve it. If I don't find anything, of course it doesn't mean there is no room, but I'm a pretty bright guy and my results are about as good as any other bright guy/woman.
I was subjected to endless amounts of this because I did military work for 17 years. You'd have some really tiny project (6 months, 2-3 developers), and they'd impose just a huge infrastructure of 'oversight'. By which I mean bean counters, rule followers, and the like - unthinking automatons trying to use rules, automatic tools, and the like. Anything to produce a simple, single number. It was all so senseless. I know that can sound like sour grapes, but every time I was in control of schedule and budget I came in on time and on to under budget. But that is because I took it day by day, looked at and understood where we were and where we needed to go, and adjusted accordingly. Others would push buttons on CASE tools and spend most of their time explaining why they were behind and over budget.
I like Fowler's conclusion - we have to admit our ignorance. It is okay to say "I don't know". Yet some people insist that you have to give an answer, even if it is trivially provable that the answer must be wrong.
by nadam on 8/29/13, 8:35 PM
Now the more productive / better group is which can do the task with smaller complexity.
Complexity measures measure size of code and number of dependencies between blocks in different ways. But even the most simple comlexity measure is quite good: just measure number of tokens in source code. (It is a bitmore sophisticated than LOC). You can then make competitons between groups, and measure their productivity. (I am writing a book now titled 'Structure of Software' which discusses what is good software structure on a very generic/abstract level. It relates to 'Design Patterns' as abstract algebra relates to algebra.)
by artumi-richard on 8/29/13, 8:57 PM
Chapter 8 "Beyond lines of Code: Do we need more complexity metrics?" by Israel Herraiz and Ahmed E Hassan.
Their short answer is that, in the case they looked at, all the suggested metrics correlated with LOC, so you may as well use LOC as it's so easy to measure.
IIRC they believe it's only good to compare LOC between different employees if they are doing pretty much the exact same task however, but since LOC is correlated with code complexity, there is some measure there.
I recommend the book, as really focusing on the science of computer science.
by gz5 on 8/29/13, 6:05 PM
Measure it. Or optimize it. Can't do both without impacting the other.
Software is a work of art and creativity, not the work of a rules-based factory.
by stonemetal on 8/29/13, 8:15 PM
Basically I see this as marketing. We may not be the fastest but who cares about that we have the special insight to build the hits that keep you in business.
by wciu on 8/30/13, 1:31 PM
The problem with productivity measures, is not how they are measured but what they are used for. Most managers want to use productivity measures to evaluate individual or team performance, however, performance is tied to incentives, so you always end up with a lot of push back from the team or someone gaming the system. (IMO, this is because of lazy managers wanting to "manage by numbers", without really understanding how to manage by numbers.)
Rather than using it as a performance management tool, productivity measures, however imprecise, can be used alongside other yardsticks as signals of potential issues. For example, if productivity measure is dropping with a particular module/subsystem, and defect rate is increasing, then one might want to find out if the code needs to be rearchitected or refactored. In these cases, it is okay to be imprecise, because the data are pointers not the end goal. When used correctly, even imprecise data can be very useful.
by dirtyaura on 8/30/13, 6:53 AM
In my opinion, we should approach measurement from a different angle: can we learn something useful about our profession by combining different types of measurements. Can we, for example, easily spot a person who is doing what Fowler is calling important supportive work. Can we detect problem categories that easily lead to buggy code and allocate more time for code quality work for tasks and less for those that are known to be more straight-forward.
by Jormundir on 8/29/13, 6:03 PM
You end up with something like feature 1: +12,544 / -237 lines. Done in 2 weeks.
Then comes feature 2, 2 and a half months later, the stats: +5,428 / -9,845.
Look at that, you had to tear down everything they wrote because they cared about amount of code over code quality. The more they brag, the more you think "oh s$%t, every line they add is a line I'm going to have to completely untangle and refactor."
I think software engineering productivity can be measured, though not well by today's standards. There will probably be a decent algorithm to do it in the future that takes in to account the power of the code, how easy it is to build on top of, how robust it is, etc.
by kailuowang on 8/29/13, 7:17 PM
True, it's hard to objectively measure the overall productivity using a universal standard, but it is relatively easier to measure the productivity fluctuation caused by the external factors. Velocity measurement in Agile practice is mostly for that end.
For the internal factors, the best way, and arguably the only effective way, to manage it is probably to hire good motivated developers. I think most top level software companies have learned that.
by mtdewcmu on 8/30/13, 1:49 AM
> Copy and paste programming leads to high LOC counts and poor design because it breeds duplication.
This problem is not insurmountable. Compression tools work by finding duplication and representing copies as (more concise) references to the original.* The size of the compressed version is an estimate of the real information content of the original, with copies counted at a significantly discounted rate. The compressed size of code could be a more robust measure of the work that went into it.
* Sometimes this is done explicitly, other times it's implicit
by alightergreen on 8/30/13, 3:59 AM
by chipsy on 8/29/13, 11:03 PM
Even if you deliver a system with a lot of features and no known bugs, if they aren't the right features, it's not valuable software.
by AlisdairSH on 8/29/13, 7:52 PM
by scotty79 on 8/30/13, 7:16 AM
by est on 8/30/13, 12:48 AM
http://en.wikipedia.org/wiki/Coastline_paradox
productivity of working on a software is like measuring fractals.
by platz on 8/30/13, 12:31 AM
by dredmorbius on 8/30/13, 9:26 PM
Count lines of code, function points, bugfixes, commits, or any other metric, and you're capturing a part of the process, but you're also creating a strong incentive to game the metric (a well-known characteristic of assessment systems), and you're still missing the key point.
Jacob Nielsen slashed through the Gordon's knot of usability testing a couple of decades back by focusing on a single, simple metric: does a change in design help users accomplish a task faster, and/or more accurately? You now have a metric which can be used independently of the usability domain (it can apply to mall signage or kitchen appliances as readily as desktop software, Web pages, or a tablet app).
Ultimately, software does something. It might sell stuff (measure sales), it might provide entertainment, though in most cases that boils down to selling stuff. It might help design something, or model a problem, or create art. In many cases you can still reduce this to "sell something", in which case, if you're a business, or part of one, you've probably got a metric you can use.
For systems which don't result in a sales transaction directly or indirectly, "usability" probably approaches the metric you want: does a change accomplish a task faster and/or with more accuracy? Does it achieve an objectively better or preferable (double-blind tested) result?
The problem is that there are relatively few changes which can be tested conclusively or independently. And there are what Dennis Meadows calls "easy" and "hard" problems.
Easy problems offer choices in which a change is monotonic across time. Given alternatives A and B, if choice A is better than B at time t, it will be better at time t+n, for any n. You can rapidly determine which of the two alternatives you should choose.
Hard problems provide options which aren't monotonic. A may give us the best long-term results, but if it compares unfavorably initially, this isn't apparent. In a hard problem, A compares unfavorably at some time t, but is better than B at some time t+n, and continues to be better for all larger values of t.
Most new business ventures are hard problems: you're going to be worse off for some period of time before the venture takes off ... assuming it does. Similarly, the choice over whether or not to go to college (and incur both debt and foregone income), to to learn a skill, to exercise and eat healthy.
It's a bit of a marshmallow experiment.
And of course, there's a risk element which should also be factored in: in hard problems, A might be the better choice only some of the time.
All of which does a real number in trying to assess productivity and employee ranking.
Time to re-read Zen and the Art of Motorcycle Maintenance.
by swombat on 8/29/13, 7:38 PM
by a3voices on 8/29/13, 6:35 PM