Why would I pay for two programmers if only one works?
I hate the idea of somebody continually watching over my shoulder...
I am not a programmer. I have been working in software companies for a long time, but I have barely ever programmed. Back in 2001, I started a software company focusing on eGovernment applications, to be custom built using JAVA. All fairly new at that time. At some point in time we switched over to full eXtreme Programming. And I was impressed: quality increased, productivity soared, the team motivated.
I tried selling the ideas afterwards. It did not work. Pair programming in particular caused major resentment. Managers hate it. It looks like a terrifying waste. Programmers hate the idea when you explain it to them. Just believe they will hate having to explain every single idea to somebody leaving dirty fingerprints on their screen, or whatever.
I once was called in to salvage a failing project. I encountered a particular problem. Half the nearshore team had quit (because of the measly salary and the desperate situation of the project), further endangering the project. They had hired three new programmers, but incorporating them was going to be difficult and it could endanger the motivation of the rest of the team. I proposed to start using pair programming when introducing the new team members. So I flew in, explained to the programmers what pair programming was and how they would start working the next day.
They were shocked. The only good piece of information about my presentation, it seemed, was that I would fly back out in two days and they had a chance of going back to normal.
The next day one of the guys had a huge set of earphones on... Four pairs were formed, three expert - novice, with me and the project manager creating a somewhat fake fourth team.
It took about 15 minutes and then the first person became aware that she actually liked the questions the novice was asking, giving her the opportunity to explain the reason behind a particular statement. Six months later, the team were still pairing - the earphone guy not included. All new programmers stayed, their modules (and the entire project) were delivered.
I had my own 'pair programming' experience. I 'pretend paired' with the project manager and for the sake of team we went through the motions and debugged a particular bit of code. He explained to me what each bit of code was doing. I am not even a novice programmer, I can just barely read JAVA code, so I was just nodding. Then at a particular statement I hesitated; I could not quite see how the explanation of the code matched with the code. The PL noticed my hesitation, went back, and sure enough, the code did not do what he supposed it did.
Pair programming works, it motivates, provides for fast and efficient learning and in this case made finding a bug easier. Is it more or less productive, is the key question?
The answer is not easy to find. The studies use small groups of students for short periods of time in limited lab settings. But short time bursts of limited code production are not really the point, a long term sustained effort on large code bases is way more relevant in real life.
So let us look at the project we ran. About 350 kLOC, initial code coverage by J-Unit test of about 70 %. The project was not yet finished and it needed to be adapted so it could accomodate a second customer with some quite different requirements. We are talking some 11 developers, 3 in-team architects, a proxy customer/in-team tester and a systems guy. We switched to full test driven development and used 'promiscuous pairing' (hate that word, I prefer rotational pairing) with teams working together for half a day before switching partners.
First of all: it worked. The team size was about double the recommendation for Scrum (5-9, we are talking 16 here), but no issues there. I believe it worked because of the pair programming and intensive rotation and related communication. It seems as if the number of pairs was more important than the number of people?
I do not know the objective productivity in say hours per function point. I only know that we managed to stay within budget, were on time and achieved a code base adapted to the customers' needs with few bugs and issues. We ran the application for a while and serviced it. The few bugs that turned up could all be resolved in a short period of time. I have been estimating software workload for a long time and all I can say is that by my gut feeling, productivity was certainly no lower than in any other project and quality was way higher.
The project had a to me 'magical moment' clarifying some important sides of productivity. I was substitute scrum master for two summer months on end. The team had by then shrunk to 10 developers and I was told that 5 more developers would be taken off the project after summer, so that I had better be prepared. I hired three novice developers and they started in the beginning of the summer.
We dreaded that period, with on average half of our experienced developers available. We had so far achieved all deadlines in the project - only just though - and based on our experience believed that all we could hope for was some reduction of the backlog and to have the novice developers up to speed after summer
But we did way better. In fact, over that summer, we achieved a productivity that could be explained by the experienced team members achieving 100 % and the novice developers 90 % of that. We had a couple of days delay and the customer never noticed because he asked to postpone delivery by a week. There is no way that we could have achieved this result without the rotational pairing. In a traditional environment, the absence of developers during vacation periods slows the project down, often for more than just the 'non-productive' period, as dependencies between modules cannot be solved optimally when the dedicated developer is absent. And adding three novice programmers to a team that is basically halved during a vacation period can just about stop all production.
This proves to me that pair programming does not mean 'productivity loss'. Even if pair programming were to lead to productivity loss in 'ideal circumstances', the better results of the team during a more difficult period can more than compensate. The 'experienced - novice' pairs worked like a dream (probably because the team members had had a decisive vote in who was hired). The focus of the team stayed high, even with team composition changing weekly.
Looking at it from a somewhat higher perspective, in this project we did Test Driven Development. Team members stated that about 55 % to 60 % of the time was spent on designing and writing test code. Since developers spend more than half of their time thinking about and developing test code, this looks like doubling the development effort. We did intensive pair programming, which to some also looks like halving productivity. The office environment looked chaotic and was really rather too noisy. And all I can say about it is that we achieved high quality, a satisfied customer and a satisfied team, stayed on time and on budget, and all in all were competitive as we won this deal in a public tender.
This example from a real production environment points out the problems of lab settings. 'Experienced - novice' pairs do not work too well in experiments. Well, that depends. If you are proud of your team and have had a say in who is going to become a novice team member, say. Or if it means that the novice will help you achieve a menacing deadline. Adding people to an established team, in order to speed up delivery or for some other reason: a real life problem that costs money and that pair programming can make much more efficient. Training, team building, no knowledge bottlenecks in the code, more satisfactory work conditions, higher quality: they all translate to money.
And finally, we are moving into a new world, with fewer managers, more self-governing teams, with learning and team work as the rewards, instead of promotions. Working in rotating pairs (not just for programming) should have a major role in that.
Update September 2019
A couple of years later and more insight in lean as well as agile...
I now realize the importance of an additional element. The way we worked is that one team member started a user story and stayed with it from start to finish. An automatic consequence of this is that we started only half as many stories as we had programmers in the team. By doing so, we effectively limited WIP (Work In Progress) to a far lower level than most teams would. No wonder we finished them faster. Also, the 'owner' of the user story tended to be the more generalist developer, calling in the aid of a more specialized programmer when needed. The Pair Programming setup was also a very Lean setup, with limited WIP and a focus on finishing a user story before starting the next one. The specialist resources were used in the most efficient way, namely when their experience was most needed. So the pair programming led to a lean way of working. This effect greatly contributed to productivity.
Lean thinking also helps to clarify the advantage we saw during the holiday period. Common code ownership made sure than no single resource was a bottleneck for continuing work.
These two thoughts lead to a third one. Productivity of the team members is less important than the productivity of the team as a whole. Don Reinertsen, the leading author on lean product development, tends to ignore productivity and focuses instead on the speed of value creation. And there, pair programming can do significantly better than the more traditional approach.