Surgical teams on GitHub: was Fred Brooks right all along?

In 1975, Fred Brooks argued that software teams should work like an operating room: one surgeon does the cutting, and everyone else makes the surgeon faster. Four decades later, we tested the idea on GitHub. The data sided with Brooks.

A 50-year-old idea about who holds the scalpel

The Mythical Man-Month is the book that gave us Brooks’s law: adding manpower to a late software project makes it later. Brooks drew his lessons from building IBM’s OS/360 in the 1960s, and in the same book he describes a way to organize a team, borrowed from Harlan Mills:

Mills proposes that each segment of a large job be tackled by a team, but that the team be organized like a surgical team rather than a hog-butchering team. That is, instead of each member cutting away on the problem, one does the cutting and the others give him every support that will enhance his effectiveness and productivity.

Fred Brooks, The Mythical Man-Month

The arithmetic behind it is unforgiving. When everyone works on everything, every pair of people is a line of communication that has to stay in sync: n(n−1)/2 of them. Ten people means 45 lines. In a surgical team, the same ten people coordinate mostly through one person, which leaves 9.

Open source looks like the opposite world: volunteers spread across time zones, many of whom never meet, coordinating through pull requests and issue trackers. Does an operating-room metaphor from the mainframe era still hold there? GitHub let us check it at scale.

Why issue closure?

Imagine you’re a company about to build a product on top of an open-source library. Stars and downloads tell you the project is popular. They don’t tell you what happens on the day you hit a bug.

Issue closure does. When users report bugs and ask for features, does the team close those issues, and how quickly? It’s a window into two things at once: how well the development process works, and how much support users can expect from the people behind the project. That is what we set out to explain.

How we tested it

• The data. We used GHTorrent, a large public archive of GitHub activity, to select a big sample of mature, active projects. No weekend experiments, no abandoned repositories.
• The method. We used survival analysis, a technique best known from medical research. Every issue is treated like a patient in a clinical study, and the event we wait for is its closure. The method also handles issues that are still open when the data ends, instead of dropping them or pretending they were closed. From it we got short-term and long-term issue closure rates for every project. Yes, a paper about surgical teams that borrows its main tool from medicine.
• The suspects. For each project we measured team size, how centralized the work is, how the project’s internal workflow looks, and the social network between its developers, plus a set of control variables.
• The model. These features went into a model that predicts a project’s issue closure rate, which let us test each of our hypotheses.

What we found

1. Bigger teams, lower closure rates. Projects with many members closed their issues at lower rates than smaller teams. More hands did not translate into faster fixes. Call it Brooks’s law in an open-source costume.

2. Centralization helps. The more concentrated the work, the higher the issue closure rate. Projects where a core group does most of the work, a surgeon in Brooks’s terms, were better at getting issues closed.

3. Busy is not the same as responsive. Desirable social network characteristics among developers came with more commits, but had no significant influence on issue closure. A well-connected community makes a project busier. It doesn’t automatically make it better at closing the loop with its users.

Put together, the results support Brooks’s surgical team as a model for open-source development on GitHub.

What I take away from it

• Choosing an open-source dependency? Look beyond stars and contributor counts. Check how issues actually get closed and whether the project has a clear core. A long list of contributors is not a support plan.
• Maintaining a project? Welcome contributors, but keep clear owners who triage, review and close. Let contributions flow through the core, not around it.
• Leading a team at work? When a deadline slips, adding people is the tempting fix. Brooks warned against it fifty years ago, and what we saw on GitHub is consistent with that. Give the people doing the cutting more support, not more peers to coordinate with.

One caveat worth keeping in mind: this is an observational study of open-source projects. It shows which project traits go together with higher issue closure rates; it is not a controlled experiment. And “centralized” describes how the work is distributed across people, not a management style.

Read the paper

The paper is open access, so it’s free to read.

Since it came out, the paper has been cited in research on open-source project health, developer metrics and risk assessment. It builds on our earlier SocInfo 2014 paper, GitHub Projects. Quality Analysis of Open-Source Software. You’ll find that one and the rest of my papers under Publications.

@article{jarczyk2018surgical,
  title     = {Surgical teams on GitHub: Modeling performance of GitHub project development processes},
  author    = {Jarczyk, Oskar and Jaroszewicz, Szymon and Wierzbicki, Adam and Pawlak, Kamil and Jankowski-Lorek, Micha{\l}},
  journal   = {Information and Software Technology},
  volume    = {100},
  pages     = {32--46},
  year      = {2018},
  publisher = {Elsevier},
  doi       = {10.1016/j.infsof.2018.03.010}
}