With the coronavirus COVID-19 taking the world by storm, and everyone tightening down in their bunkers, it seems like a good time to think about how we work together. Because of the virus, technology conferences have shuttered and even frequent office dwellers and meeting makers are learning how to work remotely. Open source software development is usually done remotely, so maybe by borrowing some of the methods of open source development, we can all find better ways to work together and stay connected?
I spoke with Danese Cooper about “InnerSource” or using open source methods to develop internal or proprietary software. Cooper is a technology executive, long time open source advocate and now president of the InnerSource Commons Foundation.
(Disclosure: I served with Danese Cooper on the board of the Open Source Initiative and I have worked with her in other capacities. She describes us as frenemies.)
What is InnerSource?
InnerSource is so named to distinguish itself from open source. Unlike open source, InnerSource is developed inside your company. According to Cooper, “InnerSource is the use of open source methods inside the firewall in a proprietary company because it’s a better way to write software.” Some companies who learned how to collaborate using InnerSource also find that they can collaborate more publicly in open source.
To many people, open source is a licensing regime and nothing more. Indeed many companies treat it that way. To those of us who have participated in open source projects, it is more of a means of collaboration, methods, and tools. The license just makes that practical on a large scale with strangers. Open source collaboration involves public repositories, committers, pull requests, various standards, and norms for how a pull request must be constructed. For instance, when I started what is now the POI project at Apache, we required unit test coverage, comments, and documentation. Those “norms” were required for the contemporary equivalent of a pull request.
In many large companies, you can’t just go changing code outside of your immediate area. Yet oftentimes you’re held responsible for not getting things done because your upstream dependency didn’t have a feature that you needed. In open source, you’d just fix the upstream dependency, submit a pull request, and continue. The maintainer might accept the pull request “as is.” Alternatively, the maintainer for that codebase might tell you that the pull request couldn’t be accepted because you used global variables and for loops instead of functional code. If rejected, you fix it, and the conversation is captured.
In essence, a company practicing InnerSource operates the same way, except the repositories are inside the company.
Why do InnerSource?
There are several goals companies have for adopting InnerSource. One is to work better remotely with a dispersed team. If you’re trying to recruit the best Node.js developers, they are not all in Silicon Valley or, in Cooper’s case, Ireland.
With COVID-19, now we are all remote workers. According to Cooper, “We’re seeing a steady interest in InnerSource. No one has said, yet, [that] they’re interested because of Coronavirus but they have said they’re interested in reducing their facilities cost.”
Conway’s law is an old adage that “Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.” As a corollary to this, zones of control inside of companies tend to produce many people solving the same problem. Cooper reports that another main driver behind InnerSource is that companies want to promote more code reuse rather than have every corporate silo write its own version of everything.
What happens when you have a lot of people writing the same thing over again? You have a resource problem where you can’t hire enough employees to avoid a bottleneck. Cooper’s former employer, PayPal, is an example of a company that was trying to hire ahead of the problem despite having around 8,000 engineers.
Another reason companies are adopting InnerSource is to drive innovation by breaking up the routine. According to Cooper, “InnerSource has been used successfully for that in some very large companies like Bosch and others that have found that really just taking people out of their day to day way of doing things and putting them in a different environment where they’re free to riff on each other’s work will often create that condition for innovation.”
In a standard SDLC (software development life cycle) or corporate version of “agile,” the documentation that is produced is often of questionable value and rarely read. In an open source or InnerSource project, documentation is super helpful — just as the artifacts produced by open source projects are better actionable documentation than, as Cooper put it, “the bullshit documentation that your team is going to write at the end of a project.”
The engineer’s conversation about how to make a given pull request mergeable contains a lot of information that never makes it into the official doc. It happens passively as part of the process because you can capture the conversation that people have.
Cooper has found through her work that InnerSource gives you an automatic bump in quality, partly because code review happens passively as part of the process. In a traditional or corporate agile process, often times people “rubber stamp” code that they are supposed to be reviewing. This happens especially if the person is more senior. It is a short hop from there to introducing problems into the codebase because the code has not been reviewed by anyone but the original developer.
According to Cooper, “Another reason is intangible, but it is a real one — employee happiness. People get really frustrated by the sitting around and waiting in feature request culture. It creates a lot of stop energy, and no amount of agile will get you out of that if that’s the thing that is blocking everything else. Being able to actually do something towards continued forward progress is surprisingly good for people.”
From personal experience, this was why I got involved in open source. I couldn’t stand to sit around and wait for reticent vendors to fix problems.
How does InnerSource relate to agile development?
Cooper says that agile development methodologies and InnerSource are “actually very harmonious except one piece of advice from scrum which suggests that colocation is desirable.” Colocation is not desirable in InnerSource, because it promotes less written interchange of knowledge and more spoken interchange of knowledge. It means those two people have shared something that there is no way to give to a third person. Unless we start recording people’s conversations in some meaningful way and making them searchable, colocation is always going to be an enemy of a good collection of documentation as part of the process.
Cooper feels that colocation is both unnecessarily expensive and silly. “It is true that a small team can get a lot done sitting around a table. It isn’t actually the way big companies organize themselves and if they try to it tends to be very expensive and unnecessary.”
Everything else in agile works just fine with InnerSource. In fact the organizational part of agile — i.e., work for two weeks, do a demo, etc. — all of that can be very helpful, especially for collaborators.
How does InnerSource work with product and program managers?
One of the challenges of InnerSource is that companies and product managers are used to top-down planning and “executive escalation.” At PayPal, despite an initial success, there was a lot of resistance because of the disruption of traditional roles and a general addiction to executive escalation.
In “Getting started with InnerSource,” Andy Oram notes that PayPal experienced a 25 percent quality increase by adopting InnerSource. Developers spent 65 percent of their time on “executive escalation” in one team because it was a regular bottleneck for the organization. However, product management undid the change to InnerSource in the next release cycle to negative effect. As a result, engineers who didn’t want to go back to the old way left.
PayPal brought in outside help to enact the organizational changes that needed to happen. The pecking order system of feature priority was addressed. Product managers switched to writing high-level stories. Having a system where the product managers could see how their stories had been translated into engineering stories gave them a stronger feeling of ownership in the result.
Cooper suggests companies go further with this change:
I’d like to see product owners become closer to community managers in the open source stuff. If the customers are their community, then collect their feedback so that the engineers don’t have to deal with the firehose of customer feedback. But I don’t want to see them burdened with writing stories.
Where can you learn more about InnerSource?
There are a number of resources available to organizations looking to understand InnerSource including two relatively short books:
PayPal also produced a series of videos on InnerSource that you can find at O’Reilly behind a paywall with a free trial. There is also the InnerSource Commons, which hosts an active community complete with a blog, wiki, and git repository.
Finally, if you have too many problems you can hire a consultant such as Cooper’s NearForm or others.
But should you really do this?
If lower facilities costs, higher quality, better technical documentation, fewer bottlenecks, global recruiting, and greater developer happiness aren’t compelling enough to get over the organizational hurdles, consider this final thought from Danese Cooper:
New recruits, especially straight out of school, are expecting more transparency and they don’t see why you wouldn’t work that way. It is often quite soul effacing to join a traditional company and be shown the way to work and discover how little agency they have as individuals. They have a hard time fitting in. As the workforce gets younger, working in this way will become more and more imperative I suspect.