For over a decade CTERA has contributed to some of the world’s largest open source projects, including Linux Kernel and Samba.
In this collaborative spirit, I’d like to share some valuable lessons we’ve learned in working with the open source community on some of the toughest OS challenges. Different open source strategies exist. As we have come to understand, it’s entirely possible to turn your contribution and efforts into a win-win for your company and the community.
Open Source 101
Here’s a quick explainer for those who are new to open source. There are three ways to use open source code:
- Use it “as is”
- Take it “as is” and then modify it to fit specific product needs
- Use, modify and then push changes back upstream (we’ll get to this later)
The General Public License (GPL) distributed with some open source software (e.g., Linux) stipulates that you can take the open source code and use it as you like for free. However, if you modify the code and monetize those changes (under certain conditions), you are obliged to make those changes available to the community.
In other words, open source communities do not like freeloaders. The GPL creates an incentive for participants to give back, rather than using their improvements solely for financial gain. This approach may seem risky from a competitive standpoint, which is why many companies reluctantly comply by publishing their changes on “buried” website pages with the hope that no one is going to notice it.
Why CTERA Contributes to Open Source Projects
So why bother investing your time and resources contributing to open source projects? The effort can often be substantial, and the code you develop could be used by competitors.
But the truth is that most software products created by leading vendors in the IT space are based on open source (Linux is most common). CTERA, for example, uses Linux and SAMBA (Windows network share) open source in its products.
In retrospect, we have found that giving back to the open source community is indeed in our company’s best interest. This wasn’t always obvious at the time, and we took a few lumps along the way, but two key reasons prove the point for us.
Reason #1: Tap into the “Wisdom of the Crowd”
CTERA’s Next3 file system, first shipped in 2010, exemplifies the benefits of leveraging the brains and resources of the open source community. We sought an open source file system that would meet our business and functional requirements, including support for local snapshots. Since we didn’t find any open source at that time that met our requirements, we decided to improve the Linux EXT3 file system by developing and adding snapshot support. One year later we launched the Next3 product and published the code in compliance with GPL rules.
But the fact that we made the code public didn’t mean that it was integrated as a mainline Linux kernel product. This would have required a substantial amount of additional work, which did not fit into CTERA’s work plan at the time.
We did, however, continue to engage with the community and presented our work on Next3 at a conference. A few months later a group of graduate students came across our work and expressed an interest to help. With expert mentoring from CTERA, these students were able to make the necessary modifications to meet the upstream standards. Not only that, the students helped to find bugs and performed testing in scenarios we didn’t even consider at the time (e.g., on stronger machines). In fact, five years later our software was running on many of those stronger machines.
Thus, by publishing our code on open source, we not only shared our product with the community, we were also able to access a deep reservoir of technological knowledge and expertise and benefit from highly professional peer review and feedback.
Reason #2: Reduce Maintenance Burden
The world of open source software can be divided into two types of development. “Upstream” refers to the existing open source project you’re taking from, while “downstream” refers to diversions from the open source (i.e., modifications to meet specific needs).
While open source code can be taken “as is,” most commercial projects comprise both upstream and downstream code. But here’s the catch – open source is not a static project. Bug fixes and new features are being added all the time. This means that each time a new update is released from upstream you need to merge your changes (i.e., downstream) with the upstream.
This typically starts as a trickle but quickly becomes a downpour that requires a major maintenance investment. The more time you fall behind, the larger the gap becomes and the effort for merging can grow exponentially over time. It’s quite common that companies get stuck in an “old” version of the project because it’s too difficult to keep up with upstream. The smartphone industry is a classic example. When companies reach the point that they can no longer justify the maintenance effort, they often have no choice but to declare end of life.
But there is a solution. If you take your downstream changes and contribute them to the open source project, the “maintainers” will adopt them and maintain the “improvements” for you. Of course, this requires that the maintainers view the improvements as valuable to the community. It also requires the code to meet the open source project’s quality standards and may involve a few rounds of revisions. Once your code is merged into the upstream version, your maintenance burden is eliminated.
Lessons Learned: Adopting an Upstream First Strategy
CTERA’s “upstream first” strategy can be attributed in part to our experience in the SAMBA open source project. SAMBA is the standard Windows interoperability suite of programs for Linux and Unix. It provides secure file services (Windows network share) for all clients using the SMB/CIFS protocol.
At the outset, we had a growing pile of modifications to the SAMBA project and it became harder and harder to merge updates from upstream. In 2015, we analyzed the maintenance burden of keeping up with SAMBA and decided to allocate resources to address this gap. The result was that we were able to merge many of our changes with SAMBA upstream and reduce our maintenance burden. Indeed, we may have acted initially for “selfish” reasons, but the community also gained in the process – creating a true win/win situation.
From that point on CTERA decided to adopt an “upstream-first strategy” with respect to new Linux kernel projects. This means engaging with the community from day 1 – working with upstream, getting feedback at the design stage and every step along the way. We have found that it’s in our long-term best interest to make the extra effort to interact with the community from the get-go.
Allaying Competitive Concerns
Companies developing open source are often concerned that their code might be used by competitors. One way that companies avoid this concern is by not engaging the community or publishing the code until a product is shipped (within the framework of GPL). While this may maximize competitive edge, there is still a lot to be gained from engaging the community early in the process.
CTERA believes that the benefits of “upstream first” outweigh the risks. Our open source strategy with respect to competition is to separate core IP from supporting technology. If you have enough core IP and create an edge over competitors in proprietary components, you don’t have to worry about sharing code defined as “supporting technology.”
Understanding the Open Source Project Challenges
Is open source for everyone? That depends to a large extent on whether you’re up to the challenges.
When you’re working with someone else’s project, you need to abide by that project’s standards. Generally, open source standards tend to be higher than average because they have the luxury of being based solely on technical considerations. Development, design and testing processes in your own company can be easily bent by business and other constraints, which results in lower software development standards.
In addition to top-notch technical skills, open source projects also require strong people skills for interaction with the community and consensus-building. Rather than reporting to a single manager, open source developers need to present their work to a large community of professionals and handle myriad comments and feedback in a diplomatic manner.
A Final Word
After a lot of hard work, perseverance, trial and error, CTERA has found a way to make its open source project investments a win-win for us and the community. It wasn’t always easy, and it required making some difficult decisions.
If your company has got the technical chops for open source, this can be a great way to leverage the knowledge of the community and reduce your maintenance burden. By choosing the right open source strategy for your company from the outset, you can marry your own business interests with those of the community.
Amir Goldstein is a well-established Linux kernel filesystem developer. As Principal Software Engineer at CTERA, Amir leads CTERA’s open source project efforts. He is an active contributor to the Overlay filesystem and to the Linux filesystem notification subsystem. Amir recently presented his latest work at the Linux Storage Filesystem & Memory Management Summit.