Innovation Happens Elsewhere: Part Two of a Conversation With Sun Microsystems Laboratories' Ron Goldman

Print-friendly Version

Prior to coming to Sun in 1998, Goldman developed a program to generate and manipulate visual representations of complex data for use by social scientists as part of a collaboration between NYNEX Science & Technology and the Institute for Research on Learning. He has a continuing interest in the design of programming languages and has developed various programming environments. Goldman has a Ph.D. in computer science from Stanford University, where he was a member of the robotics group.

He currently researches alternative software development methodologies and new software architectures, with a particular focus on how to make software more robust.

What is it about the current computing environment that makes open source so important?

First, let me say that open source has been important since the beginning of digital computing. The first computer-users' group, SHARE, was formed in 1955 in order to share source code and solutions to problems with IBM's recently introduced 701 computer. It was a vibrant community activity, as is any successful open-source project. Similar groups quickly sprang up focused on other computer systems. Some of these groups, such as SHARE, continue to be active today.

"Open source has been important since the beginning of digital computing."     Ron Goldman Senior Staff Engineer at Sun Microsystems Laboratories

What makes open source even more important today is the size and complexity of modern computer systems. Instead of a single monolithic application, we are now seeing more and more application ecosystems where many separately developed pieces of software interact with each other to accomplish a desired task. Applications are no longer stand-alone pieces of software. Instead, they rely on a host of local and network services, and many of those services were written by other organizations.

So just as the members of SHARE in 1955 saw no point in each having to reinvent the wheel -- at the time, all the 701 users were writing their own utilities and programs -- today's organizations don't want to use their limited resources writing their own web servers, databases, scripting languages, and so on. They want to put their efforts only into those parts of the software stack that are unique to their business. Anything else is just overhead. Keep in mind that businesses spend way more money developing customized software for internal use as opposed to buying shrink-wrapped software applications. It's much easier, quicker, and cheaper to build on top of existing open-source software. Also, open-source software tends to use open standards, which means that applications built on it will more likely be interoperable -- a major win for today's interdependent software applications.

One very concrete way to see the importance of open source to business is that employees are often paid to work on open-source projects. According to the 2002 Boston Consulting Group Hacker Survey, 30 percent of the people working on open-source projects are paid to do so as part of their regular jobs.

As Dick Gabriel and I describe in our "Mob Software" essay, we foresee that more and more software will be developed as open source -- that is, developed incrementally by a geographically diverse group of people working for many different organizations. Software is too important to be left to just programmers and software companies.

Tell me about your book, Innovation Happens Elsewhere: Open Source as Business Strategy.

For five years, up until 2004, my colleague Dick Gabriel and I worked as advisers on various projects inside Sun, such as Jini technology, the NetBeans IDE, OpenOffice, JXTA, java.net, and, most recently, OpenSolaris. We explored ways to create real communities around each Sun-sponsored project. We worked with managers to discuss how open source would help them meet their business goals and how their projects would need to change to succeed as open source.

As we were advising these different groups at Sun about open-source business strategies, we quickly got tired of having to present the same information to each group. So we looked around to see if there were any good books that we could recommend to people, but there weren't any. There were interesting books on the history of various open-source projects, books on the low-level mechanics, and a few theoretical treatises, but nothing that really went into the business reasons for using open source or that looked at open source at a strategic level.

As a result, we started to write up an internal open-source handbook that we could give to folks at Sun. We kept adding material to it, and our handbook grew and grew. Eventually, we realized that we had enough material for a book that would be of interest to managers and developers outside of Sun whose companies were also trying to make sense of open source.

If anyone wants to take a look at our book, we have made an online version available. We welcome any comments or suggestions on how to improve it.

How does a company that sponsors projects work with volunteers and others so that everyone benefits?

It's harder for a company than an individual to sponsor and run an open-source project. Everyone tends to mistrust a company's motives, so it's very important to be up-front about how the open-source effort is part of the company's business strategy. If it isn't clear how the company plans to benefit, then potential contributors will be suspicious and worry that they will somehow be exploited.

"The biggest hurdle many companies face is learning how to open up their development process."     Ron Goldman Senior Staff Engineer at Sun Microsystems Laboratories

Having an explicit governance process can help alleviate community fears that the company wants to control the project. For example, the Jini project has two legislative houses -- one for individuals and one for organizations -- and a nine-person Technical Oversight Committee -- three people elected from each house and three appointed by Sun -- to provide direction and supervision.

Giving the community more votes than Sun helped establish a level playing field and assured people that everyone would be treated fairly. Most open-source projects can get by with a less formal process, but company-sponsored projects probably should favor some form of voting -- for example, Apache -- rather than the benevolent-dictator model -- for example, Linux.

The biggest hurdle many companies face is learning how to open up their development process. Their engineers need to interact with the rest of the community on the public mailing lists. Design and technical decisions need to be made openly after public discussion. This is not to say that the community tells the company what to do -- though the company is wise to listen to the community's ideas and feedback. Rather, it means that, like any open-source project, individuals and organizations choose what they will work on, and the community as a whole selects what works best.

Through public discussions, the community will come to know some of the company's engineers as individuals and begin to trust them. It's this trust that will help create a project in which people outside the company will feel respected and encouraged to contribute.

Lots of folks misuse the word community to describe customers or a user group. I like the definition that Amy Jo Kim has in her great book, Community Building on the Web: Secret Strategies for Successful Online Communities: A community is "a group of people who have some shared purpose, interest, or goal, who get to know each other better over time."

Is there a key to successful open-source projects?

Passionately shared goals. Successful open-source communities are very passionate about the technology, whether they're working on NetBeans, GNOME, or OpenSolaris. Open source is an evolutionary process: It starts with a working piece of code and a community of people who care about it. People make design suggestions to improve it, implement new features, fix bugs, try out new ideas, and help each other. This attracts more people to the project and can produce a lot of pleasant surprises.

And it doesn't become too chaotic?

No, because a central core group chooses what code actually gets accepted into the official version of the project. At any point in time, there is the most recent stable official release, one or more versions in development, and possibly several experimental versions. The open-source license gives anyone the right to experiment with the code, spurring innovations, some of which are returned to the community.

"Open source is an evolutionary process: It starts with a working piece of code and a community of people who care about it."     Ron Goldman Senior Staff Engineer at Sun Microsystems Laboratories

In his book The Wisdom of Crowds, James Surowiecki discusses three prerequisites for a crowd to be wise and not a mob: diversity, independence of mind, and a way of aggregating results.

Open source provides diversity, especially if you include the users. Open-source participants are quite independent and arrive at conclusions on their own. And open source encourages aggregation as the community evaluates innovation and chooses what to incorporate -- it may be an individual who is in charge of a particular module, or it could be a larger group. Open source meets all three goals, which is essential for the community to develop software that's better than can be produced by an individual.

Open source also requires trust. If people are suspicious or afraid of getting flamed or put down, they won't join the community. If the core team becomes alienated from the community, it ends the openness. In some cases, everyone outside the core team decides to separate and create their own code. At times, some of the core people come along. You could liken it to a little revolution due to trust breaking down.

Is there a positive side to this?

There definitely is. For instance, numerous BSD projects, such as FreeBSD, OpenBSD, and NetBSD, have both shared and different goals. It made sense for them to fork off into different projects. Other projects, such as gcc, have forked, and then later the two forks merged back into a single, improved project. Even for projects like GNU Emacs and XEmacs that forked over 15 years ago, some believe that the competition between the two groups has made for more rapid development of Emacs.

Why do you claim that innovation happens elsewhere?

"Innovation happens elsewhere" is a quote from Bill Joy that he made after observing that there are always more smart people outside a company than inside it. So the question is how to somehow harness the innovation that these other people are creating. Open source turns out to be an excellent way to do so.

In our book, we talk a bit about how open source is a gift economy, and one of the results is that it really lowers the potential barriers organizations have to collaboration. It allows competitors like Sun and IBM to cooperate on many different open-source projects, such as Tomcat and Derby. Rather than having to negotiate a legal contract, a company or government organization just needs to assign some of its employees to add whatever features it needs to an open-source application. It benefits directly, and so does the larger community.

The Internet has been a fantastic enabler of open source, allowing people with a common interest to communicate with each other and collaborate on code development even though they are scattered all over the globe.

What are the biggest misconceptions about open source?

The biggest misconception managers have is that they can cut back on the number of people in their group working on a project by making it open source, because the community will pick up the slack. Actually, it's just the opposite: An open-source project takes more resources than an internal proprietary one. Employees working on open source might spend an additional 25 percent of their time on community interaction. It costs more to do open-source development, but the benefits of better design and increased quality justify those costs. The community contributes to the quality of the final product by reporting and fixing bugs, and by innovating.

Another misconception is that open source is business as usual and doesn't require fundamental change. A company must open up its development process to include the community to reap any benefits from open source. When designing in the open, you get immediate feedback such as, "Well, that's not what I want to do with this," or "Why doesn't it do this or that?" And so you can build things right the first time.

That's a major win right there. Most software projects fail not for technical reasons but because the product they build is not one their customers want. You must be flexible enough to take the information your community gives you and use it. If you fail to recognize innovation or simply slough people off, you can miss opportunities.

"The biggest misconception managers have is that they can cut back on the number of people in their group working on a project by making it open source, because the community will pick up the slack."     Ron Goldman Senior Staff Engineer at Sun Microsystems Laboratories

Are there any areas where you think using using open-source development is not advisable?

Open source is not always appropriate. It requires solid business reasons. If what a company is thinking of open sourcing is not something that others outside of the company care about, you won't get a community. If aspects of the software or plans must be kept secret, then open source won't work -- though it may be possible to use the practices of open source but to limit who can participate.

A number of companies have created gated communities with some of their partners, in which they can collaborate. You can also use open-source development within a company, making it easier for employees in different parts of the organization to work together.

Open-source development requires being willing to listen to what the community says. Having a controlling attitude or not respecting outside contributors can cause an open-source project to fail.

When thinking about using an open-source application, it is important to consider a number of factors, just as when selecting any commercial software. First, even if there is no licensing fee, one needs to look at the total cost of ownership, which includes support and training. The maturity of the project, along with the size and involvement of the project's community, indicate how healthy the project is and how much support can be expected from other community members.

Open source requires a different mind-set, that of participant and co-developer, not being a passive customer, so if a company or its IT organization is not comfortable with a more active role, then it should stick to more traditional practices.

And finally, many organizations want a guaranteed level of support that an open-source community, by itself, may not be able to offer. Fortunately, an increasing number of companies specialize in supporting open-source applications.

What's going on at java.net?

Let me first explain a bit about how java.net came to be. Back in 2002, a number of web sites were hosting open-source projects and a number of other sites were focused on different aspects of Java technology. A number of us at Sun thought that if only we could bring these various efforts together somehow, that we could achieve a critical mass for Java software and increase the innovation and collaboration among Java programmers.

When we looked at the existing web sites, we saw that they were missing some important pieces. For java.net, we took the standard open-source project infrastructure tools and added wikis and blogs to enhance the ways that people could communicate. We also wanted to create a newspaper telling folks what's new and important in the Java technology world. So we made sure that we had an editor to highlight the latest Java technology news on the front page, spotlight interesting projects, and help create a community.

We also wanted to work with existing web sites, not compete against them. So we encouraged pointers on java.net to anything interesting related to the Java programming language. We also made sure articles and blogs on java.net had RSS feeds so that other web sites could track them and point their readers to java.net.

All in all, it seems to have worked: Over 200,000 people have signed up as members, and they are working on over 2500 different projects. More and more, Sun is moving the development of its Java technology-related projects to java.net. For instance, check out Project GlassFish, which is a Java application server. Also, the next version of Java SE, version 6, is in development there. Although it is not open source -- it is licensed under the Java Research License (JRL) -- Java programmers can access the source code, fix bugs, and work with the development team.

What one message about open source would you like to leave managers and developers with?

Successful open-source projects are all about community. For larger projects, there are usually multiple communities, each with their own goals. For example, OpenOffice has separate communities based on people's native language (such as English, French, Chinese), on topics (such as marketing, helping users), and on development of the code.

"Successful open-source projects are all about community... usually multiple communities..."     Ron Goldman Senior Staff Engineer at Sun Microsystems Laboratories

It is important to recognize that each community has different goals and requires different support if it is to grow and flourish. It is also important to realize that people can contribute in many ways: graphic design, writing documentation and tutorials, user interface design, marketing, answering questions on the mailing lists, project management, testing, and, of course, writing code.

Open source is an evolving methodology. The way that current open-source projects are run is just the beginning. We need to explore new ways of doing things. For instance, users are often a neglected resource. People who can't code -- or who know how but don't have the time to -- are often disenfranchised from a project's decision making. We need to better involve the people who actually use an application in the design of that application. After all, who knows better than they what is needed?

Finally, welcome the unexpected! The work that other people do will amaze and delight you, and that's one of the major strengths of open source.

Conscientious Software: Part One of a Conversation With Sun Microsystems Laboratories' Ron Goldman
Innovation Happens Elsewhere
Mob Software: The Erotic Life of Code
The Boston Consulting Group Hacker Survey
Community Building on the Web: Secret Strategies for Successful Online Communities