Sunday, August 19, 2007

Open Source

Steve K. sent me an email yesterday, containing this response to something positive I'd said about open source software:
From your mentioning of open source, I’m kind of gathering that you’re a fan of it. I’m not sure I am. It seems like often when I’m looking for a solution, the open source world wants me to download the latest source code, plus the source to 3 other open source projects that this one depends on, and then compile the whole mess to get it going. I usually don’t want to mess with all that. I know there are exceptions to this.
I've seen this general kind of perception from quite a few people before, and I vaguely remember when I first started using some open source software I had a similar reaction. It's just so darned different than the commercial software world, where all the software comes neatly packaged (usually in a box!), with manuals that follow a general convention, and so on.

But different isn't always bad, and I've come to truly appreciate the differences in open source software – to me, the pros far outweigh the cons. And with experience and practice, it turns out that many of my initial perceptions – like the ones that Steve K. still has – weren't true at all.

For example, the issue about needing to build open source software from the source code. By definition, this is possible to do with all open source software. However, I use many packages on a daily basis where I have never even downloaded the source code, much less built it. For example: Tomcat, MySQL, Hibernate, Apache Web Server, MantaRay, JFreeChart, OpenLDAP, Ubuntu, Ethereal, OpenPERL, and many, many more. These include both complete, stand-alone applications and software components. The last time I can remember actually building an open source package from source code was's Java implementation, which I extended slightly.

Ok, so maybe I didn't have to actually build these things. So what? Why would I want to use these open source things instead of commercial products?

That's an easy one: because they're better. Just plain better. I don't mean that all open source packages are better than all commercial packages – but the actively worked on open source packages are consistently better.

There's a good reason for this, and it's not magic: visibility. Every line of the source code for an open source package is available for scrutiny by the best software engineers in the world (all of them!). There are no secrets. Open source software cannot hide its dirty laundry behind a corporate firewall or PR machine; it's all there, for everyone to examine. The culture of every active open source project I've ever run into is all about quality – whereas the culture of every commercial project I've ever been involved with (and that's many of them!) is all about time-to-market.

This is a profound difference, and it shows up in another way as well.

Every programmer who's been around more than a few weeks can tell you horror stories about this or that bug they've run into with an operating system or an application. I have personally run into dozens of bugs in Windows, Microsoft SQL, Visual Basic, Norton Utilities, and a long list of other commercial applications. These are very frustrating situations to deal with, sometimes raising challenging or even impossible obstacles to getting your commercial software finished on time and within budget. The vendors tend to not be cooperative in the first place, and even when you can convince them that they've got a bug, you won't have a fixed release in your hands for months or even years.

This problem simply doesn't exist with an actively developed open source project. First of all, there's no mystery about whether there is a bug – you can even go prove it yourself if necessary, by examining the source code. However, I have only had to go that far just one time in many bug submissions to open source projects. By far the most common response from the open source communities has been along these lines:
“Gee, thanks for finding that bug! You're our hero for the day!” Then, a few hours to a few days later, I'll get an email that says “On that bug you submitted – we have a fix, and it's available in this build. Would you test it for us?”

How refreshing is that? My favorite experience like this occurred with MySQL. A couple of years ago, I discovered a bug with CBlobs in their Java SQL driver implementation. I reported it with a code example. Two hours later, I got an email with the binary of a new build attached – and my bug was fixed! The next day that driver revision was posted on the MySQL site. That example is my favorite mainly for its contrast with Microsoft SQL. About five years ago, after a big battle with their support people, I finally got Microsoft to acknowledge a bug in one of the SQL functions. That was the last I ever heard about it, after repeated inquiries over the next couple of years. I've stopped caring about this stupid bug, but the point is that the bug is still there, five years later.

Open source is better. That's why I like it and advocate it. The bumps in the road on open source are real (though getting better all the time) – but they are a small price to pay for the overwhelmingly better quality of the open source products…


  1. There's a great book about computer culture...

    "In the beginning was the command line..." by Neal Stephenson.

    He's one of my favorite Sci-Fi authors so this non-fiction book is a bit of a departure for him. The book is good, but what makes it great is a chapter-long analogy of the different OS systems, where he makes the following comparisons:

    Apple = A company that produces sleek motorcycles that almost always work but that you can't tinker with.

    Microsoft = A company that took their bicycle and slapped on an ill-fitting motor, then spent the rest of their budget on advertising.

    BeOS (the book is a few years old) = A company that sells batmobiles

    Linux = Not so much a company as a commune of people in tepees who build tanks and give them away for free, and if they break they'll come to your house and fix it, then take what they've learned and go to everyone else's tank and fix them for free too.

    I've always loved the analogy, and you comment that Open Source feels so different reminded me of it.

  2. I’m a fan of open source software but there are definitely a couple of pitfalls to be aware of.

    There are a couple of ways of using open source software.

    • There are open source products/tools that you may use personally, like ClamWin or something like that.
    • There are open source products that you may use in operating an enterprise business like Tomcat.
    • Then there is open source products that you use like Hibernate or log4j in another commercial product. Each different type of use carries with it different risks.

    The risks of using something like ClamWin or PDFCreator aren’t that significant and for me, are far out-weighed by the benefit of not having to pay for commercial software. If they quit being supported or have some flaw I can’t live with, I can always move on to one that works better for me and I’m not out much.

    For the business, relying on an open source product may carry more risks if it something goes wrong. Particularly as they don’t typically come with any kind of support. So sticking with long established products or where maybe a 3rd party provides support can help reduce that risk.

    For software development, you can save significant time by leveraging open source software as part of your own product. There are the normal risks that there will be a significant bug that can’t immediately be solved or worked around and you may have to swap out the component either with different open source software or by writing your own. Though, as Tom mentions, the open source community can be pretty responsive and/or you can go in and figure it out yourself (which can introduce legal issues). In this category there are significant legal risks that we mitigate by working with attorneys familiar with open source licenses. There are so many licenses and variations each with their own restrictions that you could really open the business to significant legal liability if you just download, use and distribute open source software.

    Most of the newer open source license types allow you to do pretty much anything you want with the open source software, but there are a lot of variations as well as some older ones that have significant restrictions. Like only allowing dynamically linking to the libraries. Others have clauses invoked if your company sues their company for any reason, others have requirements that you provide or offer the source for their product with your distribution. Many require that you include the text of their license with your product.

    All of these little I’s and T’s better be crossed or you open yourself up to potential liability and if you millions in revenue stopped all of a sudden by legal action, well that would be bad. So anyway, to way to mitigate this risk is to ensure a solid process for incorporating open source software into your commercial product. License reviews before it ends up in your source tree. A process for reviewing whether that particular open source software is needed, or if the needs can be fulfilled by some other, more friendly software.