Welcome to the sixth annual Best of Open Source Software Awards, otherwise known as the Bossies. If you've enjoyed our previous Bossies, you're in for a treat: This year, with the help of our extended InfoWorld family of contributing editors, we've pulled together more than 100 Bossie-worthy products in seven categories, from application development tools to -- for the first time -- games and other fun stuff.
- Bossie Awards 2012: The best open source applications
- Bossie Awards 2012: The best open source application development tools
- Bossie Awards 2012: The best open source data center and cloud software
- Bossie Awards 2012: The best open source databases
- Bossie Awards 2012: The best open source desktop applications
- Bossie Awards 2012: The best open source networking and security software
- Bossie Awards 2012: Now for something completely different
Just as the Bossies have grown fatter, so has the totality of the open source beast. Sourceforge claims more than 4 million downloads, 3,402 code commits, and 722 bugs were tracked on the day I'm writing this paragraph alone. The most popular by far is the VLC video player, which continues to beat many commercial players. And all it takes is a glance at GitHub to see that JavaScript code for the browser (jQuery) and the server (Node) continue to draw downloads and bug fixes like flies to flypaper.
We've taken it upon ourselves to plow through all that frenetic activity and dig up the juiciest, smartest, and most useful open source software available. If you'd just like to page through from beginning to end, start here. Stick around in this article and you'll get a tour of the important trends in open source this year.
Hadoop: An elephant born tap-dancing
Nothing in open source is more exciting than the constellation of software around Hadoop. Technically, Hadoop is just a small part of a big stack of software that keeps a number of machines crunching together on a single problem, but as you may have noticed, your boss's boss has learned to drop Hadoop as a buzzword. So we often overlook related programs such as Pig or Hive though they can be more useful than Hadoop itself.
Hadoop is the poster child for the big data. It began as a small experiment based on Google's MapReduce technology and grew into a stack of code for those who need to do big things with data spread out across a rack of nodes. The tool has been so successful, we've heard rumors that the Google engineers who pioneered the MapReduce paradigm are jealous of the innovation going on in Hadoop. Google got the ball rolling, but the open source nature of Hadoop allowed the rest of the Internet to surpass the biggest dog in big data.
This is especially important to the perennial also-ran, Yahoo, which was one of the early believers in Hadoop and supported much of the original work. Open source believers point to Hadoop as a huge victory for the open source strategy: Yahoo shared with everyone else and everyone else shared back, yielding a whole new software ecosystem -- one that's driving the hottest industry trend today.
Embedded in Hadoop is the ongoing tension between open and proprietary. A swarm of startups have sprung from the open source Hadoop code, adding just enough proprietary contributions to attract and retain customers. This debate is being played out as one company alone, Hortonworks, tries to keep its entire platform open. Will Hortonworks succeed? One competitor told me archly, "It's nice to see that Hortonworks finally got a platform out."
Yet pragmatists see this tension as a creative force that fosters exciting new businesses. The core of Hadoop is still pretty close to a standard, which makes life easier for everyone. The extras keep everything running and pay for the upkeep of the core. Programmers need to eat, and the secret sauce is the best way to justify salaries, while the core remains open and improving.
From rows and relations to keys and columns
Hadoop and its satellites are not the only projects working to solve large and complex data problems -- or simpler data problems for that matter. After decades of throwing every sort of data into relational database management systems, we're now seeing a slew of open source alternatives to the traditional data store.
Call them NoSQL, not only SQL, or un-SQL, these alternatives range from refreshingly simple to startlingly sophisticated. Many of them offer high performance or horizontal scalability by trading away some of the power of the relational database. The differences often lie in the tradeoffs they've made to accommodate certain kinds of use cases.
For instance, key-value stores such as Couchbase and Cassandra offer high performance and high scalability where preserving relationships among the data isn't a priority. Both will integrate with Hadoop, and serve as good analytical stores for semi-structured data. Cassandra offers a column-oriented solution, while Couchbase is evolving into a document database -- yet another way to organize the keys and values.
A document database is nothing but a key-value store in which the values are JSON. Programmers like them because they fall squarely into their object-oriented paradigm. Plus, the elements inside the JSON document can be indexed to speed up searches. They're a natural way to store clumps of related elements -- like the makings of a blog post or a patient record -- that don't fit neatly into a relational model. All of this explains the popularity of MongoDB.
There's a different project for each kind of problem. Some of the answers, such as graph database Neo4j, come into play when the relational powers of the traditional RDBMS aren't relational enough. In the past, depending on the requirements, the architect would tweak the configuration of MySQL or Oracle in a different way. Now, the architect can choose a completely different project.
Puffing up the private cloud
With a private cloud, you can borrow technologies and architectures pioneered by public cloud providers and apply them to your own data center. A bunch of open source projects have emerged to offer software to accomplish just that.
The OpenStack open source project in particular has gained surprising momentum in the private cloud space. This stack of Apache 2-licensed bits provides a framework for managing virtualized compute, storage, and networking resources -- with identity, monitoring, and self-service thrown in for the ride.
Billing itself as a "cloud operating system," OpenStack was initially developed by Rackspace and NASA. Now governed by a separate Foundation, OpenStack claims more than 192 participating companies, including Canonical, Cisco, Dell, HP, IBM, Red Hat, VMware, and a gaggle of cloud startups. Many of these companies plan to offer "packaged" versions because -- as with the Linux kernel -- the raw OpenStack bits are not something you'd normally download and put into production.
But OpenStack is not the only open source private cloud game in town. The best-known open source competitor to OpenStack is Eucalyptus, developed at the University of California and intended to mimic Amazon Web Services, with full API compatibility. CloudStack, an open source project launched by Citrix in April, is well-positioned for use by cloud service providers, with a great Web UI for administering cloud resources.
To the Internets and beyond!?
Open source is outgrowing the world of computers. The success of open source software means that developers are giving it a whirl for everything from automobiles to clothing.
Take routers -- AutoAP, for instance, is a fascinating bit of code that can turn your wireless router into a node in a self-organizing network. It turns the once passive box for setting up Wi-Fi connections into an active participant that's constantly looking to bond with any neighbor.
Most people are using this to set up self-organizing wireless repeating networks in their home, but from time to time the networks jump the fence and start mixing. Dreamers suggest that one day there will be enough AutoAP-flashed routers to meld together into a seamless internetwork that doesn't need the big ISPs. People will have an "Internet" that simply exists and doesn't need a central corporation to carry the bits.
This same out-of-the-desktop-box thinking is also powering XBMC, the open source "set-top box" software that wants to lead the PC army to take over the living room. XBMC already has a firm beachhead with a number of alluring plug-ins. All it needs is good, affordable hardware -- which may be coming as people build cheap Android PCs with HDMI connectors.
There are even more radical applications of open source. The Arduino hardware project continues to attract hackers who itch for more than just a screen and keyboard. Arduino hardware is being sewn into garments, attached to plants, joined to coffeemakers, integrated into model airplanes, and on and on. It's going pretty much anywhere a laptop can't go.
The hardware that began in a small Italian company is attracting the attention of big companies including RadioShack, which wants to return to its roots as a candy store for tinkerers. Some recent projects include mind-controlled robots, a glitchbox for playing music, and an Arduino-powered box that gives a plant the ability to water itself.
Then there are people exploring open source movie cameras like the Apertus Axiom, a design for a high-end video camera that the creators are hoping to crowdfund. Not to mention people building open source ham radios like the DStar, people creating open source instruments like the Zoybar, and even people rebranding the old recipe files as open source food.
The proliferation of all of this open source is why Sam Muirhead, a writer and artist, has vowed to make this next year his "year of open source," in which he'll replace all of the items around his house with real open source alternatives. If he can't get the schematics and the rights to reproduce something, he won't buy it.
Challenges await, but the good news is that there are still people who are willing to jump in and start everything anew. I'm personally looking around in my couch cushions for enough change to buy a Rally Fighter, an SUV that's in the beginning of a 2,000-car manufacturing run. You can download the 3D CAD drawings to build your own. Or you can just work with the designers in Arizona who will help you through a six-day-long "build experience."
The open source movement has taken us on an astounding journey. What began as quixotic quest that all software should be free has grown into the engine of innovation and change that has already extended beyond the bounds of the computer industry.
This article, "Bossies 2012: The Best of Open Source Software Awards," was originally published at InfoWorld.com. Follow the latest developments in open source at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.