Saturday, September 08, 2007

Open Source Misconceptions: Evaluating Software

I've noticed lately in the library software world lately that there seems to be a false distinction that's being made about commercial vs open source software. I"m not saying there's not difference, but I'm saying when evaluating software it's often useful to ignore whether it's open source, proprietary and actually decide some of the qualities you're looking for. I watch a lot of food network. They constantly have food contests where judging is done by choosing some qualities (originality, smell, flavor, flammability, what not). There's a table where the food's marked with numbers or letters. Judges taste the food, rate it in every category.

So, ok, we can't do it blind. But doing criteria can help avoid some biases.

So, for example, let's look at some possible categories.

Don't be fooled here. Multiple vendors can service proprietary software, just the same for open source software. True, open source supports will be quick to remind people that you can pay someone to develop any software. But an actual vendor with experience is really required.

Reputation is important here. Very important. What's the use of going with a vendor that's infamous for taking money and bug reports and doing nothing for years. Of course, people in the library world seem terrified about complaining about bad vendors. That's another post though.

Ease to modify:
Difficult to judge if you're not experienced. Some software is really easy to configure, but a pain to extend and modify. If it has an API, direct database access, or the code is visible, it's probably easier to modify than something locked on a vendor service.

Ease to configure:
A bit different than the above. Is there any way to change how the software function? Do you have to stumble through badly documented and bizarre text files? A screen full of unexplained little icons?

Expense of the software itself:
Well, it's a consideration. Really.

Quantity of customers/community:
Are there a lot of people using the software? Some guy and his friend?

Quality of customers/community:
Are they enhancing it, tweaking it, generally loving it? Or do they mostly buy it, install it on some server, and then write a bit in the newsletter and forget about it?

How long has the vendor/community/software been around? How healthy does it look?

You'll probably see some general trends that distinguish open source from proprietary solutions, but you might be surprised when you start examining it. Some open source projects might have a vibrant community with lots of users. Others are dead on arrival in the undergraduate's dorm room. A vendor might have built up an excellent product with a high level of quality. Or it could have transformed into a company full of managers and salesman striving to milk every dollar out of product they actually no longer know how to enhance or fix.

So, hopefully we can start moving beyond simplifications.


jrochkind said...

Nice post.

There are of course, a bunch of requirements that we need to look at that aren't included in your post: those related to the actual function of the software, and whether it does what you need or not. Those can be the hardest to evaluate, but there too, you need to evaluate software on that basis, open source or not.

Codex Monkey said...

True, some of those are easy to assume or take for granted. Then you find out not everyone has the same concept of the problem the software is supposed to address or what are options that cannot be sacrificed.