There is a pervasive belief in the software world that you should never re-invent the wheel and that an existing library is always the best solution. While there is some merit to the sentiment that re-inventing the wheel is often pointless or dangerous, I have recently come to the conclusion that this is not always the case.
Let me tell you a story about how this conclusion came about.
This all came about because I needed the PHP IMAP extension installed for a particular site at $DAYJOB. I didn’t want to have the baggage of the extension for every site since exactly zero other sites are currently using it. Of course, that’s not normally a problem. You just build the extension as a shared object and then load it for that one site. Easy as anything, right? Well, not so much, but that wasn’t the first problem.
First, the PHP IMAP extension depends on the libc-client.a library file from the UW IMAP software package. This package hasn’t had an update in five or so years. Of course, that isn’t necessarily a problem in itself. The problem is that it has an antiquated build system that doesn’t even have a sensible “make install” target as far as I can tell.
Okay, so you go ahead and figure out how to build the software for your system. After finding out that it doesn’t know how to find the OpenSSL header files properly and stumbling around for half an hour to figure that one out, you get it built. Then you are left staring at your screen wondering how you install it so that the PHP IMAP extension can find it.
Aha! There’s a recipe in the comments on the PHP IMAP documentation. Oh, wait. There’s half a dozen recipes, all different. Well, that sucks. You flail around trying various bits until you hit on one that actually works. You install the entire source tree somewhere, create a couple of folders, copy a few files, and you’re off to the races.
And then the PHP build fails. Obscure looking errors about relocations and other business like that. Things that make no sense to a mere mortal like you. So off to the interwebs you go to look up what that scary looking message means. And you find out. It seems you need to build a “relocatable” version of the libc-client.a file. But how?
You flail around for a while and discover that there is no possible way to do so. In fact, the creators of this specific library have explicitly not supported even that much because “shared libraries are pointless”. Never mind the fact that you aren’t trying to use it as a shared library but simply link it into a loadable module, and that requires the same compiler flags. Eventually, you find the magic incantation thing to do to make that build properly, rebuild the library file, and you’re off.
This time, the PHP build completes and you’re happy. But it took far too long to accomplish this.
Okay, so what led to all these problems? First and foremost, the c-client.a file created when building the UW IMAP distribution is not intended as a standalone library. It is not intended to be installed separately. It is simply a package that collects the common bits used by the binaries provided the the UW IMAP package. Examined in this context, the “shared libraries are pointless” sentiment makes sense, though the explanation given by the authors betrays a deep misunderstanding of how shared libraries are supposed to be built and maintained.
Second, other UW software (pine for instance) builds using the same UW IMAP library. At least it did last time I build pine, which was, admittedly, many years ago. This leads people to the notion that the library really is intended for other software to use, never mind the fact that the pine build process is confusing as a result.
However, it came about, the PHP IMAP extension authors chose to depend on a library that isn’t intended as a standalone library. As a result, the PHP IMAP extension is miserable to build and install. Even if your distribution provides the library package built sensibly, the distribution maintainers have to deal with manually maintaining a build system. In this case, that’s not much of a burden given that updates to the source package are few and very far between.
Anyway, back to my point. In this case, relying on the UW IMAP “c-client” library was a terrible choice. Another library that implements IMAP in a standalone manner that is intended to be used as a library for third party software might not have been. I might go so far as to say that absolutely nobody should rely on the UW IMAP “c-client” library because it is a steaming pile of turds, that assessment being due to the terrible build system it uses.
Rather than leave you with that totally unhelpful statement, I’m going to finish with a checklist you can use to determine if it is sensible to use a particular library.
- Does the library come standard with the vast majority of the systems you are targetting? If so, you’re probably safe to use it, but consider the other items on this list.
- Does the library come in an independent software distribution that can be easily installed independent of other unneeded software? That is, can a potential user download a source distribution, follow a simple build process (such as “configure; make; make install”) and end up with a usable installation of the library? Note that this need not be restricted to strictly source distributions. A binary distribution that has a simple installation may be acceptable.
- Does the library maintainer support usage of the library in third party applications? If not, you’ll find that the ABI will probably change randomly between library versions, or other incompatibilities will appear randomly and these will cause unending trouble for either you, your users, or both.
- Does the library come with sensible API documentation? If not, odds are pretty good it’s not actually intended for general usage even if it happens to be packaged separately and nicely. Lack of a documented API means the library maintainers are also working in the dark about what it supposed to be supported and how and it makes knowing what is a bug and what is proper function impossible to determine reliably. If you have to read the source code of the library to work out how to use it, you shouldn’t be using it.
- Does the library do something non-trivial? Are you using the library to do something trivial that you could easily implement in a short function? If so, you don’t need the library. Just write the function. Your users will appreciate that you have not needlessly added yet another dependency.
- Are you bringing in a large library just to use a tiny portion of it? This is similar to the previous one. This will be more of a problem with “utility” libraries than with libraries that implement a particular thing, but it’s not exclusive to them. For instance, bringing in a massive library just to get some nice wrappers around the libc string functions is dumb. You might as well just write your own nice wrappers. Also, just because you happen to be parsing a tag soup file, it doesn’t mean you need a validating XML parser, especially if you don’t care about validating against DTDs and all that stuff.
- Does the abstraction provided by the library make sense for your purposes? That is, are you going to be working around the library as much as you’re working with it? If so, that is a good indication that the library is inappropriate for your use case. This is not a hard fast rule, though. This will require a judgement call.
- Finally, consider any dependencies the library has and run those through the same checklist. Consider any dependencies you wouldn’t already be using against the size of this library. Remember. Anyone building your software will have to follow the entire dependency chain all the way down. Pulling in one “nice” library that happens to depend on 75 other libraries of varying quality and utility is a sure way to have anyone building your software throw up their hands in despair and pitch it in the rubbish bin. Remember that the more dependencies you have, direct or indirect, the more opportunities there are for random problems.
I’ll leave you with one final thought. If you are writing a library, you should be expecially ruthless about paring down extenal dependencies and making sure you don’t fun afoul of items in the above checklist. Your users will thank you. Or, at least, the distribution maintainers will thank you. And, if your library isn’t intended for general usage, make that clear in the documentation somewhere.
Remember. Using an external library isn’t necessarily bad. Just think about it carefully before you do.