With All Hallows Eve upon us once more, as the souls of the dead come to haunt us, it’s time to recount terrifying tales and scary stories… about software. You might think that research software is safe from such gruesome goings-on but you would be wrong, for there are many undead projects out to devour us.
Here’s how to recognise some of these spooky software, along with other pestilent projects, and dispatch them back whence they came.
Vampires: suck the lifeblood out of other projects around them, draining them of resources. Vampire software projects are typically successful (vampires are very charismatic), they’re just not great to have too close by. They are created when a group has developed a successful piece of research software, but the resources needed to feed it have grown out of hand, leading to other projects being drained of their lifeblood (developers) to keep the vampire alive. You need to decide whether to drive a stake through a vampire’s heart to save your village (team), or just restrict its activities with judicious use of garlic and feature roadmaps.
Zombies: continue making slow, shambling progress, but have no life to them and never will. Instead they feed on people’s brains. Typically the result of a good initial grant proposal which got infected by scope creep, they can also be the result of a previously dead project being resurrected by another injection of resources when they should have been left in peace. On their own, they are merely a nuisance but you can easily be overwhelmed if they aren’t dealt with properly. Can be killed by decapitating using a more honest end of funding review or controlled by shackling in a repository and removing nearby brains.
Ghosts: are pale apparitions of seemingly useful software, never quite manifesting on this plane of reality. This is usually due to a possessive PI who wants to keep the software, and the papers it generates, restricted to their group and chosen collaborators. The PI pays lip service to open source to satisfy funding agencies, but qualified by things like “collaboration agreements” and “memoranda of understanding”. Should this software be provided with resources to rematerialise its corporeal form? No one knows, because the software isn’t available.
Revenants: are a particularly nasty type of project where the software ideas of a particular person come back to haunt the - normally junior - developer working on them, driven by jealousy or greed stemming from a longing for the things of life which it once had (time to write code). Often summoned by a professor asking their student to take a look at a piece of code they themselves wrote, but so protective of the code that it’s impossible to make progress on them. The most powerful (also known as Draugr) possess superhuman strength, can increase their size at will, and carry the unmistakable stench of decay. The only way to dispel a revenant is for our hero to stand up to them and force them through the corpse-door (normally into a committee), ensuring it’s shut to seal them out.
Frankenstein’s Monster: demonstrates what happens when a well-meaning idea gets out of control. Normally, reuse is good for research software. Not this time. To get ahead, a brilliant but misunderstood PI has cobbled together various bits of old code that they’ve dug up, some of which has been sitting around for a bit, and some of which doesn’t quite fit. At first it seems to do what its master wants it to do, but it quickly gets out of control and wreaks havoc on research in the area because it’s impossible to understand why it’s doing the things it does. Best to just lay this particular creation to rest, and start again.
Banshees: lament for what is to come, often a harbinger of death. Here the project leader is so haunted by the failure to keep alive previous loved ones (software), that they believe this project to be doomed too, condemning the software project. The only way to resist the banshee is to seal your ears to the screams, and make progress regardless.
Lich: are brought to life by a powerful incantation or spell. Their bodies are particularly cadaverous or skeletal, because it is essentially the sheer power of the necromancer (PI) controlling them that makes them alive. Pretty much every blockchain project falls into this category just now. The good news is that when the power (money) that keeps the incantation going stops, so does the lich.
Ghouls: feast on the corpses of other projects. This is a particularly nefarious piece of software because it has managed to outlive so many others. The skill of the ghoul is that it is greedy, and manages to steal resources or even kill children. Unlike the zombie, they are hard to avoid because of their prominence and renown, and they are protective of their territory. You see this most often with software that has been created in an area which previously has had a lot of funding, but now most of the other projects have died. It is almost impossible to stop a ghoul, and you’re often better to avoid the area completely.
Plague rats: are small-scale software that seem to reproduce uncontrollably, spawning new versions, and feasting on resources. Because of lack of documentation, quality controls, and community engagement, an individual package never achieves a significant user base. Instead, PIs elect to write their own from scratch instead of contributing their features to existing code. The more mediocre the software is, the more PIs think, “I bet my postdoc could write a better code than that, so we’ll just start over”, and the cycle continues. Eventually, every lab has its own code, all of which have significant feature overlap, but none of which are sustainable. Requires a pied piper to come in and remove the infestation, or at least come up with a coherent policy for picking platforms.
Parasites: are low quality, poorly supported software which slowly suck away at the resources of their host. Often found at major experiments or facilities, the parasite previously supported the host by producing the definitive simulations for it but becomes cumbersome to maintain over time. Now it’s firmly attached to the facility, and you can’t kill the parasite without severely harming the host. Getting rid of parasites is extremely difficult, and sometimes you just have to wait until the host is undergoing surgery—a major upgrade or long maintenance shutdown—to treat the parasitic infestation.
For many, the biggest software sustainability challenge is not how to sustain software that everyone agrees is important. It is, instead, to understand what software has reached “end-of-life”. But even then it may not be enough, as software can resist termination, or resurrect itself, joining the ranks of the undead. Now you know how to recognise the scariest software projects out there, it’s up to you to join the fight to banish them!
The original idea behind this blog post was first presented by Neil Chue Hong as a white paper at the 2019 Collegeville Workshop on Sustainable Scientific Software (CW3S19) before being revised and extended for this blog post, which is cross-posted on the SSI, BSSw, and URSSI sites. Daniel S Katz provided feedback on the original version. Benjamin Cowan haunted the document with the plague rat and parasite examples, along with a revised ghost description, for this version. Neil is always looking for new specimens - email to contribute.
Neil Chue Hong is Director of the Software Sustainability Institute (SSI) and Senior Research Fellow at the EPCC, University of Edinburgh. Benjamin Cowan is a Senior Research Scientist at Tech-X Corporation.