Living Without the Bunnies

LilyPond is doing great! LilyPond is headed for disaster!

On one hand, we're doing great. The new stable version 2.14.0 is finally out, we've finally started the Grand Organization Project to resolve various administrative problems, and there are no huge problems which will stop us from releasing 2.16.0 in a few months.

On the other hand, we still have far too many tasks concentrated in far too few people. This is not healthy, both for the individuals doing the work, but also for the project as a whole.

"Sustainable development" in software projects

Every person will leave the LilyPond project sooner or later. Our lives change, our hobbies shift, and unfortunately our health and lifespans are not infinite. What would happen if, for example, I decided that my PhD thesis and future career were more important than LilyPond and that I should drop out the project?

(this is not an idle question; I honestly should quit LilyPond. Some open-source volunteering would still be good for my career, but it would be much more benefitial for me to spend that time on Marsyas, ChucK, or even STK instead. Those project are much more aligned with my research and potential tenure-track jobs, and I would learn much more by working on them instead of LilyPond.)

Such problems are charmingly referred to a project's Bus factor: how many people would need to be hit by a bus before the project falls into disarray? LilyPond has a quite low bus factor -- losing as few as three people would cause massive problems.

I gave a presentation at RMLL 2010 about this last year, and I was reminded of this by a recent discussion about mentoring efforts by a gnome developer looking at the approximate 25% success rate (over a variety of projects, including LilyPond).

Decentralization

A low "bus factor" means that work (or knowledge or power) is highly concentrated in a few people. It's much better for the project if work/knowledge/power is distributed more evenly -- losing one person will cause fewer problems.

There are other benefits to decentralization, of course. A lower workload on "central" people will lessen burn-out, which may encourage them to stick around for longer. That can, in turn, lessen the workload on other people: there are many cases where an "old-timer" can make a few suggestions in a 5-minute email which save a new contributor literally hours of work and frustration. Everybody wins!

Dangers of burn-out

I'm (in)famous on the LilyPond mailing lists for arguing against new proposals. "We don't have the resources / nice idea, but wait until... / you can do that with a user .ly file / etc".

In some cases this is due to my subscribing in a certain philosophy of design: Il semble que la perfection soit atteinte non quand il n'y a plus rien à ajouter, mais quand il n'y a plus rien à retrancher. (perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away; from quotes from Antoine de Sainte Exupéry). A similar idea was mentioned by Steve Jobs in a recent interview:

People think focus means saying yes to the thing you've got to focus on. But that's not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I'm actually as proud of the things we haven’t done as the things I have done. Innovation is saying 'no' to 1,000 things.

But the other part is that I simply don't want more work. Every single time that somebody comes up to me with an excited new idea, it breaks my heart because my first thought is "oh boy, now I need to express polite enthusiasm". It doesn't matter if it's over email, in a chat, in a meeting, or in a private conversation. Doesn't matter if it's from a longtime developer or an unknown user. Doesn't matter if I think it's a brilliant idea, an obvious but useful improvement, or a pie-in-the-sky ludicrous concept. My first thought is always the same: "bleh, more work".

At the moment, I'm committed to doing 10 hours of LilyPond work per week. I've "saved up" a few hours from the previous two months (including my vacation) because I knew that I'd be busier after 2.14 came out -- at the time of this writing, I have 12 hours of LilyPond time left. Each Monday morning, I add 10 hours to that figure. By my estimation, only considering the published Grand Organization Project and Grand LilyPond Input Syntax Standardization backlogs, I won't be able to do make any LilyPond bugfixes or new features of my own until the end of 2011. My entire time will be spent organizing and helping other people to do work.

You can help

Get involved! Anything that makes LilyPond less centralized will help.

Even if it has nothing to do with me personally, there will probably be a "trickle up" effect from reducing the burden of other people. For example, if a translator started doing some of the work of Francisco (the Translation Meister), he would have more time to investigate problems himself instead of asking me to help. Similarly, if a developer offers to seriously mentor one or two new programmers, that might mean one or two fewer programmers who come to me for help because everybody else is ignoring them. If you're a new contributor, then spend some extra time talking to other new contributors -- provide moral support and somebody to commiserate with, even if you can't discuss anything technical!

As for me personally... I spend about 2 hours a week on completely mindless administration. Stuff that could almost be done by a computer program. I could easily write 5 or 6 sentences on paper, and somebody could follow those blindly. For example, twice a week I send a "patches countdown" email.

1. go to the list of patches for review
2. pick two or three issues
3. find the codereview URL inside the issue
4. click on that link
5. if there are any reviews, check that the last review is not negative
6. include the subject line and link to the codereview page in an email to lilypond-devel

All I need to offload this task is a reliable person who doesn't mind a bit of boredom offering to do it. 15 minutes, twice a week. If there's anything unusual, I'm happy to discuss it (and possibly modify the instructions to handle such cases). But I'm not asking for any knowledge or creativity.

There are other tasks which would require knowledge and/or creativity. For example, many people ask for a changelog -- what's new in each release? If somebody is interested in lilypond development, they could spend 1-2 hours a week reading the developer email list and the git changelog, then writing a 3-4 sentence summary for each release. That would be a nice "gentle" introduction to how things work, and it would benefit many users.

Similarly, there are some tasks which are regular weekly affairs, and others which are one-time. Here's a great non-really-technical one-time task: our list of places to send release announcements to is quite old. Half of the links don't work. So I haven't bothered to make any release announcements (other than our own webpage) at all. If somebody wanted to spread the word, they could spend 2-5 hours looking for relevant places to announce lilypond, maybe asking for other suggestions from lilypond-user, and make a nice list. Then either send our release announcement to those places himself, or else ask me to do it. I wouldn't mind doing it as long as all the links work and I don't need to hunt around to find out how to send the announcement (i.e. give me a specific link to the announcement form, not a generic link to the main website!)

Square pegs into round holes

In the past, we haven't done a good job of organizing people in LilyPond. Some people are suited to regular work; other people are suited to one-time jobs. Some people like technical challenges and creative freedom; other people prefer a rigid set of guidelines so they know that they're not making any mistakes.

We need people for all sorts of tasks, and over the next few weeks, I'm really going to try to make sure that everybody has a job that suits them. We've lost a lot of potential effort by asking people to do tasks which don't really appeal to them, and then not having those jobs "open" when another person offers to help.