Skip to main content

LilyPond 2.14 is out

I'm happy to announce the release of GNU LilyPond 2.14. LilyPond is a music engraving program, devoted to producing the highest-quality sheet music possible. It brings the aesthetics of traditionally engraved music to computer printouts. Many improvements have been made in the past two and a half years since the previous main stable version. (yes, that's a long time between stable versions!)

LilyPond is doing great! LilyPond is headed for disaster!

On one hand, we're doing great. The new stable version 2.14.0 is finally out, we've finally started the Grand Organization Project to resolve various administrative problems, and there are no huge problems which will stop us from releasing 2.16.0 in a few months.

On the other hand, we still have far too many tasks concentrated in far too few people. This is not healthy, both for the individuals doing the work, but also for the project as a whole.

"Sustainable development" in software projects

Every person will leave the LilyPond project sooner or later. Our lives change, our hobbies shift, and unfortunately our health and lifespans are not infinite. What would happen if, for example, I decided that my PhD thesis and future career were more important than LilyPond and that I should drop out the project?

(this is not an idle question; I honestly should quit LilyPond. Some open-source volunteering would still be good for my career, but it would be much more benefitial for me to spend that time on Marsyas, ChucK, or even STK instead. Those project are much more aligned with my research and potential tenure-track jobs, and I would learn much more by working on them instead of LilyPond.)

Such problems are charmingly referred to a project's Bus factor: how many people would need to be hit by a bus before the project falls into disarray? LilyPond has a quite low bus factor -- losing as few as three people would cause massive problems.

I gave a presentation at RMLL 2010 about this last year, and I was reminded of this by a recent discussion about mentoring efforts by a gnome developer looking at the approximate 25% success rate (over a variety of projects, including LilyPond).

Decentralization

A low "bus factor" means that work (or knowledge or power) is highly concentrated in a few people. It's much better for the project if work/knowledge/power is distributed more evenly -- losing one person will cause fewer problems.

There are other benefits to decentralization, of course. A lower workload on "central" people will lessen burn-out, which may encourage them to stick around for longer. That can, in turn, lessen the workload on other people: there are many cases where an "old-timer" can make a few suggestions in a 5-minute email which save a new contributor literally hours of work and frustration. Everybody wins!

Dangers of burn-out

I'm (in)famous on the LilyPond mailing lists for arguing against new proposals. "We don't have the resources / nice idea, but wait until... / you can do that with a user .ly file / etc".

In some cases this is due to my subscribing in a certain philosophy of design: Il semble que la perfection soit atteinte non quand il n'y a plus rien à ajouter, mais quand il n'y a plus rien à retrancher. (perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away; from quotes from Antoine de Sainte Exupéry). A similar idea was mentioned by Steve Jobs in a recent interview:

People think focus means saying yes to the thing you've got to focus on. But that's not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I'm actually as proud of the things we haven’t done as the things I have done. Innovation is saying 'no' to 1,000 things.

But the other part is that I simply don't want more work. Every single time that somebody comes up to me with an excited new idea, it breaks my heart because my first thought is "oh boy, now I need to express polite enthusiasm". It doesn't matter if it's over email, in a chat, in a meeting, or in a private conversation. Doesn't matter if it's from a longtime developer or an unknown user. Doesn't matter if I think it's a brilliant idea, an obvious but useful improvement, or a pie-in-the-sky ludicrous concept. My first thought is always the same: "bleh, more work".

At the moment, I'm committed to doing 10 hours of LilyPond work per week. I've "saved up" a few hours from the previous two months (including my vacation) because I knew that I'd be busier after 2.14 came out -- at the time of this writing, I have 12 hours of LilyPond time left. Each Monday morning, I add 10 hours to that figure. By my estimation, only considering the published Grand Organization Project and Grand LilyPond Input Syntax Standardization backlogs, I won't be able to do make any LilyPond bugfixes or new features of my own until the end of 2011. My entire time will be spent organizing and helping other people to do work.

You can help

Get involved! Anything that makes LilyPond less centralized will help.

Even if it has nothing to do with me personally, there will probably be a "trickle up" effect from reducing the burden of other people. For example, if a translator started doing some of the work of Francisco (the Translation Meister), he would have more time to investigate problems himself instead of asking me to help. Similarly, if a developer offers to seriously mentor one or two new programmers, that might mean one or two fewer programmers who come to me for help because everybody else is ignoring them. If you're a new contributor, then spend some extra time talking to other new contributors -- provide moral support and somebody to commiserate with, even if you can't discuss anything technical!

As for me personally... I spend about 2 hours a week on completely mindless administration. Stuff that could almost be done by a computer program. I could easily write 5 or 6 sentences on paper, and somebody could follow those blindly. For example, twice a week I send a "patches countdown" email.

  1. go to the list of patches for review
  2. pick two or three issues
  3. find the codereview URL inside the issue
  4. click on that link
  5. if there are any reviews, check that the last review is not negative
  6. include the subject line and link to the codereview page in an email to lilypond-devel

All I need to offload this task is a reliable person who doesn't mind a bit of boredom offering to do it. 15 minutes, twice a week. If there's anything unusual, I'm happy to discuss it (and possibly modify the instructions to handle such cases). But I'm not asking for any knowledge or creativity.

There are other tasks which would require knowledge and/or creativity. For example, many people ask for a changelog -- what's new in each release? If somebody is interested in lilypond development, they could spend 1-2 hours a week reading the developer email list and the git changelog, then writing a 3-4 sentence summary for each release. That would be a nice "gentle" introduction to how things work, and it would benefit many users.

Similarly, there are some tasks which are regular weekly affairs, and others which are one-time. Here's a great non-really-technical one-time task: our list of places to send release announcements to is quite old. Half of the links don't work. So I haven't bothered to make any release announcements (other than our own webpage) at all. If somebody wanted to spread the word, they could spend 2-5 hours looking for relevant places to announce lilypond, maybe asking for other suggestions from lilypond-user, and make a nice list. Then either send our release announcement to those places himself, or else ask me to do it. I wouldn't mind doing it as long as all the links work and I don't need to hunt around to find out how to send the announcement (i.e. give me a specific link to the announcement form, not a generic link to the main website!)

Square pegs into round holes

In the past, we haven't done a good job of organizing people in LilyPond. Some people are suited to regular work; other people are suited to one-time jobs. Some people like technical challenges and creative freedom; other people prefer a rigid set of guidelines so they know that they're not making any mistakes.

We need people for all sorts of tasks, and over the next few weeks, I'm really going to try to make sure that everybody has a job that suits them. We've lost a lot of potential effort by asking people to do tasks which don't really appeal to them, and then not having those jobs "open" when another person offers to help.

Vivi's first academic publication: SMC 2011

The first paper about Vivi was published in the Sound and Music Conference 2011. It is published under a Creative Commons license:

percival-smc2011.pdf

In other news, I'm still working on the refactoring of the source code. I barely got any work done during my 3-week vacation, which I guess could be considered to be a good thing?

I'll be making some posts about general-interest parts of the vacation over the next few days or weeks, but at the moment my focus has to be work. I spent 3 days in bed (almost) recovering from exhaustion, so I've got even more work to catch up on.

Artifastring 1.4

Artifastring 1.4 is now out (now including video output!), along with a heartwarming story: another person is working on this project! Welcome to Marcos Press ("tdy")!
http://percival-music.ca/artifastring/

I introduced Vivi to the lilypond-user mailing list, sparking a long discussion, on both the technical accomplishments of Vivi and the musical implications of such work. But in the middle of it, I received a very kind offer of a violin model in blender. Blender is a Free 3D content creation suite; amonst other things, it can create animations. I was using that for Vivi already, but only with very primitive output -- 4 cylinders for the strings, 1 cylinder for the bow, cones for the fingers, etc.

He sent along an image of the violin, and it looked quite nice. So I wrote back to say that I was very interested, and did he also have a bow model?

He replied that he didn't have a bow model yet, but that he'd do it right away, because the bow was easier to do than a violin. A few hours later, he sent me a file with both models.

The small picture I saw earlier did not do it justice. His models were literally jaw-dropping. It includes a sound post. It includes the wires attaching the tailpiece to the knob at the bottom of the violin. It includes curved f-holes. The strings have different colors of windings. The tailpiece has actual holes -- or rather, not just circular holes, but correctly-shaped holes -- into which the strings fit. The bridge has notches in which the strings fit. Ditto for the nut. The strings wind around the tuning pegs.

The below picture doesn't do it justice, either... I should have rotated the violin a bit more so that you could see more details of the bridge. But spending the whole blog post talking about a video without showing any graphics would be a bit weird.

Frame from video (click to enlarge)

After I got the file, I must have spent about 30 minutes just looking at the model. Zooming in, zooming out, rotating... everywhere I looked, there was an incredible amount of detail. I need to learn how to make a "fly-by animation" in blender, to show off all the pieces of this.

The model also included some text inside the violin, like a "maker's mark". I didn't notice it for the first 20 minutes or so, but when I read it, I almost cried.

from tdy's 1st blender modeling for Graham Persival Vivi System Vivi's 2nd Violin Loving OpenSource 2011

Now, I've been working on open-source projects for a long time... since 2003, actually. I've released my own software as open source. At times, I've spent over 60 hours a week on existing open-source projects. I've trained approximately 30 people to do documentation work (admittely, about half of them just read the guidelines I wrote and didn't need any further guidance). I've directly caused literally hundreds of hours of work, mostly on LilyPond. I've reviewed patches, mentored new contributors, and seen those contributors to become much more knowledgeable than me and tell me off for things (that's my favorite part of mentoring :). I've had people express interest in work that I'd abandoned, and I passed the source code along to them.

But this is the first time that somebody's sent a contribution to one of my own projects, and it's a whole new feeling.


I also feel even more inspired to make Vivi better -- with Marcos' work, the sound quality is much worse than the video quality. I need to improve her bow control much more. I was already planning on doing that, of course, but now I have even more motivation. :)

Here's an example of the videos it produces:
Black-box testing

So again, Marcos -- thank you so much! To other people: I've got a github repository now, and more bits and pieces will be coming in the next few days. What are you waiting for? :)

https://github.com/gperciva/artifastring