Thursday, May 15, 2014

TrackDynamics

For the last several days I've been working on one of those cases where you pull on a little root and wind up uprooting half the yard.

The initial problem was a note came out an octave too high through midi thru.  It turned out this was due to how I implemented instruments playing in octaves, and getting the instrument (pemade) from one instantiation of the score, while getting the transpose signal from the kantilan instantiation of the score.  Cmd.Perf should show a single consistent picture of a block, from only one call.

This led me to investigate how TrackDynamics were collected, which has always been a terrible hack, due to band-aids added during several past debugging sessions.  The problem arises when tracks are sliced during inversion.  I wind up with a bunch of fragmentary signals, and each note track is evaluated twice, once before and once after inversion.  And since inversion means a note track appears twice, which one do I want for TrackDynamics?  Well, the one on the bottom, because that one has the scale set.  But its controls should come from the one on top, otherwise, it sees a slice of the controls of one particular note.

So you see, all problems all eventually lead back to track slicing.  That single feature has been responsible for more headache and bugs and hacks than any other feature, probably by an order of magnitude.

Anyway, TrackDynamics was just the start, because that lead me to become unhappy with how orphan slicing works.  The connection was that I needed to evaluate track titles to get the TrackDynamics right, but slicing wanted to strip out empty tracks, so it needed an extra step where it extracted the titles so they could be evaluated separately.  And it did that in the special orphan slicing code, but not normal note slicing... or something like that.  It also made me unhappy because I wanted to simplify TrackDynamics collection to just take the first one for a track and not do any hairy and order-dependent merging, and orphan extraction was a source of especially unintuitive evaluation order since they were evaluated out of order with the rest of the track.  In any case, I thought that I could just make a note track evaluate its subtracks directly if it doesn't have any events, and then just get rid of the empty track stripping and separate title extraction and evaluation mess.

Well, that was the first plan.  The problem is that this made the orphan slicing code get increasingly complicated because it then had to handle orphans below multiple levels.  It worked out recursively as part of track derivation before, but trying to get a single list of orphan regions to derive meant I had to do it in the slice function, which is the same as how note slicing worked.  It became increasingly complicated, and increasingly duplicating note slicing, so I figured I could get rid of the whole thing by having note tracks do a note slice on empty regions, and directly deriving what came out.

That got rid of orphan slicing entirely, which was great... except it led to some further problem I forget, but was related to a totally empty track suddenly turning into a separate slice for each of the notes under it.  In any case, I really wanted a plain slice, because I wasn't taking advantage of the separate note slices.

So that was the final iteration: back to a special slice_orphans (I was worried I wouldn't be able to use that name!), but it's just a very simple wrapper around a plain slice.

Aside from all the changes, there were two messes in there.  The first was all the various TrackTree fields used to track slice information, like 'tevents_range' and 'tevents_shifted', and 'tevents_end'.  Initially slicing just sliced events.  But I discovered that I still needed track information that was lost, such as the next event, or the position of the event in TrackTime (e.g. for cache invalidation), and added fields in an ad-hoc way as I needed them.  Because they were confusing, I put extensive comments on each one, but reading the comments later I still didn't understand, and existing code seemed to use them in an inconsistent way.  Eventually I figured that out and was able to get rid of tevents_range entirely.

The second was overlap detection.  This was an endless game of whack-a-mole, fixing one test would break another, fixing the other would break the first.  The problem is that I can't keep the whole thing in my head, so all my concentration is used on just solving one problem, and I don't notice when there's some underlying contradiction that makes the whole thing impossible.  Trees are always this way, I can't understand anything two dimensional.  I drew lots of ASCII diagrams and finally figured out something that seemed to work.

So after all that... I think it was overall a good thing.  Slicing is simplified a bit, and it needs all the help it can get.  And getting back to TrackDynamics, that was easy enough to solve once I'd gotten the distractions out of the way.

But wait... there's more!  This whole mess turned up that gender ngoret didn't work properly in several cases because it couldn't get the pitch of the previous event.  That took me down the rabbit hole again, because it annoyed me that something as simple as getting the previous note should be so error-prone... due to slicing, as usual.  To make another long story short, I thought up solution A, implemented it halfway, thought up solution B and implemented it most of the way with a feeling of deja vu.  Then I stumbled across a large comment explaining solutions A, B and C, and why A and B wouldn't work, so I implemented C.  It was even cross referenced in several places to avoid exactly the problem than wound up happening.  Part of the problem was that I left part of A in place, for performance reasons, and this let me to entirely forget about the whole thing.  Anyway, I eventually decided the reasons B wouldn't work could be worked around, and, after a few bug fixing cycles, finally got it apparently working.



Stepping back, all of this came about trying to figure out how to write pemade and kantilan parts for gender wayang pieces.  They are the same, except kantilan is an octave higher.  Sounds really simple, right?  Well they should be evaluated separately, so variable parts come out differently.

So I need to derive the score twice, once with the >polos = >pemade-umbang | >sangsih = >pemade-isep (abbreviated in score to >p = >p-umbang etc.), and once with >polos = >kantilan-umbang | >sangsih = ... | %t-diatonic = 5.  That in turn required me to add the instrument aliasing feature so >x = >y would work.  And of course the %t-diatonic line turned up the TrackDynamic problem that kicked this whole thing off.  All this just to play the same thing up an octave, and on different instruments!

And I'm not even satisfied with this setup because I wind up with 'pemade' and 'kantilan' blocks that just set various fields and then call the 'score' block.  And that's just to get the text out of the track titles, since block titles wrap to display long text while track titles scroll, so they can't accept anything long.  It could also go in the event text, which also wraps, but... oh I don't know, maybe it should do that.

This approach is means if I want pemade and kantilan to differ, I have to come up with some conditionals so I can give alternate score sections (also not supported yet, also on the TODO list).  An approach based on integration would be to extend score integration to make a copy of an entire hierarchy of blocks.  That way I have an "ideal" part, and then pemade and kantilan realizations, which can have individual edits, but still merge changes made to the ideal part.  But while score integration already does this for a single block, a whole hierarchy of calls seems like a whole other thing.  Or perhaps I can just write a function that sets up a bunch of score integrates.


I remember a time, I don't know how many years ago, in the attic on a Saturday.  At that time I had no wife and no girlfriend, and Saturday was free from dawn to midnight.  I resolved to spend the whole time working on the sequencer.  I was working on the skeleton operations, which are simple graph operations: add an edge, remove an edge, and the like.  It was being extremely difficult, and progress was extremely slow, but I was determined to force my way through sheer bloody-mindedness.  Perhaps sometimes problems solve themselves when you stop paying attention to them, but in my experience they more often just don't get solved.  I thought, I will do this now, and complete it, it will be done and I won't have to come back to it, and so I will focus until it's gone, and then move on.  It was really hot, and I thought that in any endeavour there will be times of agonizing slow progress on something that seems so far removed from your goal, and days spent in grinding drudgery.  Every journey has moments of despair, perhaps many of them.  The only way out is through.

Sunday, October 6, 2013

umbang isep, sangsih polos

I finally finished the wayang samples, which means I need to start getting serious about notation for Balinese-style music.  The most basic level of that is dealing with pengumbang and pengisep, and polos / sangsih pairs.  Namely, I need notation that takes a single part and splits it into parts, whether unison, kempyung, or kotekan.  And of course, every time I think I have a general powerful system that can handle anything, I find out that even the most basic fundamental notation will require adding new features, or further generalizing existing ones.

I initially wanted to implement the "pasang" calls (which is my name for instruments that are actually pairs of instruments) by setting a part=polos or part=sangsih environ val, and letting the default note call apply the appropriate instrument.  But then I ran into the core problem which seems to underlie just about all notation problems: order of evaluation.  Namely, if I want to apply these calls to arbitrary notation rather than just single notes, they have to be postproc calls.  Only then do I have access to the flat stream of notes, and know how fast the concrete tempo is (to know if I can add noltol or not).  But at that point it's too late to ask the note call to emit sangsih or polos or both.

So the note call emits the nonexistent "pasang" instrument (e.g.  ">wayang"), and a postproc later looks at inst-polos and inst-sangsih env vars to split that into, say ">wayang-umbang" and ">wayang-isep", which are real instruments and have allocation.  Now we have yet another order of evaluation problem: by the time the postproc switches the instrument, the pitch is already determined.  Pitches delay evaluation and can continue to be affected by controls, but not environ, and the tuning is environ, not a control, so it's too late to change the tuning.



    Notes from the TODO file:

    Tuning is incorrect if postproc changes the instrument, because the new instrument doesn't bring its tuning along with it.  I could fix that by having the instrument bring its environ into scope during conversion as well as derivation.

    • The easiest way is to move environ to Patch.  But this means Derive needs to look up Patch, or at least get a function for it.
    • But Environ is technically code, since it has PitchControl, which has has a pitch signal.  So I can't save it unless I make a RawEnviron out of RawVal.  Hassle and complicated.
    • Wait, environ is no good, because pitch doesn't take environ.  It's already baked in by the time the pitch is realized.  So I'd have to represent umbang/isep as a signal, e.g. umbang=0 isep=1.  It's a bit sketchy because umbang-isep is not additive, while PitchSignal.appy assumes all pitch-affecting signals are additive.  Do I want to keep the env var?  Try removing for now.  But for this to work, inst has to be able to introduce signals.  If I added that, it would basically be a one-off for this case.
    • Now making pitches take environ looks more attractive: it means I can change the key after the fact too, the environ replaces rather than adds, and possibly even fits better with a mythical future unification of environ and signals.
    • On the other hand, letting the instrument introduce signals could be a useful generalization.  Also, it seems like an umbang-isep signal could be a useful generalization, because it lets me explicitly place a pitch in the center.  Actually, %ombak=0 already does that.  So it's actually kind of non-orthogonal.
    • Which is better?  I'm going with adding Environ to the pitch.  Who knows if that is the right choice, but I have to choose something.

    Goals for today:

    • Get this umbang/isep thing sorted out.
    • Practice mridangam, lesson on Tuesday!
    • Practice kendang for Semara Dahana to get ready for lesson on Friday.
    • Sort out samples for mridangam and kendang, and figure out how to automatically infer dynamic levels.

    Saturday, September 7, 2013

    keyboards

    2013-09-07

    A common theme of a lot of my posts is me noting how something that seems simple is actually really complicated, followed by a long exposition of how complicated it really is.  I guess that's because I'm continually surprised by how much work it is to do anything.  Even in haskell, a language which is supposed to conducive to expressing complicated things.

    By writing these expositions I'm trying to answer a question that is always in my mind while programming.  That is, is music just inherently complicated, or am I failing to see some important abstraction or generalization or even specialization that would simplify things?  When I struggle through some never-ending feature-add or bug-fix session I'm wondering what I did wrong, and how I could avoid it next time.

    Of course this is one of those perennial questions about programming, about inherent versus incidental complexity.  People who are good at what they do, and good languages, are supposed to reduce the latter.  I'd like to do that, to become good at what I do, and ultimately save time.

    Today I wanted to finish a feature to allow piano and ASCII keyboard input for scales with varying numbers of degrees, then slice up and create a sampler patch for the recorded kendang samples from last week, practice mridangam, and then practice kendang (using the new patch).  The topeng dancer is coming to tomorrow's rehearsal and I'd like to practice the cedugan part so as to not embarrass myself too badly.

    It's looking like the feature add is not going to get done in time, so I'm putting it on hold while I go to practice.  I've actually been working on it for a full week, on and off.  It's so hard that it's discouraging, and that slows me down.  And it sounds so simple!


    The basic problem is that I have input, coming from either a piano-style MIDI keyboard, or the ASCII keyboard mapped to an organ-like layout, and I need to figure out what note in the current scale should come out.

    Previously I had a (somewhat) straightforward way.  The input key was a Pitch.InputKey, which was just a newtype around an integer, which was the number of semitones from MIDI note 0, otherwise known as the Midi.Key.  Then it was up to the scale to map from InputKey to a Pitch.Note.

    This worked well enough at first, but when scales got more complicated it started to have problems.

    It started with relative scales, e.g. sa ri ga style.  I wanted sa to always be at a constant place, so I had C emit sa.  That works fine diatonically but runs into trouble when you start using accidentals.  If D major is being played starting at C, the black keys are in the wrong place, and you wind up with some notes unplayable.  It got worse when I tried to get Bohlen-Pierce to map right.  It has 9 diatonic and 13 chromatic steps per "octave" (tritave, actually), so its layout is nothing like a piano keyboard's.

    So it seemed like I should treat the ASCII and piano keyboards differently, so I could make the ASCII keyboard always relative, while the piano keyboard would remain absolute as usual.  So the InputKey would get its own structure, with octave, pitch class, and accidentals.  This involved updating Pitch.InputKey (and renaming Pitch.Input while I was at it), and then redoing how all the scales converted input to symbolic pitches, and that involved a bunch of refactoring for scales, and then fussing around with octave offsets to get them roughly consistent across scales.

    The actual conversion is a bit of head-hurting modular arithmetic, to convert from a 7 or 10 key keyboard to an n-key scale, wrapping and aligning octaves appropriately.  In theory it's straightforward, but finicky and hard for me to keep straight in my head.

    But now it all seems to work, and I can get rid of an awful hack that hardcoded Bohlen-Pierce in favor of all scales adapting based on the number of pitch classes.

    Worth it?  Maybe?  Pitch input is what I spend most time on when actually writing music, so anything that makes that smoother seems worthwhile.


    What else?  Slow progress lately, since I've been distracted by travel and sampling.  Speaking of which, I recently finished the patches for the gender wayang.  That's another thing that sort of boggles the mind.  Pemade and kantilan, with umbang and isep variations, ten keys on each, with four dynamics and from three to eight variations of each, played with gender panggul and calung panggul, and muted kebyar style and loose gender style.  That's 4,313 samples recorded, edited, organized, and mapped to the right key, velocity range, round robin configuration, and envelope.  2.6gb worth, and it took two weekends to record and several weeks on-and-off to do the editing.  Even with lots of automation, it's a lot of work, and you need a well defined process and lots of notes to not make mistakes.  There are still tweaks to be made, with jagged dynamics to be smoothed and missing articulations to be rerecorded.  I guess that's another thing that's complicated about music.

    And I'm not done yet, I still need to do the gender rambat and reyong / trompong, both of which have more keys.

    Saturday, June 22, 2013

    sketch-13-06-19

    A couple of days ago, I had an idea while biking home.  So when I got home I wrote it down and then started fiddling to see where else it would go.  And it basically turned into a piece, which is now in need of wrapping up, and some name better than "sketch-13-06-19".

    That's a bit of a milestone, because it's the first time I've used the sequencer to actually write something beyond a notation test, or a transcription of an old piece.  It's not really what I would have chosen, since it came about by happenstance.  It's the sort of intricate bit of video-gamey clockwork that I was doing back in high-school, and is mostly in piano because that was a convenient sound when I was writing the sketch and now it's a bit stuck there, and it doesn't really take advantage of any interesting new capabilities.  I wasn't trying to create something controlled, and so I got a bit of a tangled undergrowth.  As ever I'm not entirely in control of what catches and what slips away.

    But I'm still pretty happy.  In a way I've been waiting thirteen years for this, and working four years on a program to get there (that would be eight years, if you count on-and-off work).  This is the first hint of the fruition of those four years, the hint that those four years may not have been in vain.  Worth it?  For whatever reasons, I don't think it would have happened any other way, perhaps because of my determination to write a program to write music, and then determination that the time I spent on the program not be a waste.  In creative endeavours we so rarely reach our goals from straight lines.

    Despite all that, it's funny how little has changed.  I'm infinitely richer in resources, influences, sounds... but I've basically picked up right where I left off.  But that's fine, the fancy things can wait.  The real job is learning to write music, and that will take many more than eight years.  I'm not in a hurry.

    Sunday, April 21, 2013

    piano sketch


    I just opened an old saved score, and played it.  Hmm, sounds like a nice little sketch, I should do something with that someday.  But the dynamics are broken... oh right, because it uses the old-style 'vel' control, and not the new 'dyn' one.  It also uses the old decimal notation, instead of hex, but it's still compatible.  Then I noticed what I thought was a weird graphical glitch that made the track skeleton look backwards, until I realized... oh right, no track slicing back then so it was all backwards.  I created a new track and it had a messed-up looking ruler.  I was about to make note of a new bug when I realized... oh right, this has old style overlay rulers, which are obsolete now.

    It's easy to forget how much things have changed.

    Saturday, March 23, 2013

    slicing

    One week of head-hurting work, all because a slur from a non-harmonic to a harmonic wasn't showing up.  It turns out it was because track slicing, always an unintuitive mess, wasn't working how I thought it did.  In fact, it was hard to think of how it worked, because it was an unintuitive mess.  So it took ages to sort out because first I had to figure out what it was doing, figure out if that made sense, figure out what would make sense, and then implement that.

    Add in a couple false starts and having to update tons of tests and it winds up eating up quite a bit of time.  The tons of tests are a result of it being too complicated in the first place, so it's a good demonstration of how complexity consumes an non-linear amount of time.  Added to that is that I'm dealing with trees which have two-dimensional structure, which are hard to visualize and to write down (in fact the tests have ASCII-art diagrams next to them).  Actually, I think the greatest time sink is general disillusionment.  When I'm unsatisfied with the whole situation I spend a lot of time trying to step back to figure out what the real problem is, or getting distracted because there's no real sense of momentum to keep my focus, or playing games because I'm just too tired to want to deal with the hairball.

    The result is that I have a moderately more useful way of turning the 2D score structure into notes and transformations, and a check which I think should cause an error for events that overlap in the wrong way, which tend to cause doubled notes or missing notes or other unintuitive things.

    The general problem is one of composing together 2D score structure, or rather turning it into the 1D structure needed to actually evaluate it.  The score can get quite complicated, because there are transformations that apply to all notes, those that only apply to one melodic line, those that apply only to lilypond realization, and those that only apply to synthesizer realization.  But the key to having a usable score notation is that the individual parts compose together in a natural and useful way.


    Sunday, January 20, 2013

    spelling


        4/4
      0 w                               | 32
      1 h               h               | 16
      2 q       q       q       q       | 8
      3 e   e   e   e   e   e   e   e   | 4
      4 s s s s s s s s s s s s s s s s | 2
      5 33333333333333333333333333333333| 1
    0 0 w-------------------------------| 32
    5 3  s.-|                             3
    4 2   e.----|                         6
    5 3    3|                             1
    3 1     q.----------|                 12
    5 3      s.-|                         3
    4 2       s-|                         2
    5 3        3|                         1
    2 0         h.----------------------| 24
    5 3          s.-|                     3
    4 2           e.----|                 6
    5 3            3|                     1
    3 1             e---|                 4
    5 3              s.-|                 3
    4 2               s-|                 2
    5 3                3|                 1
    2 0                 h---------------| 16
    5 3                  s.-|             3
    4 2                   e.----|         6
    5 3                    3|             1
    3 1                     q.----------| 12
    5 3                      s.-|         3
    4 2                       s-|         2
    5 3                        3|         1
    2 0                         q-------| 8

        3+3/8
      0 h.                      h.
      1 q.          q.          |
      2 e   e   e   e   e   e   |
      3 s s s s s s s s s s s s |
    0 0 h.----------------------|
    4 3  s.-|
    3 2   s-|
    4 3    3|
    2 1     q-------|
    4 3      s.-|
    3         s-|
    4          3|
    2           e---|
    4            s.-|
    3             s-|
    4              3|
    1               q.----------|
    4                s.-|
    3                 s-|
    4                  3|
    2                   q-------|
    4                    s.-|
    3                     s-|
    4                      3|
    2                       e---|
    4                        s.-|
    3                         s-|
    4                          3|

        2+2+2/8
      0 h.
      1 q       q       q       |
      2 e   e   e   e   e   e   |
      3 s s s s s s s s s s s s |
      4 333333333333333333333333|
      0 h.----------------------|
    4 2 3s.-|
    3 2 2 s-|
    4 2 3  3|
    2 1     e---|
    4 2      s.-|
    3 2       s-|
    4 2        3|
    1 0         h---------------|
    4 2          s.-|
    3 2           s-|
    4 2            3|
    2 1             e---|
    4 2              s.-|
    3 2               s-|
    4 2                3|
    1 0                 q-------|
    4 2                  s.-|
    3 2                   s-|
    4 2                    3|
    2 1                     e---|
    4 2                      s.-|
    3 2                       s-|
    4 2                        3|

      0 h.
      1 q               q               q               |
      2 e       e       e       e       e       e       |
      3 s   s   s   s   s   s   s   s   s   s   s   s   |
      4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 |
      5 666666666666666666666666666666666666666666666666|
    0 0 h.----------------------------------------------|
    5 3  3.-|
    4 2   s.----|
    5 3    6|
    3 2     s---|
    5 2 3    3.-|
    4 2 2     3-|
    5 2 3      6|
    2 1         e-------|

    I spent the weekend drawing diagrams like those.  A musician might recognize them as the "optimal" way to spell a note beginning at each point in a measure.  Since my score uses a low level start time + duration concept, I need to figure out how to spell rhythms automatically when I convert to lilypond.  Previously I used a quick and simple technique that worked surprisingly well, but only for duple.  Since it made a 6/8 piece totally ugly I decided it was time to solve the problem for real.  That turns out to be either way more complicated than I thought it would be, or I'm doing it totally wrong!

    In any case, it was an exercise in design-by-diagram, since I'm definitely not smart enough to visualize rhythmic notation in my head.

    First strategy: try to end up on a major division which isn't over the barline.  So define a "rank" for each possible position in the measure (I only go to 128th notes, so there are 128 per whole note), and pick the duration (possibly including dotted durations) which ends on the lowest rank.  Exploring this with diagrams, it seemed to line up with my expectations:

      0 w                               | 32
      1 h               h               | 16
      2 q       q       q       q       | 8
      3 e   e   e   e   e   e   e   e   | 4
      4 s s s s s s s s s s s s s s s s | 2
      5 33333333333333333333333333333333| 1
    5 3  s.-|                             3
      5  w-------------------------------|
      5  h.----------------------|
      5  h---------------|
      5  q.----------|
      5  q-------|
      5  e.----|
      5  e---|
    * 3  s.-|
      5  s-|
      4  3|

    4 2   e.----|
      >   w-------------------------------|
      4   h.----------------------|
      4   h---------------|
      4   q.----------|
      4   q-------|
    * 2   e.----|
      5   e---|
      4   s.-|
      3   s-|
      4   3|

      0 w                               | 32
      1 h               h               | 16
      2 q       q       q       q       | 8
      3 e   e   e   e   e   e   e   e   | 4
      4 s s s s s s s s s s s s s s s s | 2
      5 33333333333333333333333333333333| 1
    5 3    3|                             1
      >    w-------------------------------|
      5    h.----------------------|
      5    h---------------|
      5    q.----------|
      5    q-------|
      5    e.----|
      5    e---|
      4    s.-|
      4    s-|
    * 3    3|

      0 w                               | 32
      1 h               h               | 16
      2 q       q       q       q       | 8
      3 e   e   e   e   e   e   e   e   | 4
      4 s s s s s s s s s s s s s s s s | 2
      5 33333333333333333333333333333333| 1
    3 1     q.----------|                 12
      >     w-------------------------------|
      4     h.----------------------|
      3     h---------------|
    * 1     q.----------|
      3     q-------|
      4     e.----|
      2     e---|
      5     s.-|
      4     s-|
      5     3|

      0 h.
      1 q.          q.
      2 e   e   e   e   e   e   |
      3 s s s s s s s s s s s s |
    4 3  s.-|
      >  w-------------------------------|
      >  h.----------------------|
      4  h---------------|
      4  q.----------|
      4  q-------|
      4  e.----|
      4  e---|
    * 2  s.-|
      4  s-|
      3  3|

    Unfortunately this doesn't quite work, but of course I had to implement it and run tests to find out.  Given 6/8, interpreted as 3+3/8, it wil happily put a half-note at the 3rd eighth note, because that ends at the end of the measure, rank 0 by definition.  But that obscures the middle of the measure, which is forbidden:

      0 h.
      1 q.          q.
      2 e   e   e   e   e   e   |
      3 s s s s s s s s s s s s |
    0 0 h.----------------------|
                h---------------| <- was
                q.----------|
                q-------|
                e.----|
                e---|             <- should have been

    Or rather, it's forbidden, but only for notes which start on rank 2!  The dotted half-note at the beginning of the measure can span it, because it starts at rank 0.  So I switched strategies: a note can extend until it reaches a division it's not allowed to span.  "Not allowed" means one rank lower for complex meters, and two ranks lower for duple ones.  Duple meters fit better with the log2 nature of note durations, so they can be more liberal in their spelling.

    Unfortunately, this also has a problem: a 5/4 measure can start off with a whole note.  That means a measure-long note would be spelled c1~c4, which is super ugly.  If I were spelling 5/4, I would need to know whether it is mostly 3+2/4 or 2+3/4 and tie accordingly.  The underlying problem is that first time in the measure is rank 0, so it will happily span the center of the measure.  Normally that's fine, but for irregularly divided measures the spelling needs to be more conservative to make the irregular division clear (and because a single note won't fit).  So I tweaked the definition of irregular measures to give the toplevel major divisions rank 0 instead of just the barline, and never cross rank 0 no matter what.  It winds up doing a few things I wouldn't do, e.g. I would use liberal duple spelling in the duple portion of a compound meter while it doesn't, but tolerance for complex spelling is individual and it doesn't hurt to be a little more conservative.

    There remains yet another problem: this spells rests incorrectly.  Since it always tries to find the longest duration that will fit, if I have dotted rests disabled, I wind up with "c4 c2 c4".  Rests are different from notes in that they are all interchangeable, so if there are two rests in a row they should be arranged to align with major beats: "c4 r4 r2".  The solution to this was to bring back the sort-by-rank idea, but only for rests.  If I'm trying to spell a rest, find the maximum duration as above, but then try all of the durations that fit within it, and pick the one that ends on the lowest rank.

    But wait!  There's one last wrinkle.  At least it's the last one I'm going to worry about today.  While I normally avoid dotted rests, they look better for the major divisions of a triple meter.  E.g. 3+3/8 should be spelled "r4. r4." rather than "r4 r8 r4 r8".  I wound up allowing dotted rests for triple meters.  I should probably only allow them at the rank that involves the triple division (the denominator's rank), and actually technically in a complex meter like 2+3/4, I should only allow them in the triple fraction of the meter, but it seems like too much bother, so I'm just enabling them universally.

    One other practical problem that I'll have to look into is that I have to define where the ranks fall for various meters, which means I have to define each meter before using it.  It seems like it should be possible to figure out everything from just looking at the name of the meter, though I'd have to write it explicitly, e.g. 3+2/4 rather than 5/4.


    This is one of those problems where it seems like there might be some simple "best" solution.  But it also might just be a set of heuristics and a notion of prettiness, which is something people are good at but machines aren't.  When I tried to analyze my own process in deciding how to spell a note, I came up with a small set of heuristics, but discovered more and more as the meters become more complicated.

    I still feel like if I were smarter I would have been able to identify some optimal heuristic from the beginning, or at least forsee the various problems without bumbling into a problem, patching it up, and then bumbling into another.


    This is hopefully the last of the major lilypond changes.  My goal is to typeset a simple piano piece acceptably, and it's getting close.  Recent other lilypond improvements:
    • Preserve enharmonics.  That meant extending pitches so they generate not just frequencies, but a symbolic note name, applying whatever diatonic and chromatic transpositions.
    • Tuplets.  This was a whole other involved saga.  Since tuplets involve nested chunks of lilypond code, whose composite has a different duration than the sum of its parts I wound up needing to interleave score derivation with lilypond "performance".  I tried to just generate pitches, but when I would up with grace notes nested in a tuplet I realized it had to be more general.
    • Grace notes.  Easy once the work was done for tuplets.
    • A bunch of simpler things, like various articulations, ottava notation and the like.  I switched from having the lilypond interpret note attributes to having the call directly emit lilypond code if it's being derived for lilypond.  It means lilypond knowledge is scattered throughout the calls instead of consolidated in the lilypond performer, but it makes it really trivial to add a lilypond interpretation to a call, and generally simplifies things, since the call has score context.
    Before:
    After:
    The rests should be dotted but the rest looks much better.