Thursday, May 15, 2014

TrackDynamics

For the last several days I've been working on one of those cases where you pull on a little root and wind up uprooting half the yard.

The initial problem was a note came out an octave too high through midi thru.  It turned out this was due to how I implemented instruments playing in octaves, and getting the instrument (pemade) from one instantiation of the score, while getting the transpose signal from the kantilan instantiation of the score.  Cmd.Perf should show a single consistent picture of a block, from only one call.

This led me to investigate how TrackDynamics were collected, which has always been a terrible hack, due to band-aids added during several past debugging sessions.  The problem arises when tracks are sliced during inversion.  I wind up with a bunch of fragmentary signals, and each note track is evaluated twice, once before and once after inversion.  And since inversion means a note track appears twice, which one do I want for TrackDynamics?  Well, the one on the bottom, because that one has the scale set.  But its controls should come from the one on top, otherwise, it sees a slice of the controls of one particular note.

So you see, all problems all eventually lead back to track slicing.  That single feature has been responsible for more headache and bugs and hacks than any other feature, probably by an order of magnitude.

Anyway, TrackDynamics was just the start, because that lead me to become unhappy with how orphan slicing works.  The connection was that I needed to evaluate track titles to get the TrackDynamics right, but slicing wanted to strip out empty tracks, so it needed an extra step where it extracted the titles so they could be evaluated separately.  And it did that in the special orphan slicing code, but not normal note slicing... or something like that.  It also made me unhappy because I wanted to simplify TrackDynamics collection to just take the first one for a track and not do any hairy and order-dependent merging, and orphan extraction was a source of especially unintuitive evaluation order since they were evaluated out of order with the rest of the track.  In any case, I thought that I could just make a note track evaluate its subtracks directly if it doesn't have any events, and then just get rid of the empty track stripping and separate title extraction and evaluation mess.

Well, that was the first plan.  The problem is that this made the orphan slicing code get increasingly complicated because it then had to handle orphans below multiple levels.  It worked out recursively as part of track derivation before, but trying to get a single list of orphan regions to derive meant I had to do it in the slice function, which is the same as how note slicing worked.  It became increasingly complicated, and increasingly duplicating note slicing, so I figured I could get rid of the whole thing by having note tracks do a note slice on empty regions, and directly deriving what came out.

That got rid of orphan slicing entirely, which was great... except it led to some further problem I forget, but was related to a totally empty track suddenly turning into a separate slice for each of the notes under it.  In any case, I really wanted a plain slice, because I wasn't taking advantage of the separate note slices.

So that was the final iteration: back to a special slice_orphans (I was worried I wouldn't be able to use that name!), but it's just a very simple wrapper around a plain slice.

Aside from all the changes, there were two messes in there.  The first was all the various TrackTree fields used to track slice information, like 'tevents_range' and 'tevents_shifted', and 'tevents_end'.  Initially slicing just sliced events.  But I discovered that I still needed track information that was lost, such as the next event, or the position of the event in TrackTime (e.g. for cache invalidation), and added fields in an ad-hoc way as I needed them.  Because they were confusing, I put extensive comments on each one, but reading the comments later I still didn't understand, and existing code seemed to use them in an inconsistent way.  Eventually I figured that out and was able to get rid of tevents_range entirely.

The second was overlap detection.  This was an endless game of whack-a-mole, fixing one test would break another, fixing the other would break the first.  The problem is that I can't keep the whole thing in my head, so all my concentration is used on just solving one problem, and I don't notice when there's some underlying contradiction that makes the whole thing impossible.  Trees are always this way, I can't understand anything two dimensional.  I drew lots of ASCII diagrams and finally figured out something that seemed to work.

So after all that... I think it was overall a good thing.  Slicing is simplified a bit, and it needs all the help it can get.  And getting back to TrackDynamics, that was easy enough to solve once I'd gotten the distractions out of the way.

But wait... there's more!  This whole mess turned up that gender ngoret didn't work properly in several cases because it couldn't get the pitch of the previous event.  That took me down the rabbit hole again, because it annoyed me that something as simple as getting the previous note should be so error-prone... due to slicing, as usual.  To make another long story short, I thought up solution A, implemented it halfway, thought up solution B and implemented it most of the way with a feeling of deja vu.  Then I stumbled across a large comment explaining solutions A, B and C, and why A and B wouldn't work, so I implemented C.  It was even cross referenced in several places to avoid exactly the problem than wound up happening.  Part of the problem was that I left part of A in place, for performance reasons, and this let me to entirely forget about the whole thing.  Anyway, I eventually decided the reasons B wouldn't work could be worked around, and, after a few bug fixing cycles, finally got it apparently working.



Stepping back, all of this came about trying to figure out how to write pemade and kantilan parts for gender wayang pieces.  They are the same, except kantilan is an octave higher.  Sounds really simple, right?  Well they should be evaluated separately, so variable parts come out differently.

So I need to derive the score twice, once with the >polos = >pemade-umbang | >sangsih = >pemade-isep (abbreviated in score to >p = >p-umbang etc.), and once with >polos = >kantilan-umbang | >sangsih = ... | %t-diatonic = 5.  That in turn required me to add the instrument aliasing feature so >x = >y would work.  And of course the %t-diatonic line turned up the TrackDynamic problem that kicked this whole thing off.  All this just to play the same thing up an octave, and on different instruments!

And I'm not even satisfied with this setup because I wind up with 'pemade' and 'kantilan' blocks that just set various fields and then call the 'score' block.  And that's just to get the text out of the track titles, since block titles wrap to display long text while track titles scroll, so they can't accept anything long.  It could also go in the event text, which also wraps, but... oh I don't know, maybe it should do that.

This approach is means if I want pemade and kantilan to differ, I have to come up with some conditionals so I can give alternate score sections (also not supported yet, also on the TODO list).  An approach based on integration would be to extend score integration to make a copy of an entire hierarchy of blocks.  That way I have an "ideal" part, and then pemade and kantilan realizations, which can have individual edits, but still merge changes made to the ideal part.  But while score integration already does this for a single block, a whole hierarchy of calls seems like a whole other thing.  Or perhaps I can just write a function that sets up a bunch of score integrates.


I remember a time, I don't know how many years ago, in the attic on a Saturday.  At that time I had no wife and no girlfriend, and Saturday was free from dawn to midnight.  I resolved to spend the whole time working on the sequencer.  I was working on the skeleton operations, which are simple graph operations: add an edge, remove an edge, and the like.  It was being extremely difficult, and progress was extremely slow, but I was determined to force my way through sheer bloody-mindedness.  Perhaps sometimes problems solve themselves when you stop paying attention to them, but in my experience they more often just don't get solved.  I thought, I will do this now, and complete it, it will be done and I won't have to come back to it, and so I will focus until it's gone, and then move on.  It was really hot, and I thought that in any endeavour there will be times of agonizing slow progress on something that seems so far removed from your goal, and days spent in grinding drudgery.  Every journey has moments of despair, perhaps many of them.  The only way out is through.