track width

Sometimes it's the tiny details that get ridiculously complicated.

Sometimes such a feature will just seem like it's only logical that it be there, and it shouldn't be that complicated, and you add hack after hack to make it work, without re-evaluating whether it's really so simple after all and whether it's worth all the hacks.

Such a feature is the ability of the same track in different views to have different widths.  It makes sense, right?  Seems so simple.

Not really.  It means the track width has to go in the view, not the block.  A view is one window on a block.  The block has tracks, but if views are to have different widths for the same track, the view also needs a parallel list of track widths for per-view track data.  There's various bits of code to try to maintain that invariant, and behave nicely if somehow it was broken and they are different lengths.  A separate verification step checks that, among other state invariants, but now it also needs to be able to modify the state in case it wanted to fix up the broken invariant.

And then what width do tracks have in a new view?  So Blocks now also have track widths, for new view defaults.  Ok, a little confusing, but not too bad.

Now there's this other little thing... there are actually two types of Track, a Block.Track and a Block.DisplayTrack.  A normal Track supports high level notions like Collapse and Mute and whatnot, but the UI level knows nothing of these things.  A Track with Collapse set turns into a track remove and then insert of a kind of tiny degenerate track called a Divider.  So if you want the real visible width, you have to look at the DisplayTrack.  In the Block, not the View.  And then when it gets expanded again it needs its old width...  from the View!  So the function to convert Block.Track -> Block.DisplayTrack... whoops, needs an extra View parameter.  But it has to return that width separately, not in the DisplayTrack... because DisplayTrack is Block-level, not View-level, remember?

DisplayTrack updates are used to tell the UI to change block-level track settings, but width can't go there of course, it needs to have its own view-level track settings update.

Here's another thing, explained from the comment on the function that does the hackery:

This is a nasty little case that falls out of how I'm doing diffs: First the view diff runs, which detects changed track widths.  Then the block diff runs, which detects changed tracks.  Replaced tracks (remove old track, insert new one with default width, which as a new track should get the default width in all views) are not distinguishable from a merely altered track (which can look like remove old track, insert new one with the same width, and should keep its width in each view).  So what I do is assume that if there's an InsertTrack and a corresponding TrackView in view_tracks of the new state, it should get the width given in the view.

But wait!  There's more!  If it's a collapsed track then the track_view_width is not the visible width, so don't emit TrackWidth updates in that case.

Later on we have this other fun comment:

The track view info (widths) is in the View, while the track data itself (Tracklikes) is in the Block.  Since one track may have been added or deleted while another's width was changed, I have to run 'Seq.indexed_pairs' here with the Blocks' Tracklikes to pair up the the same Tracklikes before comparing their widths.  'i' will be the TrackNum index for the tracks pre insertion/deletion, which is correct since the view is diffed and its Updates run before the Block updates.  This also means it actually matters that updates are run in order.  This is a lot of subtlety just to detect width changes!

I don't often have 9 line comments to explain code, and when you see two in one file you know something has gone seriously wrong.

Of course these functions weren't originally there, but were written after seeing some odd behaviour, debugging until I barely understood the situation, and how to hack in workarounds so the odd behaviour would go away.

Now here's another one: loading a new block has messed up track widths.  It turns out the problem is that creating a new view has to default the track widths properly, from the View track widths if they are set, from the Block otherwise.

So, I'm getting rid of per-view track width.

Now, laid out like this, it's pretty obvious that either I'm organizing this data totally wrongly, or per-view track width is simply not worth it.  It's a stupid little almost useless feature with a stupid amount of code and debugging to support it.  But at the time, it was always one more little hack to support this little feature that seemed logical, and like it gave a more flexible UI.  If my code couldn't support the obvious behaviour, then my code was broken, not my expectations for the behaviour.  And a lot of the complexity was that it introduced a whole new possibility of per-view track data.  I already had per-block track data (mute, collapse, etc.) and per-view block data (selection, scroll, zoom, etc.), so per-view track data seemed logical enough.  The problem was that it never wound up being used for anything other than the stupid feature and it messed up the nice one dimensional containment hierarchy of track -> block -> view.  How much time did it take to implement and debug that?  I don't know, but it sure was gratifying to remove it.

Also, I have sort of a philosophy that I want to be minimal in the sense of having few types, but maximal in the flexibility of how they can be combined.  So I have only Views, Blocks, Tracks, and Rulers.  But any number of Views can be open on one Block, the same Track can appear in any number of Blocks, there's no restriction of the placement or ordering of Rulers and Tracks (and Dividers), etc.  The c++ level doesn't understand any of this, it's totally flat, so haskell has to explicitly propagate a block change to all its views.

"Maximal" flexibility is a dangerous concept.

Go for "reasonably" flexible instead.


Post scriptum: after all the dust settled, the change got rid of about 80 lines.  And about 20 of those were comments.  But the comments were only huge because they were documenting around 20 lines of really icky stuff.  So it's a win!

cmd hiccup and data dependency

Editing a control that invalidates the entire piece, say affects the amount of randomness in the start times of every note, winds up locking up the app for almost 2 seconds.  That shouldn't happen.  The performance should be forced in the background, and it should wait a second or so before starting anyway so that subsequent keystrokes can kill the rederive before it starts.  So I did the only thing I could think of and put in prints with timing info to find where the hiccup is.  This is in the IO loop, so that's possible.  Then I do the binary search thing, edit, compile, check, etc.  I'm compiling because seems like a hassle to set up a test right now that reproduces the entire cycle but maybe I should have.

Turns out I'm checking for damage to in the resulting performance, and that field is marked strict along with the others, so even though the edit damage is accumulated independently of the performance results in that data structure, the strictness annotations force it all.  I think?  In any case, removing the !s causes the delay to disappear from that spot and reappear later on.  Now it's the *next* msg that hangs, in this case the key up from the key that invalidated the score.  That doesn't make any sense, a key up should be cheap.

More prints and binary search.  And now the trail leads into pure code.  The lag shows up on forcing the result of the second cmd.  So clearly cmds are doing something to make the eventual status depend on something inside the state which in turn is depending on some non-lazy part of the performance.  How am I supposed to know what any of those things in the chain are?  Forcing a particular value can result in an unknown amount of computation, and you can't tell from looking at the value what it is and who created it.

It's at times like these that I'm ambivalent about the value of laziness.

You also can't constrain the dependency links statically, they arise implicitly through data dependence.  Actually there are ways, but I understand them only poorly.  For example, if you accumulate a list by passing from function to function but only appending, even getting the head will force the entire computation because there's no static proof that you didn't prepend an element somewhere.  But if you accumulate the list in a such a way that it's impossible to prepend an element, say by not passing the head to subsequent functions, or using a Writer where the mappend is an append, the dependency is broken.  By encoding the accumulation pattern into the structure of the computation you not only get better laziness but also a clearer expresion of how a value will be accumulated.  I did the same thing when instead of collecting results by passing the accumulating value around in a StateT, I put the value into Monoid, passed mempty to subcomputations, and merged their results back into the StateT.  I feel like this was an important insight, but it came late, and I've never seen it discussed specifically.

Monads are another way to statically constrain data dependency.  I feel like there should be a more organized way to think about this, or at least a set of guidelines and "design patterns" based around controlling data dependency.  It's important, right?  Especially in functional programming...  in imperative programming every single line has an explicit dependence on the previous line which does make things clear but solves the problem only in the sense that amputation simplifies health care.

Since reduction of data dependency not only allows laziness but also allows parallelism, there must be a fair weight of research bearing down on the problem.  It's hopeful for the future of laziness I suppose.  Perhaps future generations will have more principled controlled ways to use it.

Back to the cmd hiccup, I'm now thinking that if the data dependency is so delicate and hard to find, what I really need a stronger wall so that cmd code cannot, just by looking at something, depend on score derivation, with the necessary exception of playing.  Relying on laziness in a "large scale" way is too error-prone without some constraints.  I suppose this is analogous to the problem of lazy IO and its solution via the constraints monads can provide.

But it's not just playing... there are a fair number of ways cmds want to use the deriver output.  Maybe it's not so bad as it seems.  The only thing that can incur a dependence on the performance is the result of the performance itself.  Hardcode lookup_performance to return Nothing and the problem is gone.  And that's the key to it all.  I've noticed this reaction to problems in pure code.  My first reaction is disillusionment and dismay, that the problem is intractable and I'll never make progress if I don't even know what steps to take.  But after that passes and I apply some higher level reasoning to the problem, it turns out to not be as intractable as I imagined.  Debugging functional code requires a greater ability to step away from the minutia and think through the problem more abstractly.

After much tracing of things that call things that call things, it becomes apparent.  A while back I added a nifty feature where instruments can bring cmds into scope.  Initially it's so drum-like instruments can bind keys to special event creation, but there are probably other uses as well.  So you have to know the current instrument in scope.  But you can't tell that just by looking at track title, since the instrument may be inherited from the environment.  In fact, a given block may have different instruments when called from different places.  So I had a bright idea and added another feature where the deriver output includes the values of the environment at each track.  Well, the environment depends on, guess what, the results of derivation.  And it's not very lazy.

So the options seem to be:

1. Abandon the idea of per-instrument cmds.  I don't think I want to do this, input flexibility is too valuable.
2. Abandon the idea of getting the instrument from the derivation, go back to parsing the track title or something.  I don't like this much because I don't want cmds to have a separate way of understanding the score than the deriver.  This is analogous to an IDE that uses an ad-hoc parser and analysis to come to possibly different conclusions than the compiler.  Of course some of that is unavoidable, e.g. incomplete expressions, but I'd like to minimize it.  So then the other way to go is reduce the expressiveness of the language so that the ad-hoc analysis is easy.  I don't think I really need arbitrary expressions being able to set the instrument.  But this generality also makes the language simpler, it's simpler that instrument is just a dynamic binding like any other.  Is there some way I can design a language which is both constrained enough to make textual analysis require minimal context, but also general and simple and not a SQL-ish mess of ad-hoc syntax and lack of abstraction?  I feel like the two goals are in conflict, and it would take someone smarter than me to resolve them simultaneously.
3. Try to avoid fetching track cmds unless I need them.  For example, if track cmds can declare they are never interested in a key up, I can skip all the work of figuring out what they are.  I don't think this works because firstly that just puts off the work until a key comes along that does apply, and secondly because the scope of interest is only going to expand as there are more cmds and more complicated ones.  In fact, track cmds already want a key up if they want to emit a MIDI NoteOff for it.  And it's complicated.  Cmd dispatch doesn't need to get any more complicated than it already is.
4. Make derivation lazy enough that looking up the istrument of a track derives very little.  Technically this should be possible and is appealing because the lazier I can make derivation the better.  Maybe I can tweak the mappend for the TrackEnviron such that it doesn't force more than necessary.  I think this is a good idea in general, but it still seems fragile.  Depend on something else, or depend on too much, and I'm back to debugging a hiccup.
5. Delay the replacement of the performance until it's been forced enough to not cause a delay.  This would keep anyone from touching the new performance until it's "safe" to do so.  I think I can do this by only replacing the last performance when the new one has had several important fields forced.  This means the evaluation thread has to actually ship the performance back to the responder thread at some point rather than replacing it synchronously and relying on laziness, but I already have a path to communicate evaluation status, so this shouldn't add undue complexity.

I think #5 is the winner.  But I should try #4 and include TrackEnviron in the laziness tests.

#5 took about 20m to implement, about a 10th of the time it took to figure out.  It means that I lose the nice behaviour where a play will immediately start forcing the derivation even if the evaluator is still waiting to see if there will be more input.  Now play will simply start the old version.  You have to wait for the evaluator's cooldown second to hear the new version.  Fortunately a bit of the UI changes color to indicate when that's happening, but it's a bit annoying.  But the big win is that the UI interaction is now gloriously lag-free.  I would need some further hackery and thread communication if I want play to cancel the cooldown and tell the evaluator to get a move on.  #5 cancelled out some of the nice effects of laziness; the implicit thread communication provided by the runtime where whoever gets to the lazy thunk first triggers its evaluation was nice.  It made play work just how I wanted with no additional work.  But like so many implicit things it's so powerful its dangerous.  How do you control it without losing the benefits?

On a whim a while back I watched "The Thin Red Line" on netflix.  The music has a series of duets, which for some reason remind me of Shaker music, with parallel fifths and fourths, harmony yielding suddenly to unison.  Then there are long melodies in a lilting French kind of style.  Some parts are stronger than others, but at its best it achieves a kind of sparse melancholy that is free from the film in the way so many things in that film are free of each other.

It's so hard to see how what I'm doing, the insufficient laziness causing hangs, incorrect redraws and fussy little bugs from typos, it's hard to see how it relates to what I really want to do.  Sometimes listening to music is not inspiring but dispiriting.

Isn't it arrogant, or at least stubborn of me to insist on doing everything my own way?  Getting stuck in toolsmithing is a classic trap.  And how do I know in advance if the tools are usable?  Ideally you are using the tools as you're developing them and I'm doing that a little, but in music it's much harder to work with incomplete tools.  You have to get an instrument into a more or less playable state before you can begin to make judgements of its musicality.  And software is such a slow way to build an instrument.

makefiles

Patch to put hsc2hs output into a parallel hierarchy in build/hsc/ instead of alongside their source files. Was it an improvement? Hard to say... some things get more complicated, but I've noticed I gradually move more and more toward separating generated output and source. That's good, right? Is this a progression that every program goes through as it gets larger, gradually accumulating more makefile cruft until it finally culminates in the Custom Build System? Is there a way to sidestep all this?

But anyway, it's an opportunity to pull out the GNU make doc and write more complicated make rules, which is always, you know, a pleasure.

linux

Oh yes, and I ported to linux the other day for fun.  "Ported" is a bit of an overstatement, since I just fixed a few minor things and it just compiled and ran.  The most amusing was #includes with the wrong case, which OS X lets me get away with.  Nice thing about cross-platform libraries.

Right, and no MIDI support because I haven't written a binding to the linux driver yet.  It's been suggested I use jack instead of directly binding to alsa, but it looks like jack wants to impose its own idea of scheduling so it might be a bit of a hassle.  That and drawing selections is really slow, but that machine draws text so slowly you can watch it repaint so there's probably something misconfigured in there.

focus

The last few days I've been involved with a rewrite of the fltk- > haskell msg format, intended to fix some annoying focus issues, where haskell's idea of the current focused window is out of sync with the GUI.  The problem is that I was using the ViewId of a Context to indicate the "scope" of the msg, and then taking that to also be the current focused window.  But it's not true, because a close msg on one window is not an indication that that window has focus now!  I had some logic to ignore those cases but there were still nagging focus errors, so I decided to apply the lesson I learned with keyboard modifiers, which is to send the complete state explicitly on every single msg, rather than sending it only when "necessary" and assuming you caught all the necessary cases.  So I added a separate "focus" field that has the current focused window in it no matter what.

It's annoying work because it's all this low level peek/poke FFI stuff.  It would have been a lot less work if I had just added the new field, but I took the opportunity to factor the C++ UiMsg struct into a real union instead of an ad-hoc mess of optional fields.  I feel like effort spent here on clarity is worth it because this is one of the few places I can actually have a runtime error and the type system won't save me.

Going back to C++ for a while, it's really kind of amazing how many opportunities there are for runtime errors.  Every time you use an index, every time you assume a pointer is non-null, every time you read something from a possibly uninitialized field (and union fields can't have default constructors!).

It was a big hassle but it looks like it's working and no fishy focus confusion yet.  And every time I do an internal refactoring like this a stumble across another set of little things that could be cleaned up, small renamings, simplifications, etc.  Hard to tell how much good that stuff does, but it's fun.

It's kind of cool that ghc's case analysis will warn about missed cases for numbers:


Ui/UiMsgC.hsc:96:32:
    Warning: Pattern match(es) are non-exhaustive
         In a case alternative:
             Patterns not matched:
                 #x with #x `notElem` [1#, 2#, 3#, 4#, 5#, 6#, 7#]


Only it warns you about the elements you *did* match instead of the ones you missed.




Work has been slow lately because of other distractions.  Playing at a wedding on the weekend, so I have to practice.  Transcribing a piece for 婉昭, and learning lilypond while I'm at it.  This is sort of two birds with one stone because it would be interesting to make a lilypond backend for karya.  Staff notation is very good at showing vertical relationships in common practice music, better than my notation.  It would be nice to have it as an auxiliary display for the cases when it's faster to listen in your head.

key bindings

In a program with no menus and no buttons, keybindings can be a problem.  Of course I side step a lot of problems by having the main interface be textual commands, and similarly to acme I can make an ad-hoc menu or button by typing text and clicking on it, but I also want to bind certain common actions to keys.  I think this is justifiable because music is more complicated than text, so there is a wider range of common actions.

Still it's embarrassing when I forget the keys for my own program and have to constantly go back to the source to remind myself.  So I wrote a little auxiliary program that extracts the global bindings and formats them nicely.  The first thing I noticed was that I'd managed to map just about every letter.  They go fast.

I'm not sure if there's a discipline to be applied for key assignment, I'm just making it up as I go along.  I think around 40 or so arbitrary words can be memorized without too much trouble, but you can go much higher if some kind of deeper structure can be imposed.

One thing I want to explore is a sort of syntax for temporarily applying time steps.  The idea is that if you can easily make a selection you can more easily apply general purpose operations.  For example, a possible sequence to align a note would be switch to meter time step to snap to a beat, perhaps temporarily switch to event edge time step by holding a key and extend the selection to the next event, and then invoke "move event" to align the event with the beat, or likewise, shift another event by the difference between the beat and this one.  Since "block end" and "block beginning" are time steps, this can remove the need for separate "select to end" or "select all" cmds.  Time step is also used for default note length, step play distance, and everything else ScoreTime-oriented, so the same techniques are applicable in many contexts.  With any luck I can develop a little vocabulary of building blocks: set time step, move or extend selection, reposition events, etc. and with familiarity play them as any instrument.  I currently only have one selection, but the possibility is open to have multiple selections, for cmds that require two time ranges, say to swap the contents.  I have never seen another app that does this though.

Of course, this is all on top of being able to map the keyboard to an organ-like layout to enter notes.  All non-modified keystrokes are captured by note input, so only the cmd keys are available in note entry mode.  The other guideline was to leave the secondary modifier unused, so it can be reserved for ad-hoc bindings created at runtime or per-score.

I initially had the ability to map any key chord combination, not just modifiers, but it was too hard to keep in sync with the state of the real keyboard, and I think the OS itself doesn't keep track of non-modifier key state, so if it swallows a key-up due to a focus change it's easy for the app to get out of sync with the real keyboard state, with annoying results.  Probably just as well since trying to do arbitrary keyboard chords is overkill, but there is still a MIDI keyboard available for binding and I haven't explored the possibilities of chording (literally) over there.  This is a bit opposite to the computer keyboard, since chords are easy but absolute positioned keys are not, unless you say all Cs are the same binding.  Perhaps a held interval can activate a certain time step: minor third for a quarter note, major third for a half note, etc.  And then the mouse chords are still unmapped.  I'm not sure that cut and paste are as useful in music as text so I don't need to follow acme's lead here, but it's a good example of well chosen primitives.

Then on top of all of this is the tension between representing transformations abstractly as symbols in the score, and applying them concretely to notes.  Anything systematic should probably be notation, rather than an action.  For example a time step of a small absolute offset from a note can be used to easily create grace notes, or I could use the same space to easily create the grace note ornament.  I could resolve some tension with a cmd to flatten certain kinds of notation back into notes for concrete manipulation, but once you go down to the low level you can't go back up easily.  In any case, I don't think I want cmds operating in RealTime, I think that should remain in the deriver's domain.

Here's the current state of the keymap:

keymap
cmd ' quit - octave -1 0, cmd 0 step rank 0+0 1, cmd 1 step rank 1+0 2, cmd 2 step rank 2+0 3, cmd 3 step rank 3+1 4, cmd 4 step rank 3+0 5, cmd 5 step rank 4+1 6, cmd 6 step rank 4+0 7, cmd 7 step rank 5+1 8, cmd 8 step_rank 6+0 9, cmd 9 step_rank 7+0 = octave +1 [, cmd [ zoom out *0.8 \, cmd \ zoom to fit ], cmd ] zoom in *1.25 _ insert track end cmd ` toggle step mode cmd a select track / all cmd b create block cmd B create block template cmd c copy selection cmd C toggle collapse cmd d delete tracks e, cmd e transpose down degree	E, cmd E transpose down octave cmd H splice track above cmd i merge selection j, cmd j insert time J, cmd J move event forward k, cmd k delete time K, cmd K move event back cmd L load M toggle mute cmd M toggle merge all cmd n create view o, cmd o join events r, cmd r redo R, cmd R resize to fit S toggle solo cmd S save cmd t insert track cmd T splice track below u, cmd u undo cmd V insert selection cmd v paste selection cmd W destroy block cmd w destroy view cmd x cut selection y, cmd y transpose up degree	Y, cmd Y transpose up octave cmd ~ invert step escape toggle val edit shift escape toggle kbd entry mode cmd escape toggle raw edit backspace, cmd backspace clear selected tab toggle method edit shift + cmd right step play tracks here cmd right step play here left shift selection left shift left extend shift selection left up rewind selection shift up extend rewind selection shift + cmd up, cmd up step play rewind right shift selection right shift right extend shift selection right down advance selection shift down extend advance selection shift + cmd down step play tracks advance cmd down step play advance ctrl click 1, alt click 1 toggle skeleton edge double-click 1 open block shift drag 1, drag 1 snap drag selection shift + cmd drag 1 extend selection cmd drag 1 drag selection