State of things

I'm finally, and slowly, beginning to work with actual music.  Here's the state of things as of a few weeks back.  Pedal is working (and a bug there just fixed), and I just added a bounced note as a sub-block, but hadn't yet added the function to rename BlockIds, so it's still called "b28" instead of "bounce".  And of course this triggered yet another bughunt when I realized that the cached bounce sub-block results weren't being invalidated by editing.

It's still primitive, awkward, and buggy, but this "bounce" thing is the first hint of my plans actually beginning to be realized.  The bounced note has a composed tempo and itself is the second note of an arpeggio, which is beginning to show how these ornaments can be composed.  That's the real test: do I reach for these tools when I want a certain musical effect.  And of course I don't know of any other sequencer that could do this sort of thing.

It turns out that the mechanisms to make things compose nicely, namely track slicing and inversion, make evaluation more complicated and lead to tricky bugs.  I wish I knew a more principled way to get the effect I want, but so it goes.  The point is not to have a pretty program, but a useful one.

Tests did their job!

Debugging a problem with a sustain pedal going down too early made me realize that I had changed the definition of the value of a signal before the first sample from zero to the first sample.  So that had the counter-intuitive effect that a pedal beginning at a certain time would actually be counted as always being down if it didn't explicitly start with zero.  So there are two solutions to that, either change the pedal call so it puts a zero sample at time 0 if there isn't already a previous sample, or change the various sample functions so a signal is considered to be 0 before the first sample.  The first way is clearly ad-hoc and gross and just delaying the problem in some other call when I've forgotten about this behaviour, so it's time to change the signal definition yet again.

I seemed to recall I had added that "take the first value" behaviour at some point for some specific reason, and some fiddling with darcs annotate I found a couple of patches from more than a year ago that fixed bugs when I forgot to update everything, but couldn't find the patch that originally introduced it.  Stupid.  I should always document choices like that.  Turns out the documentation for the Signal module was several design changes out of date, and still documented the previous zero behaviour.

While I didn't find the rationale, checking the patches meant I had the set of functions that would have to be changed in mind, so I was more confident I wouldn't miss something.  So I made the change and ran the tests.

This sort of subtle interpretation change is pretty scary because it yields no type errors, and no crashes, but just that the derivation will be incorrect in some subtle way, such as the original sustain pedal problem.  But fortunately, every time I track down some subtle derivation bug I added a test for it, and when I ran the tests, sure enough the pitch signal sharing function failed its test.

Two notes overlapping but with different pitches have different pitch curves, as expected.  If those pitch curves are parallel, they can share a channel.  However, if a pitch signal is considered to start with the note, and signals are zero before the first sample, it looks like the second note jumps from pitch 0 to its real pitch, which is not parallel!

But why does the overlap check even look at the pitch curve before the beginning of the second note?  It turns out there's a bit of time inserted called the "control lead time" so controls can be set a little in advance of the note start.  MIDI has no concept of simultaneity and a pitch bend sent at the same time as a note on will cause an audible artifact.  There are tests explicitly verifying that notes that are overlapping after taking the control lead time into account still can't share channels so it was definitely intentional, and it seems to still makes sense, but I think adding the lead time to the end of the decay of the previous note rather than subtracting from the start of the current note would achieve the same effect without having to look at the pitch signal before the note begins.

So, give it a lot of thought, make the two line change of moving an addition from one expression to another, write a comment explaining why, rerun the tests.  This is exactly the kind of subtle issue that without the presence of the tests there is absolutely no way I would have foreseen.


And I have to say, a 13 line function with 22 lines of comments, 7 arguments, and a long history of minor tweaks is definitely a sore point.  I've thought a lot at various points about how to make it easier to explain, clearer, less error-prone, but failed every time.

As an aside, I have a new method of debugging derivation issues lately.  I wrote a function to dump the score from the UI in the simplified haskell literal format that the test functions use to construct a test score.  Then copy and paste that into a test, and see if running it there reproduces the problem.  I write a little predicate to characterize the problem (e.g. this particular note is on the wrong channel) or just eyeball it, and start cutting out score to see what happens to the result.  It seems to work ok but is still more clumsy than I want.  Hopefully I can think of something better later.

Since often the problem is that something *sounds* wrong, and has to be turned into a series of messages that *look* wrong, I also wrote a synthesizer state emulator.  Given a list of messages and their times, it can signal things like stuck notes, who is overlapping who, who has pitch bends where, etc.  I haven't actually found this one very useful yet, but even if I don't use it much for testing it should eventually become part of the process to turn recorded MIDI back into high level score.


And so, the final practical result of all that debugging is that I can play the piece back and the pedal is finally in the right place.  But music being what it is, I've gotten rather used to the pedal going down earlier.  Is it better there or did I just get used to it?  Who knows...

track width

Sometimes it's the tiny details that get ridiculously complicated.

Sometimes such a feature will just seem like it's only logical that it be there, and it shouldn't be that complicated, and you add hack after hack to make it work, without re-evaluating whether it's really so simple after all and whether it's worth all the hacks.

Such a feature is the ability of the same track in different views to have different widths.  It makes sense, right?  Seems so simple.

Not really.  It means the track width has to go in the view, not the block.  A view is one window on a block.  The block has tracks, but if views are to have different widths for the same track, the view also needs a parallel list of track widths for per-view track data.  There's various bits of code to try to maintain that invariant, and behave nicely if somehow it was broken and they are different lengths.  A separate verification step checks that, among other state invariants, but now it also needs to be able to modify the state in case it wanted to fix up the broken invariant.

And then what width do tracks have in a new view?  So Blocks now also have track widths, for new view defaults.  Ok, a little confusing, but not too bad.

Now there's this other little thing... there are actually two types of Track, a Block.Track and a Block.DisplayTrack.  A normal Track supports high level notions like Collapse and Mute and whatnot, but the UI level knows nothing of these things.  A Track with Collapse set turns into a track remove and then insert of a kind of tiny degenerate track called a Divider.  So if you want the real visible width, you have to look at the DisplayTrack.  In the Block, not the View.  And then when it gets expanded again it needs its old width...  from the View!  So the function to convert Block.Track -> Block.DisplayTrack... whoops, needs an extra View parameter.  But it has to return that width separately, not in the DisplayTrack... because DisplayTrack is Block-level, not View-level, remember?

DisplayTrack updates are used to tell the UI to change block-level track settings, but width can't go there of course, it needs to have its own view-level track settings update.

Here's another thing, explained from the comment on the function that does the hackery:

This is a nasty little case that falls out of how I'm doing diffs: First the view diff runs, which detects changed track widths.  Then the block diff runs, which detects changed tracks.  Replaced tracks (remove old track, insert new one with default width, which as a new track should get the default width in all views) are not distinguishable from a merely altered track (which can look like remove old track, insert new one with the same width, and should keep its width in each view).  So what I do is assume that if there's an InsertTrack and a corresponding TrackView in view_tracks of the new state, it should get the width given in the view.

But wait!  There's more!  If it's a collapsed track then the track_view_width is not the visible width, so don't emit TrackWidth updates in that case.

Later on we have this other fun comment:

The track view info (widths) is in the View, while the track data itself (Tracklikes) is in the Block.  Since one track may have been added or deleted while another's width was changed, I have to run 'Seq.indexed_pairs' here with the Blocks' Tracklikes to pair up the the same Tracklikes before comparing their widths.  'i' will be the TrackNum index for the tracks pre insertion/deletion, which is correct since the view is diffed and its Updates run before the Block updates.  This also means it actually matters that updates are run in order.  This is a lot of subtlety just to detect width changes!

I don't often have 9 line comments to explain code, and when you see two in one file you know something has gone seriously wrong.

Of course these functions weren't originally there, but were written after seeing some odd behaviour, debugging until I barely understood the situation, and how to hack in workarounds so the odd behaviour would go away.

Now here's another one: loading a new block has messed up track widths.  It turns out the problem is that creating a new view has to default the track widths properly, from the View track widths if they are set, from the Block otherwise.

So, I'm getting rid of per-view track width.

Now, laid out like this, it's pretty obvious that either I'm organizing this data totally wrongly, or per-view track width is simply not worth it.  It's a stupid little almost useless feature with a stupid amount of code and debugging to support it.  But at the time, it was always one more little hack to support this little feature that seemed logical, and like it gave a more flexible UI.  If my code couldn't support the obvious behaviour, then my code was broken, not my expectations for the behaviour.  And a lot of the complexity was that it introduced a whole new possibility of per-view track data.  I already had per-block track data (mute, collapse, etc.) and per-view block data (selection, scroll, zoom, etc.), so per-view track data seemed logical enough.  The problem was that it never wound up being used for anything other than the stupid feature and it messed up the nice one dimensional containment hierarchy of track -> block -> view.  How much time did it take to implement and debug that?  I don't know, but it sure was gratifying to remove it.

Also, I have sort of a philosophy that I want to be minimal in the sense of having few types, but maximal in the flexibility of how they can be combined.  So I have only Views, Blocks, Tracks, and Rulers.  But any number of Views can be open on one Block, the same Track can appear in any number of Blocks, there's no restriction of the placement or ordering of Rulers and Tracks (and Dividers), etc.  The c++ level doesn't understand any of this, it's totally flat, so haskell has to explicitly propagate a block change to all its views.

"Maximal" flexibility is a dangerous concept.

Go for "reasonably" flexible instead.


Post scriptum: after all the dust settled, the change got rid of about 80 lines.  And about 20 of those were comments.  But the comments were only huge because they were documenting around 20 lines of really icky stuff.  So it's a win!

cmd hiccup and data dependency

Editing a control that invalidates the entire piece, say affects the amount of randomness in the start times of every note, winds up locking up the app for almost 2 seconds.  That shouldn't happen.  The performance should be forced in the background, and it should wait a second or so before starting anyway so that subsequent keystrokes can kill the rederive before it starts.  So I did the only thing I could think of and put in prints with timing info to find where the hiccup is.  This is in the IO loop, so that's possible.  Then I do the binary search thing, edit, compile, check, etc.  I'm compiling because seems like a hassle to set up a test right now that reproduces the entire cycle but maybe I should have.

Turns out I'm checking for damage to in the resulting performance, and that field is marked strict along with the others, so even though the edit damage is accumulated independently of the performance results in that data structure, the strictness annotations force it all.  I think?  In any case, removing the !s causes the delay to disappear from that spot and reappear later on.  Now it's the *next* msg that hangs, in this case the key up from the key that invalidated the score.  That doesn't make any sense, a key up should be cheap.

More prints and binary search.  And now the trail leads into pure code.  The lag shows up on forcing the result of the second cmd.  So clearly cmds are doing something to make the eventual status depend on something inside the state which in turn is depending on some non-lazy part of the performance.  How am I supposed to know what any of those things in the chain are?  Forcing a particular value can result in an unknown amount of computation, and you can't tell from looking at the value what it is and who created it.

It's at times like these that I'm ambivalent about the value of laziness.

You also can't constrain the dependency links statically, they arise implicitly through data dependence.  Actually there are ways, but I understand them only poorly.  For example, if you accumulate a list by passing from function to function but only appending, even getting the head will force the entire computation because there's no static proof that you didn't prepend an element somewhere.  But if you accumulate the list in a such a way that it's impossible to prepend an element, say by not passing the head to subsequent functions, or using a Writer where the mappend is an append, the dependency is broken.  By encoding the accumulation pattern into the structure of the computation you not only get better laziness but also a clearer expresion of how a value will be accumulated.  I did the same thing when instead of collecting results by passing the accumulating value around in a StateT, I put the value into Monoid, passed mempty to subcomputations, and merged their results back into the StateT.  I feel like this was an important insight, but it came late, and I've never seen it discussed specifically.

Monads are another way to statically constrain data dependency.  I feel like there should be a more organized way to think about this, or at least a set of guidelines and "design patterns" based around controlling data dependency.  It's important, right?  Especially in functional programming...  in imperative programming every single line has an explicit dependence on the previous line which does make things clear but solves the problem only in the sense that amputation simplifies health care.

Since reduction of data dependency not only allows laziness but also allows parallelism, there must be a fair weight of research bearing down on the problem.  It's hopeful for the future of laziness I suppose.  Perhaps future generations will have more principled controlled ways to use it.

Back to the cmd hiccup, I'm now thinking that if the data dependency is so delicate and hard to find, what I really need a stronger wall so that cmd code cannot, just by looking at something, depend on score derivation, with the necessary exception of playing.  Relying on laziness in a "large scale" way is too error-prone without some constraints.  I suppose this is analogous to the problem of lazy IO and its solution via the constraints monads can provide.

But it's not just playing... there are a fair number of ways cmds want to use the deriver output.  Maybe it's not so bad as it seems.  The only thing that can incur a dependence on the performance is the result of the performance itself.  Hardcode lookup_performance to return Nothing and the problem is gone.  And that's the key to it all.  I've noticed this reaction to problems in pure code.  My first reaction is disillusionment and dismay, that the problem is intractable and I'll never make progress if I don't even know what steps to take.  But after that passes and I apply some higher level reasoning to the problem, it turns out to not be as intractable as I imagined.  Debugging functional code requires a greater ability to step away from the minutia and think through the problem more abstractly.

After much tracing of things that call things that call things, it becomes apparent.  A while back I added a nifty feature where instruments can bring cmds into scope.  Initially it's so drum-like instruments can bind keys to special event creation, but there are probably other uses as well.  So you have to know the current instrument in scope.  But you can't tell that just by looking at track title, since the instrument may be inherited from the environment.  In fact, a given block may have different instruments when called from different places.  So I had a bright idea and added another feature where the deriver output includes the values of the environment at each track.  Well, the environment depends on, guess what, the results of derivation.  And it's not very lazy.

So the options seem to be:

1. Abandon the idea of per-instrument cmds.  I don't think I want to do this, input flexibility is too valuable.
2. Abandon the idea of getting the instrument from the derivation, go back to parsing the track title or something.  I don't like this much because I don't want cmds to have a separate way of understanding the score than the deriver.  This is analogous to an IDE that uses an ad-hoc parser and analysis to come to possibly different conclusions than the compiler.  Of course some of that is unavoidable, e.g. incomplete expressions, but I'd like to minimize it.  So then the other way to go is reduce the expressiveness of the language so that the ad-hoc analysis is easy.  I don't think I really need arbitrary expressions being able to set the instrument.  But this generality also makes the language simpler, it's simpler that instrument is just a dynamic binding like any other.  Is there some way I can design a language which is both constrained enough to make textual analysis require minimal context, but also general and simple and not a SQL-ish mess of ad-hoc syntax and lack of abstraction?  I feel like the two goals are in conflict, and it would take someone smarter than me to resolve them simultaneously.
3. Try to avoid fetching track cmds unless I need them.  For example, if track cmds can declare they are never interested in a key up, I can skip all the work of figuring out what they are.  I don't think this works because firstly that just puts off the work until a key comes along that does apply, and secondly because the scope of interest is only going to expand as there are more cmds and more complicated ones.  In fact, track cmds already want a key up if they want to emit a MIDI NoteOff for it.  And it's complicated.  Cmd dispatch doesn't need to get any more complicated than it already is.
4. Make derivation lazy enough that looking up the istrument of a track derives very little.  Technically this should be possible and is appealing because the lazier I can make derivation the better.  Maybe I can tweak the mappend for the TrackEnviron such that it doesn't force more than necessary.  I think this is a good idea in general, but it still seems fragile.  Depend on something else, or depend on too much, and I'm back to debugging a hiccup.
5. Delay the replacement of the performance until it's been forced enough to not cause a delay.  This would keep anyone from touching the new performance until it's "safe" to do so.  I think I can do this by only replacing the last performance when the new one has had several important fields forced.  This means the evaluation thread has to actually ship the performance back to the responder thread at some point rather than replacing it synchronously and relying on laziness, but I already have a path to communicate evaluation status, so this shouldn't add undue complexity.

I think #5 is the winner.  But I should try #4 and include TrackEnviron in the laziness tests.

#5 took about 20m to implement, about a 10th of the time it took to figure out.  It means that I lose the nice behaviour where a play will immediately start forcing the derivation even if the evaluator is still waiting to see if there will be more input.  Now play will simply start the old version.  You have to wait for the evaluator's cooldown second to hear the new version.  Fortunately a bit of the UI changes color to indicate when that's happening, but it's a bit annoying.  But the big win is that the UI interaction is now gloriously lag-free.  I would need some further hackery and thread communication if I want play to cancel the cooldown and tell the evaluator to get a move on.  #5 cancelled out some of the nice effects of laziness; the implicit thread communication provided by the runtime where whoever gets to the lazy thunk first triggers its evaluation was nice.  It made play work just how I wanted with no additional work.  But like so many implicit things it's so powerful its dangerous.  How do you control it without losing the benefits?

On a whim a while back I watched "The Thin Red Line" on netflix.  The music has a series of duets, which for some reason remind me of Shaker music, with parallel fifths and fourths, harmony yielding suddenly to unison.  Then there are long melodies in a lilting French kind of style.  Some parts are stronger than others, but at its best it achieves a kind of sparse melancholy that is free from the film in the way so many things in that film are free of each other.

It's so hard to see how what I'm doing, the insufficient laziness causing hangs, incorrect redraws and fussy little bugs from typos, it's hard to see how it relates to what I really want to do.  Sometimes listening to music is not inspiring but dispiriting.

Isn't it arrogant, or at least stubborn of me to insist on doing everything my own way?  Getting stuck in toolsmithing is a classic trap.  And how do I know in advance if the tools are usable?  Ideally you are using the tools as you're developing them and I'm doing that a little, but in music it's much harder to work with incomplete tools.  You have to get an instrument into a more or less playable state before you can begin to make judgements of its musicality.  And software is such a slow way to build an instrument.

makefiles

Patch to put hsc2hs output into a parallel hierarchy in build/hsc/ instead of alongside their source files. Was it an improvement? Hard to say... some things get more complicated, but I've noticed I gradually move more and more toward separating generated output and source. That's good, right? Is this a progression that every program goes through as it gets larger, gradually accumulating more makefile cruft until it finally culminates in the Custom Build System? Is there a way to sidestep all this?

But anyway, it's an opportunity to pull out the GNU make doc and write more complicated make rules, which is always, you know, a pleasure.

linux

Oh yes, and I ported to linux the other day for fun.  "Ported" is a bit of an overstatement, since I just fixed a few minor things and it just compiled and ran.  The most amusing was #includes with the wrong case, which OS X lets me get away with.  Nice thing about cross-platform libraries.

Right, and no MIDI support because I haven't written a binding to the linux driver yet.  It's been suggested I use jack instead of directly binding to alsa, but it looks like jack wants to impose its own idea of scheduling so it might be a bit of a hassle.  That and drawing selections is really slow, but that machine draws text so slowly you can watch it repaint so there's probably something misconfigured in there.

focus

The last few days I've been involved with a rewrite of the fltk- > haskell msg format, intended to fix some annoying focus issues, where haskell's idea of the current focused window is out of sync with the GUI.  The problem is that I was using the ViewId of a Context to indicate the "scope" of the msg, and then taking that to also be the current focused window.  But it's not true, because a close msg on one window is not an indication that that window has focus now!  I had some logic to ignore those cases but there were still nagging focus errors, so I decided to apply the lesson I learned with keyboard modifiers, which is to send the complete state explicitly on every single msg, rather than sending it only when "necessary" and assuming you caught all the necessary cases.  So I added a separate "focus" field that has the current focused window in it no matter what.

It's annoying work because it's all this low level peek/poke FFI stuff.  It would have been a lot less work if I had just added the new field, but I took the opportunity to factor the C++ UiMsg struct into a real union instead of an ad-hoc mess of optional fields.  I feel like effort spent here on clarity is worth it because this is one of the few places I can actually have a runtime error and the type system won't save me.

Going back to C++ for a while, it's really kind of amazing how many opportunities there are for runtime errors.  Every time you use an index, every time you assume a pointer is non-null, every time you read something from a possibly uninitialized field (and union fields can't have default constructors!).

It was a big hassle but it looks like it's working and no fishy focus confusion yet.  And every time I do an internal refactoring like this a stumble across another set of little things that could be cleaned up, small renamings, simplifications, etc.  Hard to tell how much good that stuff does, but it's fun.

It's kind of cool that ghc's case analysis will warn about missed cases for numbers:


Ui/UiMsgC.hsc:96:32:
    Warning: Pattern match(es) are non-exhaustive
         In a case alternative:
             Patterns not matched:
                 #x with #x `notElem` [1#, 2#, 3#, 4#, 5#, 6#, 7#]


Only it warns you about the elements you *did* match instead of the ones you missed.




Work has been slow lately because of other distractions.  Playing at a wedding on the weekend, so I have to practice.  Transcribing a piece for 婉昭, and learning lilypond while I'm at it.  This is sort of two birds with one stone because it would be interesting to make a lilypond backend for karya.  Staff notation is very good at showing vertical relationships in common practice music, better than my notation.  It would be nice to have it as an auxiliary display for the cases when it's faster to listen in your head.

key bindings

In a program with no menus and no buttons, keybindings can be a problem.  Of course I side step a lot of problems by having the main interface be textual commands, and similarly to acme I can make an ad-hoc menu or button by typing text and clicking on it, but I also want to bind certain common actions to keys.  I think this is justifiable because music is more complicated than text, so there is a wider range of common actions.

Still it's embarrassing when I forget the keys for my own program and have to constantly go back to the source to remind myself.  So I wrote a little auxiliary program that extracts the global bindings and formats them nicely.  The first thing I noticed was that I'd managed to map just about every letter.  They go fast.

I'm not sure if there's a discipline to be applied for key assignment, I'm just making it up as I go along.  I think around 40 or so arbitrary words can be memorized without too much trouble, but you can go much higher if some kind of deeper structure can be imposed.

One thing I want to explore is a sort of syntax for temporarily applying time steps.  The idea is that if you can easily make a selection you can more easily apply general purpose operations.  For example, a possible sequence to align a note would be switch to meter time step to snap to a beat, perhaps temporarily switch to event edge time step by holding a key and extend the selection to the next event, and then invoke "move event" to align the event with the beat, or likewise, shift another event by the difference between the beat and this one.  Since "block end" and "block beginning" are time steps, this can remove the need for separate "select to end" or "select all" cmds.  Time step is also used for default note length, step play distance, and everything else ScoreTime-oriented, so the same techniques are applicable in many contexts.  With any luck I can develop a little vocabulary of building blocks: set time step, move or extend selection, reposition events, etc. and with familiarity play them as any instrument.  I currently only have one selection, but the possibility is open to have multiple selections, for cmds that require two time ranges, say to swap the contents.  I have never seen another app that does this though.

Of course, this is all on top of being able to map the keyboard to an organ-like layout to enter notes.  All non-modified keystrokes are captured by note input, so only the cmd keys are available in note entry mode.  The other guideline was to leave the secondary modifier unused, so it can be reserved for ad-hoc bindings created at runtime or per-score.

I initially had the ability to map any key chord combination, not just modifiers, but it was too hard to keep in sync with the state of the real keyboard, and I think the OS itself doesn't keep track of non-modifier key state, so if it swallows a key-up due to a focus change it's easy for the app to get out of sync with the real keyboard state, with annoying results.  Probably just as well since trying to do arbitrary keyboard chords is overkill, but there is still a MIDI keyboard available for binding and I haven't explored the possibilities of chording (literally) over there.  This is a bit opposite to the computer keyboard, since chords are easy but absolute positioned keys are not, unless you say all Cs are the same binding.  Perhaps a held interval can activate a certain time step: minor third for a quarter note, major third for a half note, etc.  And then the mouse chords are still unmapped.  I'm not sure that cut and paste are as useful in music as text so I don't need to follow acme's lead here, but it's a good example of well chosen primitives.

Then on top of all of this is the tension between representing transformations abstractly as symbols in the score, and applying them concretely to notes.  Anything systematic should probably be notation, rather than an action.  For example a time step of a small absolute offset from a note can be used to easily create grace notes, or I could use the same space to easily create the grace note ornament.  I could resolve some tension with a cmd to flatten certain kinds of notation back into notes for concrete manipulation, but once you go down to the low level you can't go back up easily.  In any case, I don't think I want cmds operating in RealTime, I think that should remain in the deriver's domain.

Here's the current state of the keymap:

keymap
cmd ' quit - octave -1 0, cmd 0 step rank 0+0 1, cmd 1 step rank 1+0 2, cmd 2 step rank 2+0 3, cmd 3 step rank 3+1 4, cmd 4 step rank 3+0 5, cmd 5 step rank 4+1 6, cmd 6 step rank 4+0 7, cmd 7 step rank 5+1 8, cmd 8 step_rank 6+0 9, cmd 9 step_rank 7+0 = octave +1 [, cmd [ zoom out *0.8 \, cmd \ zoom to fit ], cmd ] zoom in *1.25 _ insert track end cmd ` toggle step mode cmd a select track / all cmd b create block cmd B create block template cmd c copy selection cmd C toggle collapse cmd d delete tracks e, cmd e transpose down degree	E, cmd E transpose down octave cmd H splice track above cmd i merge selection j, cmd j insert time J, cmd J move event forward k, cmd k delete time K, cmd K move event back cmd L load M toggle mute cmd M toggle merge all cmd n create view o, cmd o join events r, cmd r redo R, cmd R resize to fit S toggle solo cmd S save cmd t insert track cmd T splice track below u, cmd u undo cmd V insert selection cmd v paste selection cmd W destroy block cmd w destroy view cmd x cut selection y, cmd y transpose up degree	Y, cmd Y transpose up octave cmd ~ invert step escape toggle val edit shift escape toggle kbd entry mode cmd escape toggle raw edit backspace, cmd backspace clear selected tab toggle method edit shift + cmd right step play tracks here cmd right step play here left shift selection left shift left extend shift selection left up rewind selection shift up extend rewind selection shift + cmd up, cmd up step play rewind right shift selection right shift right extend shift selection right down advance selection shift down extend advance selection shift + cmd down step play tracks advance cmd down step play advance ctrl click 1, alt click 1 toggle skeleton edge double-click 1 open block shift drag 1, drag 1 snap drag selection shift + cmd drag 1 extend selection cmd drag 1 drag selection

fixed lists

I had an unsafe function:


map_signals :: [TrackLang.Control] -> [TrackLang.PitchControl]
    -> ([Signal.Y] -> [Pitch.Degree] -> state -> Score.Event

        -> Derive.Deriver ([Score.Event], state))
    -> state -> Derive.Events -> Derive.Deriver ([Derive.Events], state)

It's unsafe because it looks up each Control and each PitchControl and passes the values back to the given function, and promises the length of [Signal.Y] will be the same as [TrackLang.Control] and likewise for Degree and PitchControl.  But it's just a promise, not a contract, so ghc started complaining about incomplete pattern matches when called like map_signals [a, b, c] [d, e] $ \[a, b, c] [d, e] -> ....  So the fixed-list package fixes that:



map_signals :: (FixedList.FixedList cs, FixedList.FixedList ps) =>
    cs TrackLang.Control -> ps TrackLang.PitchControl
    -> (cs Signal.Y -> ps Pitch.Degree -> state -> Score.Event
        -> Derive.Deriver ([Score.Event], state))
    -> state -> Derive.Events -> Derive.Deriver ([Derive.Events], state)

Now 'ps' and 'cs' are specified to have the same lengths, and no more warnings.  They get called almost the same:


map_signals (a :. b :. c :. Nil) (d :. e :. Nil) $
    \(a :. b :. c :. Nil) (d :. e :. Nil) -> ...


I wasn't too worried about type errors since the lists and patterns are adjacent, but it's a neat trick anyway.

step play complete

Finally after spending too much time on it, it's done.  I wound up implementing both forwards and backwards stepping, and simplified things in the process.  Previously I had a stream of MIDI messages and would step along them by finding the step position, calculating the next step, finding the resultant RealTime, and playing that many MIDI messages.

I changed it to produce all the step points along with the MIDI state at each point, right off the bat, and store that.  Then stepping is just moving forward one state, diffing it with the previous one to generate a set of MIDI messages, and playing those.  Now backwards is as easy as forwards.  And the StepState is much simplified, and the stepping functions don't have to know anything about the steps or about RealTime, or anything like that.

In a strict language, this approach wouldn't work because it would immediately want to realize the entire score just to play a single note.  Sometimes I'm on the fence about whether laziness is worth it in the long run, but this is another point for laziness.

I'm still not sure whether backwards scrubbing is really worth all that craziness, but it's kind of fun to mess around with.

rats are ideal

When rats wash a lot there's a sort of a sweetish wet fur smell.  I like it, and it brings up all kinds of associations with late night hacking, or gaming, or whatever else, since they hang out on my lap all night grooming themselves.

This is why rats are pretty much the ideal hacking pet.  A dog smells bad and usually doesn't fit on your lap.  A cat may or may not deign to sit on your lap, and may or may not decide to claw you if it's feeling particularly content, but will certainly be shedding hair into your face and decide the keyboard is a better place to relax anyway.

Rats hang out and do their thing while you hang out and do your thing.

context sensitive bugs

I spent a little while trying to track down a text entry bug that happened when I was distracted IMing with  惠茹.  But when I came back to fix it, I couldn't get it to happen again.  Figures that it vanishes as soon as I can pay attention.

Well, what if there actually is a connection?  And then I had it: the bug only shows up when the Chinese IME is activated.

What haskell doesn't have

(but it does have fun!)


At one point, someone on buzz was advocating haskell, and he pulled out the usual "look how elegant the fibonacci function looks!" thing.  Someone else was doing the "the point must be productivity, and you can't prove it adds more productivity" thing.  I think the point is fun, not productivity.  I wrote a big long response, but then felt like getting involved in a big language discussion on work time was kind of a waste of time, so didn't bother posting it.  And it was mostly going to be a self-indulgent preaching to the choir thing anyway.  But then I thought, well, as long as I've written it I might as well put it somewhere.  And I don't believe I've noticed anyone saying it in exactly this way before.  Anyway, where it is:


I know people like to pull out fibonacci and quicksort, but they don't seem all that compelling to me.

Instead:

Think about the whole thing with iterators or generators or whatever else that so many imperative languages have.  That's all gone.

Select vs. threads vs. coroutines is also gone, but that's just a ghc feature.  All that flap about asynchronous events and event based programming?  Gone.

Think about that whole null pointer thing that all these imperative languages have.  Null pointer exceptions?  Gone!

Think about that whole thing with reference vs. values.  That's gone.  Or think about the thing with identity vs. equality vs. deep equality, that's gone too.

All that stuff about copy constructors or equals methods or all the 'this.x = x' constructor garbage, all that boring boilerplate is gone.  No more writing toString()s or __str__()s by hand, you can get a useful one for free.

Or how about all that stuff about "const" or "final" or "const correctness" and separate const and non-const methods, that all just goes away and good riddance.

Type casts are gone too.

All that hairy generics stuff is vastly simplified without subtyping.  Contravariance vs. covariance, sub/super constraints on generics, multiple vs.  single inheritance, the whole is-a vs. has-a thing is gone.  Yes it means there's no dynamic dispatch, but I don't miss it much.  If you want to parameterize on an operation, just pass the operation as a parameter, instead of passing an object that dynamically dispatches on one of several operations.

Think about all the mandatory type declarations those not-python languages have.  That's gone.  Or maybe think about how slow and single-threaded and hard to deploy and runtime-error-loving python is, that's gone too.

That whole thing about control structures being built-in syntax and different from functions?  That's gone.  It's hard to quantify the difference it makes because you stop thinking about the difference between them and just structure your code as feels most natural.  Like the whole order of definition thing (that's gone too), it's just one more restriction removed.  The whole statement vs. expression thing is gone too.

That lengthy edit, compile, wait, debug cycle is mostly gone too.  It usually takes less than a second to reload a module and start running the code you wrote one second ago.


And of course, this is just a list of the things that are gone.  There's an equally large list of new things that are there, but it might be hard to see how those are useful from the outside.

I have a project with more than 200 source files, each one defines from 5 to 20-ish data types.  If it were java or c++, with their heavyweight class declarations, each one would be a directory with 5 to 20 files in it, about one per data type.  No one is going to go to all that work, so the result is that you create types for the big abstractions and pass the rest around as ints and doubles and strings, or Maps and Lists.  The result is undescriptive signatures, search and replace expeditions when a type changes, reusing "almost right" types that results in extra "impossible" error cases to deal with, mixed up values, and of course runtime type errors.

Similarly, when defining a new function requires minimum 3 lines of boilerplate and some redundant type declarations that need to be updated whenever your types change and maybe defined half a screen away, you create a lot fewer functions, and factor less.  If it's too annoying to do, no one will do it.  And if they do, they will do it because they feel obligated to do the right thing, even if it's tedious.  Not because they're having a good time.

To me, the main compelling thing is *fun*.  All of that stuff which is gone is mostly boring bookkeeping crap.  I never get excited thinking about putting in new 'const' declarations or equals() or toString() methods.  I don't look forward to all the TypeErrors and NameErrors and ValueErrors I might face today, or waiting for the application to relink.  Programming is more fun without that stuff.

step play

So here's what I put in my TODO file that finally got me to start the blog.  This is what I've been working on for the last week or so:

Adding a simple feature: play back score step by step instead of in realtime.  It's useful to hear how each note gets added to the mix and listen for as long as I like, so I want a feature where I can press a key to advance from note to note.

Simple, right?  Well, first I need to step note by note, so that means adding "event edge" to the time step system.  No wait, it's actually note beginnings, and that triggers a rewrite of time steps to support merged steps and multiple steps (e.g. since I want to start a little while before the insertion point, and "start 4 notes back" seems like a useful thing to be able to say).  Wait, and then it needs an additional mode to stop as soon as can no longer step instead of aborting, because if you say "start from 4 notes back" it would be better to start from 3 notes back if the fourth doesn't exist.

Ok, then I need an efficient way to seek around in the note stream.  So add a simple kind of zipper ([reverse a], [a]) and some tricky functions to zip forward and backward.  All of this prompts some additions and then reorganization of the State record.  I've found that it's nice to divide state up along its access pattern: a constant bit, a monoidal "collecting" bit, and a true stateful bit that must be threaded since previous computations may rely on the modifications of previous ones.  It makes it easier to understand what should be reverted and what should be retained after a computation aborts, and clear what the troublesome bits are likely to be (it's the true stateful part).

Now, how to display the current position of the step play?  Well, another selection like the realtime playback selection would be logical, but then I run into the problem that a user selection is a single contiguous block, but playback can be at different times simultaneously due to multiple tempi.  The realtime playback gets around that by going directly to the GUI instead of being stored in the song state... which makes sense since you don't want to save the realtime playback position with the piece.  So now I need a separate non-contiguous selection concept, or a way to extend the realtime playback hack to work with step time playback.  This involves somewhat deep changes so I punt for the moment on correctly displaying multiple tempi.

Now the actual playing of the notes... forward works fine, but what about backward?  MIDI is extremely stateful so you can't just play it backward.  So, record the state at each point on the way forward, and then when going backward emit the messages that take the synthesizer state from the current one to the previous one.  Fortunately I already have a "synthesizer simulator" used for tests that I can repurpose for this.  I punt again and only play forward.  I'll do backward later.

Ok, and then for the tests... the existing test framework is based around running a single command and examining the results, I need to extend it to support chaining of several commands and inspecting the state in between.  After a couple of false starts here along with a general refactoring it's good enough to do what I want, and turns up a few bugs, but there are still more that only come up when testing by hand.

One of these bugs leads to another bug-fixing session in the time step code for some cases that didn't get tested.  Another bug reveals that converting from RealTime to ScoreTime to RealTime isn't identity, there's a tiny bit of error that I never noticed before because nothing else tries to do a round-trip (and it does the round trip because I need to step forward by a certain amount relative to a certain track, then find out the real time difference to play events, and then find out the subsequent score position on *all* tracks).  On further thought, it's apparent that it's because RealTime is fixed point and ScoreTime is a Double, so of course you can't convert without losing precision.  RealTime is fixed point so that floating imprecision doesn't cause things to no longer be lined up right when it comes time to performance (you can't hear the difference, but if a pitch doesn't line up with its note then it screws up channel allocation).

So... either switch ScoreTime to fixed as well, or switch RealTime to Double and have a separate fixed Timestamp?  My knee-jerk reaction is to want to switch ScoreTime to fixed, but that's the reaction that turns these simple feature additions into epic polar expeditions.  Maybe I don't need to solve the problem, and can fake it by not bothering to display the position correctly in the presence of multiple tempi.

I always had a bad feeling about using Double for time, all the way from the beginning.  In fact, originally the time was integral, but then I switched it to floating since it seemed like it's more convenient to work with and of such high precision that any built up inaccuracy should be well below the 5ms or so I require to be able to hear the difference.  It's convenient to be able to work with arbitrarily divisible units, especially since the convention was that all time is normalized from 0--1 and then stretched to its final size.  But I had a bad feeling about it even back then.  Well, those chickens may be coming home to roost, I guess.  Unfortunately it's a hassle of a change since it goes all the way down to the C++ layer.

Anyway, the main point was that every seemingly simple little thing gets complicated in a hurry.  I keep thinking some day I should have enough bugs worked out and enough test utilities and enough support in place that this won't happen any more.  But now I'm thinking... maybe that never happens.  Maybe it's unavoidable that every 5 minutes of adding something new is preceded by 2 hours of fiddling behind the scenes.

work continues

I'm working on a music sequencer, and this is a development blog for it.

As I work, I find myself taking notes to myself.  Usually they're just fragmentary bits planning for some feature or bug fix, but sometimes I write more, and it's occurred to me that maybe sometime in the future it will be interesting to have a more organized record of my thought process.  And rather than it going in a bunch of fragmentary files in some directory I'll lose track of, maybe I can put it on the web.  So this is mostly a record for myself, but maybe someone else will find it interesting too.

I've been thinking about the need for a new music composition program and how I could do it ever since highschool, so that's about 15 years now.  I went through some more organized planning in college but wasn't confident enough in my programming skill to know where to start.  After that I got wrapped up in performance and haven't seriously written much music for a long time.  Around 6 years ago I was back in the US and didn't have a job and thought it was a good time to start on a project, so I spent about a month and got most of the UI up---the UI is very simple so that's not saying much.  Then I got a job at Google and got distracted for a long time (though I do remember I discussed some parts of it in the interviews).  Then, about 3 years ago I got serious about working on it again, rewrote the UI stuff and got started on the rest.

So the first darcs checkin is Jan 2008.  I thought it might take a few years, but it's been three so far and will probably be a while yet.  In the meanwhile, I've learned another language, lived in another country, and gotten married.  This is by far the largest project I have ever attempted.  I'm wary of the endless developing and never completing trap, but it really has turned out to be much more complicated than I thought.  Time will tell.

I'd like to think I've learned a lot, about application design (it's also the first GUI-owning program I've written), about dealing with larger programs, and about haskell, which is the main implementation language.  But it's hard to judge about these things.  All I can say now is that I've done things that I hadn't previously done, but not whether I could do them quicker or better than I might have previously, or even if I did them particularly well in the first place.  Maybe if I keeping notes will help with the long term perspective.

Here's the current stats on line count, as measured by wc:

11-07-10
   39775 *.hs     268 files
    2381 *.h      34 files
    5589 *.cc      28 files
     496 *.py       7 files
   48241 total     337


Oh, and the blog title is a bit of a pun.  "Karya" means "work" and can also mean work in the sense of a musical composition, so it seemed an appropriate name for the sequencer.