2013-09-07
A common theme of a lot of my posts is me noting how something that seems simple is actually really complicated, followed by a long exposition of how complicated it really is. I guess that's because I'm continually surprised by how much work it is to do anything. Even in haskell, a language which is supposed to conducive to expressing complicated things.
By writing these expositions I'm trying to answer a question that is always in my mind while programming. That is, is music just inherently complicated, or am I failing to see some important abstraction or generalization or even specialization that would simplify things? When I struggle through some never-ending feature-add or bug-fix session I'm wondering what I did wrong, and how I could avoid it next time.
Of course this is one of those perennial questions about programming, about inherent versus incidental complexity. People who are good at what they do, and good languages, are supposed to reduce the latter. I'd like to do that, to become good at what I do, and ultimately save time.
Today I wanted to finish a feature to allow piano and ASCII keyboard input for scales with varying numbers of degrees, then slice up and create a sampler patch for the recorded kendang samples from last week, practice mridangam, and then practice kendang (using the new patch). The topeng dancer is coming to tomorrow's rehearsal and I'd like to practice the cedugan part so as to not embarrass myself too badly.
It's looking like the feature add is not going to get done in time, so I'm putting it on hold while I go to practice. I've actually been working on it for a full week, on and off. It's so hard that it's discouraging, and that slows me down. And it sounds so simple!
The basic problem is that I have input, coming from either a piano-style MIDI keyboard, or the ASCII keyboard mapped to an organ-like layout, and I need to figure out what note in the current scale should come out.
Previously I had a (somewhat) straightforward way. The input key was a Pitch.InputKey, which was just a newtype around an integer, which was the number of semitones from MIDI note 0, otherwise known as the Midi.Key. Then it was up to the scale to map from InputKey to a Pitch.Note.
This worked well enough at first, but when scales got more complicated it started to have problems.
It started with relative scales, e.g. sa ri ga style. I wanted sa to always be at a constant place, so I had C emit sa. That works fine diatonically but runs into trouble when you start using accidentals. If D major is being played starting at C, the black keys are in the wrong place, and you wind up with some notes unplayable. It got worse when I tried to get Bohlen-Pierce to map right. It has 9 diatonic and 13 chromatic steps per "octave" (tritave, actually), so its layout is nothing like a piano keyboard's.
So it seemed like I should treat the ASCII and piano keyboards differently, so I could make the ASCII keyboard always relative, while the piano keyboard would remain absolute as usual. So the InputKey would get its own structure, with octave, pitch class, and accidentals. This involved updating Pitch.InputKey (and renaming Pitch.Input while I was at it), and then redoing how all the scales converted input to symbolic pitches, and that involved a bunch of refactoring for scales, and then fussing around with octave offsets to get them roughly consistent across scales.
The actual conversion is a bit of head-hurting modular arithmetic, to convert from a 7 or 10 key keyboard to an n-key scale, wrapping and aligning octaves appropriately. In theory it's straightforward, but finicky and hard for me to keep straight in my head.
But now it all seems to work, and I can get rid of an awful hack that hardcoded Bohlen-Pierce in favor of all scales adapting based on the number of pitch classes.
Worth it? Maybe? Pitch input is what I spend most time on when actually writing music, so anything that makes that smoother seems worthwhile.
What else? Slow progress lately, since I've been distracted by travel and sampling. Speaking of which, I recently finished the patches for the gender wayang. That's another thing that sort of boggles the mind. Pemade and kantilan, with umbang and isep variations, ten keys on each, with four dynamics and from three to eight variations of each, played with gender panggul and calung panggul, and muted kebyar style and loose gender style. That's 4,313 samples recorded, edited, organized, and mapped to the right key, velocity range, round robin configuration, and envelope. 2.6gb worth, and it took two weekends to record and several weeks on-and-off to do the editing. Even with lots of automation, it's a lot of work, and you need a well defined process and lots of notes to not make mistakes. There are still tweaks to be made, with jagged dynamics to be smoothed and missing articulations to be rerecorded. I guess that's another thing that's complicated about music.
And I'm not done yet, I still need to do the gender rambat and reyong / trompong, both of which have more keys.
Saturday, September 7, 2013
Saturday, June 22, 2013
sketch-13-06-19
A couple of days ago, I had an idea while biking home. So when I got home I wrote it down and then started fiddling to see where else it would go. And it basically turned into a piece, which is now in need of wrapping up, and some name better than "sketch-13-06-19".
That's a bit of a milestone, because it's the first time I've used the sequencer to actually write something beyond a notation test, or a transcription of an old piece. It's not really what I would have chosen, since it came about by happenstance. It's the sort of intricate bit of video-gamey clockwork that I was doing back in high-school, and is mostly in piano because that was a convenient sound when I was writing the sketch and now it's a bit stuck there, and it doesn't really take advantage of any interesting new capabilities. I wasn't trying to create something controlled, and so I got a bit of a tangled undergrowth. As ever I'm not entirely in control of what catches and what slips away.
But I'm still pretty happy. In a way I've been waiting thirteen years for this, and working four years on a program to get there (that would be eight years, if you count on-and-off work). This is the first hint of the fruition of those four years, the hint that those four years may not have been in vain. Worth it? For whatever reasons, I don't think it would have happened any other way, perhaps because of my determination to write a program to write music, and then determination that the time I spent on the program not be a waste. In creative endeavours we so rarely reach our goals from straight lines.
Despite all that, it's funny how little has changed. I'm infinitely richer in resources, influences, sounds... but I've basically picked up right where I left off. But that's fine, the fancy things can wait. The real job is learning to write music, and that will take many more than eight years. I'm not in a hurry.
That's a bit of a milestone, because it's the first time I've used the sequencer to actually write something beyond a notation test, or a transcription of an old piece. It's not really what I would have chosen, since it came about by happenstance. It's the sort of intricate bit of video-gamey clockwork that I was doing back in high-school, and is mostly in piano because that was a convenient sound when I was writing the sketch and now it's a bit stuck there, and it doesn't really take advantage of any interesting new capabilities. I wasn't trying to create something controlled, and so I got a bit of a tangled undergrowth. As ever I'm not entirely in control of what catches and what slips away.
But I'm still pretty happy. In a way I've been waiting thirteen years for this, and working four years on a program to get there (that would be eight years, if you count on-and-off work). This is the first hint of the fruition of those four years, the hint that those four years may not have been in vain. Worth it? For whatever reasons, I don't think it would have happened any other way, perhaps because of my determination to write a program to write music, and then determination that the time I spent on the program not be a waste. In creative endeavours we so rarely reach our goals from straight lines.
Despite all that, it's funny how little has changed. I'm infinitely richer in resources, influences, sounds... but I've basically picked up right where I left off. But that's fine, the fancy things can wait. The real job is learning to write music, and that will take many more than eight years. I'm not in a hurry.
Sunday, April 21, 2013
piano sketch
I just opened an old saved score, and played it. Hmm, sounds like a nice little sketch, I should do something with that someday. But the dynamics are broken... oh right, because it uses the old-style 'vel' control, and not the new 'dyn' one. It also uses the old decimal notation, instead of hex, but it's still compatible. Then I noticed what I thought was a weird graphical glitch that made the track skeleton look backwards, until I realized... oh right, no track slicing back then so it was all backwards. I created a new track and it had a messed-up looking ruler. I was about to make note of a new bug when I realized... oh right, this has old style overlay rulers, which are obsolete now.
It's easy to forget how much things have changed.
Saturday, March 23, 2013
slicing
One week of head-hurting work, all because a slur from a non-harmonic to a harmonic wasn't showing up. It turns out it was because track slicing, always an unintuitive mess, wasn't working how I thought it did. In fact, it was hard to think of how it worked, because it was an unintuitive mess. So it took ages to sort out because first I had to figure out what it was doing, figure out if that made sense, figure out what would make sense, and then implement that.
Add in a couple false starts and having to update tons of tests and it winds up eating up quite a bit of time. The tons of tests are a result of it being too complicated in the first place, so it's a good demonstration of how complexity consumes an non-linear amount of time. Added to that is that I'm dealing with trees which have two-dimensional structure, which are hard to visualize and to write down (in fact the tests have ASCII-art diagrams next to them). Actually, I think the greatest time sink is general disillusionment. When I'm unsatisfied with the whole situation I spend a lot of time trying to step back to figure out what the real problem is, or getting distracted because there's no real sense of momentum to keep my focus, or playing games because I'm just too tired to want to deal with the hairball.
The result is that I have a moderately more useful way of turning the 2D score structure into notes and transformations, and a check which I think should cause an error for events that overlap in the wrong way, which tend to cause doubled notes or missing notes or other unintuitive things.
The general problem is one of composing together 2D score structure, or rather turning it into the 1D structure needed to actually evaluate it. The score can get quite complicated, because there are transformations that apply to all notes, those that only apply to one melodic line, those that apply only to lilypond realization, and those that only apply to synthesizer realization. But the key to having a usable score notation is that the individual parts compose together in a natural and useful way.
Add in a couple false starts and having to update tons of tests and it winds up eating up quite a bit of time. The tons of tests are a result of it being too complicated in the first place, so it's a good demonstration of how complexity consumes an non-linear amount of time. Added to that is that I'm dealing with trees which have two-dimensional structure, which are hard to visualize and to write down (in fact the tests have ASCII-art diagrams next to them). Actually, I think the greatest time sink is general disillusionment. When I'm unsatisfied with the whole situation I spend a lot of time trying to step back to figure out what the real problem is, or getting distracted because there's no real sense of momentum to keep my focus, or playing games because I'm just too tired to want to deal with the hairball.
The result is that I have a moderately more useful way of turning the 2D score structure into notes and transformations, and a check which I think should cause an error for events that overlap in the wrong way, which tend to cause doubled notes or missing notes or other unintuitive things.The general problem is one of composing together 2D score structure, or rather turning it into the 1D structure needed to actually evaluate it. The score can get quite complicated, because there are transformations that apply to all notes, those that only apply to one melodic line, those that apply only to lilypond realization, and those that only apply to synthesizer realization. But the key to having a usable score notation is that the individual parts compose together in a natural and useful way.
Sunday, January 20, 2013
spelling
4/4
0 w | 32
1 h h | 16
2 q q q q | 8
3 e e e e e e e e | 4
4 s s s s s s s s s s s s s s s s | 2
5 33333333333333333333333333333333| 1
0 0 w-------------------------------| 32
5 3 s.-| 3
4 2 e.----| 6
5 3 3| 1
3 1 q.----------| 12
5 3 s.-| 3
4 2 s-| 2
5 3 3| 1
2 0 h.----------------------| 24
5 3 s.-| 3
4 2 e.----| 6
5 3 3| 1
3 1 e---| 4
5 3 s.-| 3
4 2 s-| 2
5 3 3| 1
2 0 h---------------| 16
5 3 s.-| 3
4 2 e.----| 6
5 3 3| 1
3 1 q.----------| 12
5 3 s.-| 3
4 2 s-| 2
5 3 3| 1
2 0 q-------| 8
3+3/8
0 h. h.
1 q. q. |
2 e e e e e e |
3 s s s s s s s s s s s s |
0 0 h.----------------------|
4 3 s.-|
3 2 s-|
4 3 3|
2 1 q-------|
4 3 s.-|
3 s-|
4 3|
2 e---|
4 s.-|
3 s-|
4 3|
1 q.----------|
4 s.-|
3 s-|
4 3|
2 q-------|
4 s.-|
3 s-|
4 3|
2 e---|
4 s.-|
3 s-|
4 3|
2+2+2/8
0 h.
1 q q q |
2 e e e e e e |
3 s s s s s s s s s s s s |
4 333333333333333333333333|
0 h.----------------------|
4 2 3s.-|
3 2 2 s-|
4 2 3 3|
2 1 e---|
4 2 s.-|
3 2 s-|
4 2 3|
1 0 h---------------|
4 2 s.-|
3 2 s-|
4 2 3|
2 1 e---|
4 2 s.-|
3 2 s-|
4 2 3|
1 0 q-------|
4 2 s.-|
3 2 s-|
4 2 3|
2 1 e---|
4 2 s.-|
3 2 s-|
4 2 3|
0 h.
1 q q q |
2 e e e e e e |
3 s s s s s s s s s s s s |
4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 |
5 666666666666666666666666666666666666666666666666|
0 0 h.----------------------------------------------|
5 3 3.-|
4 2 s.----|
5 3 6|
3 2 s---|
5 2 3 3.-|
4 2 2 3-|
5 2 3 6|
2 1 e-------|
I spent the weekend drawing diagrams like those. A musician might recognize them as the "optimal" way to spell a note beginning at each point in a measure. Since my score uses a low level start time + duration concept, I need to figure out how to spell rhythms automatically when I convert to lilypond. Previously I used a quick and simple technique that worked surprisingly well, but only for duple. Since it made a 6/8 piece totally ugly I decided it was time to solve the problem for real. That turns out to be either way more complicated than I thought it would be, or I'm doing it totally wrong!
In any case, it was an exercise in design-by-diagram, since I'm definitely not smart enough to visualize rhythmic notation in my head.
First strategy: try to end up on a major division which isn't over the barline. So define a "rank" for each possible position in the measure (I only go to 128th notes, so there are 128 per whole note), and pick the duration (possibly including dotted durations) which ends on the lowest rank. Exploring this with diagrams, it seemed to line up with my expectations:
0 w | 32
1 h h | 16
2 q q q q | 8
3 e e e e e e e e | 4
4 s s s s s s s s s s s s s s s s | 2
5 33333333333333333333333333333333| 1
5 3 s.-| 3
5 w-------------------------------|
5 h.----------------------|
5 h---------------|
5 q.----------|
5 q-------|
5 e.----|
5 e---|
* 3 s.-|
5 s-|
4 3|
4 2 e.----|
> w-------------------------------|
4 h.----------------------|
4 h---------------|
4 q.----------|
4 q-------|
* 2 e.----|
5 e---|
4 s.-|
3 s-|
4 3|
0 w | 32
1 h h | 16
2 q q q q | 8
3 e e e e e e e e | 4
4 s s s s s s s s s s s s s s s s | 2
5 33333333333333333333333333333333| 1
5 3 3| 1
> w-------------------------------|
5 h.----------------------|
5 h---------------|
5 q.----------|
5 q-------|
5 e.----|
5 e---|
4 s.-|
4 s-|
* 3 3|
0 w | 32
1 h h | 16
2 q q q q | 8
3 e e e e e e e e | 4
4 s s s s s s s s s s s s s s s s | 2
5 33333333333333333333333333333333| 1
3 1 q.----------| 12
> w-------------------------------|
4 h.----------------------|
3 h---------------|
* 1 q.----------|
3 q-------|
4 e.----|
2 e---|
5 s.-|
4 s-|
5 3|
0 h.
1 q. q.
2 e e e e e e |
3 s s s s s s s s s s s s |
4 3 s.-|
> w-------------------------------|
> h.----------------------|
4 h---------------|
4 q.----------|
4 q-------|
4 e.----|
4 e---|
* 2 s.-|
4 s-|
3 3|
Unfortunately this doesn't quite work, but of course I had to implement it and run tests to find out. Given 6/8, interpreted as 3+3/8, it wil happily put a half-note at the 3rd eighth note, because that ends at the end of the measure, rank 0 by definition. But that obscures the middle of the measure, which is forbidden:
0 h.
1 q. q.
2 e e e e e e |
3 s s s s s s s s s s s s |
0 0 h.----------------------|
h---------------| <- was
q.----------|
q-------|
e.----|
e---| <- should have been
Or rather, it's forbidden, but only for notes which start on rank 2! The dotted half-note at the beginning of the measure can span it, because it starts at rank 0. So I switched strategies: a note can extend until it reaches a division it's not allowed to span. "Not allowed" means one rank lower for complex meters, and two ranks lower for duple ones. Duple meters fit better with the log2 nature of note durations, so they can be more liberal in their spelling.
Unfortunately, this also has a problem: a 5/4 measure can start off with a whole note. That means a measure-long note would be spelled c1~c4, which is super ugly. If I were spelling 5/4, I would need to know whether it is mostly 3+2/4 or 2+3/4 and tie accordingly. The underlying problem is that first time in the measure is rank 0, so it will happily span the center of the measure. Normally that's fine, but for irregularly divided measures the spelling needs to be more conservative to make the irregular division clear (and because a single note won't fit). So I tweaked the definition of irregular measures to give the toplevel major divisions rank 0 instead of just the barline, and never cross rank 0 no matter what. It winds up doing a few things I wouldn't do, e.g. I would use liberal duple spelling in the duple portion of a compound meter while it doesn't, but tolerance for complex spelling is individual and it doesn't hurt to be a little more conservative.
There remains yet another problem: this spells rests incorrectly. Since it always tries to find the longest duration that will fit, if I have dotted rests disabled, I wind up with "c4 c2 c4". Rests are different from notes in that they are all interchangeable, so if there are two rests in a row they should be arranged to align with major beats: "c4 r4 r2". The solution to this was to bring back the sort-by-rank idea, but only for rests. If I'm trying to spell a rest, find the maximum duration as above, but then try all of the durations that fit within it, and pick the one that ends on the lowest rank.
But wait! There's one last wrinkle. At least it's the last one I'm going to worry about today. While I normally avoid dotted rests, they look better for the major divisions of a triple meter. E.g. 3+3/8 should be spelled "r4. r4." rather than "r4 r8 r4 r8". I wound up allowing dotted rests for triple meters. I should probably only allow them at the rank that involves the triple division (the denominator's rank), and actually technically in a complex meter like 2+3/4, I should only allow them in the triple fraction of the meter, but it seems like too much bother, so I'm just enabling them universally.
One other practical problem that I'll have to look into is that I have to define where the ranks fall for various meters, which means I have to define each meter before using it. It seems like it should be possible to figure out everything from just looking at the name of the meter, though I'd have to write it explicitly, e.g. 3+2/4 rather than 5/4.
This is one of those problems where it seems like there might be some simple "best" solution. But it also might just be a set of heuristics and a notion of prettiness, which is something people are good at but machines aren't. When I tried to analyze my own process in deciding how to spell a note, I came up with a small set of heuristics, but discovered more and more as the meters become more complicated.
I still feel like if I were smarter I would have been able to identify some optimal heuristic from the beginning, or at least forsee the various problems without bumbling into a problem, patching it up, and then bumbling into another.
This is hopefully the last of the major lilypond changes. My goal is to typeset a simple piano piece acceptably, and it's getting close. Recent other lilypond improvements:
- Preserve enharmonics. That meant extending pitches so they generate not just frequencies, but a symbolic note name, applying whatever diatonic and chromatic transpositions.
- Tuplets. This was a whole other involved saga. Since tuplets involve nested chunks of lilypond code, whose composite has a different duration than the sum of its parts I wound up needing to interleave score derivation with lilypond "performance". I tried to just generate pitches, but when I would up with grace notes nested in a tuplet I realized it had to be more general.
- Grace notes. Easy once the work was done for tuplets.
- A bunch of simpler things, like various articulations, ottava notation and the like. I switched from having the lilypond interpret note attributes to having the call directly emit lilypond code if it's being derived for lilypond. It means lilypond knowledge is scattered throughout the calls instead of consolidated in the lilypond performer, but it makes it really trivial to add a lilypond interpretation to a call, and generally simplifies things, since the call has score context.
Before:
After:
The rests should be dotted but the rest looks much better.
Saturday, December 29, 2012
Lag, drag, and memory corruption.
LAG AND DRAG
I'm definitely not loving laziness at the moment. That's because I've spent the last two days trying to track down excessive amounts of DRAG in score derivation. Not only are they excessive, but they grow more and more on each derivation. That shouldn't happen because each derivation is independent and clears out the old one, but clearly something is being retained.
Part of the problem is that since I wasn't sure where the problem was, I hesitated to spend time creating an automatic reproduction, since I've done that sort of thing in the past only to find out the problem is related to the specific way that the GUI interaction handles the data. This is a symptom of laziness space leaks being non-local. And since debugging is basically trial and error, it took a long time to change something, recompile, run again, poke a bit, quit, and inspect the heap dump (though the ekg library helps). Once I figured out what I had to force to get the DRAG to go away, I spent a lot of time on a kind of binary search either forcing bits of data or clearing it, to narrow down which bits did the trick. It's complicated because frequently it's more than one thing, i.e. more than one bit of output is retaining the data.
The results aren't simple either. For example making the cache strict would take about 5 MB off the growth... but I had to touch the UI for that to happen, so it's clearly being hidden behind another thunk. And then 5 more MB would be cleared up when I ran another derive, but forcing everything would clear it all up immediately.
There are numerous clues, if only I knew how to interpret them, I took to writing them down because I can't remember them all at once:
I eventually figured out that the continually growing part was actually buried in PitchSignals, which are the only bit of the Score.Event which is code, not data. That means, of course, rnf doesn't work on it. It turns out they capture the environ in a closure. While the environ itself is small, it's clearly not entirely evaluated, because rnf'ing the environ before creating the closure wiped out the DRAG. The environ is basically a Map, but despite my attempts to make insertion strict, the only thing that's worked is rnf'ing the environ before each closure is created. Maybe I can use ghc-heap-view to see which parts of the environ are unevaluated. But anyway, that explains the the retains-forever thing, since the still-valid parts of the cache are copied to the new cache.
I've given up on the idea of lazy derivation, since it's not working out in practice, and I want to parallelize derivation. But I feel like the work making derivation lazy wasn't totally a waste, because it was basically eliminating data dependencies, and that's a prerequisite for parallelization as well.
If my usual practice is to strictify every data structure I use, and I have all sorts of problems due to built-in data structures being lazy, wouldn't it be easier if everything was strict to begin with? I still use laziness for Perform, and I think it's useful there, but it's just a stream. In a default-strict language maybe I could just mark that one thing lazy and not have to deal with all this mess in the first place.
I'm definitely not loving laziness at the moment. That's because I've spent the last two days trying to track down excessive amounts of DRAG in score derivation. Not only are they excessive, but they grow more and more on each derivation. That shouldn't happen because each derivation is independent and clears out the old one, but clearly something is being retained.
Part of the problem is that since I wasn't sure where the problem was, I hesitated to spend time creating an automatic reproduction, since I've done that sort of thing in the past only to find out the problem is related to the specific way that the GUI interaction handles the data. This is a symptom of laziness space leaks being non-local. And since debugging is basically trial and error, it took a long time to change something, recompile, run again, poke a bit, quit, and inspect the heap dump (though the ekg library helps). Once I figured out what I had to force to get the DRAG to go away, I spent a lot of time on a kind of binary search either forcing bits of data or clearing it, to narrow down which bits did the trick. It's complicated because frequently it's more than one thing, i.e. more than one bit of output is retaining the data.
The results aren't simple either. For example making the cache strict would take about 5 MB off the growth... but I had to touch the UI for that to happen, so it's clearly being hidden behind another thunk. And then 5 more MB would be cleared up when I ran another derive, but forcing everything would clear it all up immediately.
There are numerous clues, if only I knew how to interpret them, I took to writing them down because I can't remember them all at once:
- Clearing both TrackDynamic and Cache gets rid of it, but only after one derive. It's retained elsewhere, but I don't know where.
- It's got something to do with perf vs. current perf, since clearing current perf doesn't do anything, but clearing perf does. They should both point to the same data, so it should be both or nothing, right?
- Forcing the cache gets rid of just one derive's worth of drag.
- Drag is mostly Map and [].
- If it grows by 15mb each time, force gets rid of 10mb. 5mb of permament growth is hiding somewhere.
- Clearing the performance entirely gets rid of the last 5mb, but it still requires a derive or force to get it to happen. Where is that reference hiding?
- The first run has no DRAG, it only appears on the second run. Clearly this is related to caching.
- The VOID grows slowly, and it's mostly []. Actually it looks like the [] is constant, and the growing part is misc other things like Double.
- There's almost no LAG, except right at the beginning. So that also points to how the cache is retained across derivations.
I eventually figured out that the continually growing part was actually buried in PitchSignals, which are the only bit of the Score.Event which is code, not data. That means, of course, rnf doesn't work on it. It turns out they capture the environ in a closure. While the environ itself is small, it's clearly not entirely evaluated, because rnf'ing the environ before creating the closure wiped out the DRAG. The environ is basically a Map, but despite my attempts to make insertion strict, the only thing that's worked is rnf'ing the environ before each closure is created. Maybe I can use ghc-heap-view to see which parts of the environ are unevaluated. But anyway, that explains the the retains-forever thing, since the still-valid parts of the cache are copied to the new cache.
I've given up on the idea of lazy derivation, since it's not working out in practice, and I want to parallelize derivation. But I feel like the work making derivation lazy wasn't totally a waste, because it was basically eliminating data dependencies, and that's a prerequisite for parallelization as well.
If my usual practice is to strictify every data structure I use, and I have all sorts of problems due to built-in data structures being lazy, wouldn't it be easier if everything was strict to begin with? I still use laziness for Perform, and I think it's useful there, but it's just a stream. In a default-strict language maybe I could just mark that one thing lazy and not have to deal with all this mess in the first place.
Currently I think I have three major leaks:
- Too lazy environ captured in PitchSignal closures. Workaround is rnf'ing the environ, but hopefully is fixable by making environ more strict.
- Events in saved in the cache are too lazy. The cache filters out log msgs about caching, since getting cached log msgs about caching from the previous cache is just really confusing. The filter, being lazy, probably retains all the events and their referents. The mysterious part is that the next run should use or toss the cache, which should clear the references either way, but doesn't. In any case, I can work around by rnf'ing the cached events, but I don't know what the fix would be. Making the entire derive output be a strict list? Or maybe its the events themselves that need to be stricter?
- Something else unknown in the performance. I infer its presence because given about 15mb of growth after one derivation, forcing the cache clears about 5mb, forcing the environ clears about 5mb, but forcing the entire performance clears it all.
Of course, capturing too much in a closure can happen in strict languages too, but it's so much easier to do so when any data structure could actually be a thunk that retains who knows how much data from who knows how far away.
MEMORY CORRUPTION
The other major problem is nastier, and I don't even have a good avenue of attack. Ever since upgrading to ghc 7.6.1, I get internal errors from the GC, which probably means memory corruption. It happens most frequently in the optimized build, and seemingly more frequently when it's using the libgit2 save format. My first suspicioun is FFI code.
It also happens in the debug build. It happens when I'm not using libgit2. It happens when I'm using the stub MIDI driver. The one time it doesn't happen is during tests. I guessed it's in the GUI binding, but it also doesn't happen when I set up a script to drive the GUI. Well, it's driving it via a socket, not with keyboard and mouse, so maybe it's in keyboard / mouse event stuff. The fact that it only happens after an unpredictable amount of interactive use really makes it hard to take the trial-and-error approach that has been so slow on the memory leak problem.
As far as I know, this bug has been present for years, since long ago I had the same symptoms and "fixed" them by changing an alignment=1 Storable to alignment=4, which made no sense, since alignment=1 was correct. But increasing the alignment could have just masked the overrun.
Valgrind shows nothing. I have no way to reproduce it that's faster than 15 minutes of normal usage. But I can't use a program that crashes every 15 minutes.
Saturday, December 1, 2012
prometheus
Not related to karya, but I wrote all this stuff a long time ago when Prometheus first came out. Bad Prometheus reviews have since become a kind of genre, so here's my entry into the genre. Naturally this will spoil the whole movie if you haven't already seen it.
The expedition was ridiculously disorganized and haphazard considering they're walking into some ancient alien complex.
Biologist guys decide "we're going back all alone" because they're scared about weird bodies and stuff. Yes, let's go back alone so we can get lost. Even on a normal expedition on Earth I don't think you get two guys throwing a fit and deciding to wander back alone through a maze. I guess they forgot about the fancy automappers, because they got lost anyway. I also think you've kind of failed as a storyteller if you need to the characters to tell us what implausible thing happened as their first line: "oh, looks like we're still here because we got lost". I guess they're fulfilling their Guys Who Get Killed First duty.
Meanwhile, our haphazard other heroes are bumping into things, knocking things over, shouting at each other, and not paying any attention to the android who can mysteriously operate alien machinery.
And someone drops her bag, and decides to dive back into the raging knife storm after it. Even if it had something they haphazardly ripped off from the complex, and there's like a million cylinders and alien bodies and whatnot in there. A dropped knapsack is totally worth a knife bath, right?
Then there's this thing where they're all mopey like "oh we didn't find anything, it's all empty, let's give up and go home" when they spent 15 minutes inside a tiny part of a giant complex and saw huge amounts of crazy stuff and alien writing and working machinery and had to run back early because the weather got bad.
There was a strange scene I forgot about with the head. If you retrieve the head of an ancient alien corpse I guess the first thing you want to do is pump it full of adrenaline, which I guess makes alien heads relive their last moments. So it will tell you... what it was running from maybe? Who knows, because they accidentally put in the chemical that makes alien heads explode. And then they're like shrug, so much for that alien head, and forgot about it entirely, which is what I did too.
And then the biologists (geologists? hard to tell since they didn't seem to do any of either) are stuck in a deeply creepy alien complex and the captain on the ship is just laughing at them. "Yeah, there's something moving down there, along with all the horribly murdered alien corpses. Sweet dreams!" It's like they all knew they were in a movie and these guys were the Guys Who Get Killed First, so the captain is just winking at the audience.
But then the strangest thing is that these guys, after doing stupid things because they're scared, and then getting terrorized more by the captain, and running in the opposite direction from the "something moving", then when they actually see a scary alien they decide it's cute and they want to pet it. Even after it hisses and makes all kinds of threatening gestures they just want to pick it up. Aw, the alien snakebeast is hissing, I think it wants to be friends. Kane wasn't that stupid.
I've never seen a C-section, but I don't think that's how it works. Vickers' surgery table is mysteriously incompatible with women (I guess it's for the robot then? But he can amputate his entire body and still be just fine.), but it does have a convenient little organ removal claw. It also has a convenient button on the outside that floods the inside with poisonous gas. Forget the critters, when you think about it the operating table is a horror movie all by itself.
Oh and before that she hits one of the doctors / orderlies (who I guess are trying to help her get exploded by an alien? when did they turn into psychopaths?) and both of them apparently decide not to chase her. Or raise an alarm. Or do anything else for the rest of the movie. It's not so much that they forgot to chase her, it's that the scriptwriter forgot they existed. Then she finishes up with her ordeal and it's like there's a tacit agreement to not mention the attempted murder-by-alien. When Burke did that in Aliens it took a power failure and alien attack to distract them from killing him on the spot.
Anyway, they get to the Engineer and they're like "let me talk", "no let *me* talk", "me talk me talk me talk!" and hitting each other. And the Engineer decides the first thing he should do after sleeping through his entire installation being wiped out, is to beat up the critters that woke him up (I understand, they *are* acting like 5 year-olds), and then go find their planet and presumably rain horrible black biological death on it.
Then the captain says "Hey guys, I think we'll have to crash into that ship and kill ourselves. Because it's taking off, and we can't let alien ships take off. And I have to be standing here while it happens." Other guys: "Oh. Ok. Sounds like we should stand here too."
The crew in Alien got wiped out, but at least they were trying.
Subscribe to:
Posts (Atom)

