March 2011 – Owen's Photolog

On Debugging

I’ve been doing more programming than usual recently, both at work writing scripts for managing video media, and at home working on the Mixxx DJ software. It seems 90% of my time is spent debugging, and the more difficult a bug is the more I find myself falling into an unproductive death spiral: try to fix the problem by tweaking a little code, recompile, test, fail, repeat. I’ll think that if I just change this one variable, or move this one bit code I’ll be done, but no it’s still broken and two hours later I’ve gotten nowhere. I keep having to relearn the same lesson: you can’t debug well if you don’t have good information.

Often my first stabs at debugging involves sprinkling print statements around my programs ((This article might be rendered entirely irrelevant if you work the Right Way, using debugging interfaces, stepping through code, and looking at live backtraces. But I suspect a lot of people start with print statements)). For simple problems this can actually work well, but as bugs get more tricky and complicated trying to analyze all that screen barf of can be difficult and a huge waste of time. It’s important to recognize when you’re not getting anywhere.

One problem I’ve been working on is the audio processing in Mixxx. When a DJ is using turntables to play music, and especially when they are scratching, the music changes speed radically, often in a tiny fraction of a second. The music will speed up, slow down, and reverse direction. The playback code has to mimic this bending, stretching, and warping that happens on an actual record player. If there are any errors in the math, it will cause ugly and unacceptable pops and clicks in the playback.

The old Mixxx code had a lot of clicks, buzzes, and pops. Certain playback speeds would cause noise, scratching would cause loud snaps, slow speeds caused rapid popping. I wasted several days trying to analyze those clicks by looking at printouts of variables, and only after I got to the point of complete frustration did I take a step back and think about how to attack the problem better.

I had been testing my code with actual music, and that was difficult because actual music often looks random up close — it’s hard to discern where the errors are in all that noise. So step one was to analyze data for the simplest possible case: a sine wave. Any errors would show up as large deviations from the smooth wave. This is a basic first step in making debugging easier: create a simple test-case. If you’re working on a huge data set, or a giant calculation, or a massive document, pare it down to the bare essentials necessary to reproduce the bug.

But even using a sine wave I couldn’t figure out what was going on. With 44,100 samples per second, there was too much data to analyze based on printed-out numbers. I recalled a bit in Michael Crichton’s “Terminal Man” ((This is why Google Books is a good idea)) that explains the problem:

“People,” Gerhard said, in mock irritation. “They just can’t handle machine data.” It was true. Machines could handle column after column of numbers. People needed to see patterns.

The book then proceeds to show both a list of numbers and a graph — the difference in readability is immediately apparent.

With some help from my dad and brother, both of whom often need to plot data, I was able to write a small script to display my debugging information:

This was a major breakthrough: find the best way to visualize your variables. I’m working with audio data, so a plot is a good choice. It took me a whole day just to adapt my program to write the audio data to a file and to write a new program to display that data, but once I was done the errors showed up as obvious spikes in the graphs. Moreover, by the shape of the waves I could get an idea of why certain clicks were happening: an off-by-one error here, a rounding error there, etc.

The graph above shows several variables: the actual generated sound wave (green), a reference sample (red), playback speed (cyan) ((except that when playback speed is zero, I set it to -5000 so I can see it clearly)), and some other internal variables. If I see a spike I can look at the other lines and figure out where to start looking for a problem — the rate going from negative to positive (a record scratch), perhaps. The output above is correct — a smooth curve that is gradually brought back to zero when the turntable stops. (OK so that linear part at the end is not ideal, it should really be a curve — but otherwise it’s good.)

This is what the old, incorrect code looks like by the way. Notice the obvious errors in the graph, how the green line jumps up and down suddenly:

This method debugging could still be improved. I need to get up and scratch the turntable manually to generate data. Time spent triggering a bug is wasted. Every second you’re clicking around is time not spend debugging, which is why you need to make testing quick and easy. A bad test case is one that requires you click some options, open a file, manipulate some UI elements, and then eventually trigger the bug. An ideal test case would be a single command or button in the UI that performs a test. This keeps your mind focused on the bug, not on the boring, repetitive actions needed to trigger the bug.

Debugging is already a slow, painful process, and the prospect of writing extra code and programs just to debug your code can feel like superfluous work. But time spent writing good test cases and analyzation tools will more than pay itself back. Not only will you solve a bug more quickly, you may not have been able to solve the bug otherwise. It’s hard to get out of the tweak, recompile, test, repeat cycle, but you won’t get unstuck until you make your job easier:

Recognize when you’re not getting anywhere
Create a simple test-case
Find the best way to visualize your variables
Make testing quick and easy

With my Mixxx work, I still haven’t made testing quick and easy. I should probably write an internal test-case that loads the sine wave and performs a few basic scratches automatically. But even doing only two out of three of the steps got me out of the debugging death spiral.

Learning to cook

I had a near-disaster experience while cooking last weekend and, because it wasn’t a disaster, made me feel like I had reached a new competence level when it comes to my cooking skills. In the past I’ve never been able to get a recipe right the first (second, third…) time. I would forget a stage of the process, or mis-time several parts so that by the time the last item was finished everything else would be cold. Constant practice has helped with those issues, but they never really go away.

Last Friday I was planning something fairly daring for me:

A recipe I had never made before
combined with a second item I had never made before
in a way that was my own idea

Normally this would be a movie with the sub-title, “A recipe… for disaster!“, but I felt up to the challenge. Specifically, I planned to make an Israeli cous cous gratin (basically a casserole) and top it with seared giant portobello mushroom tops.

Everything was going smoothly with the gratin, so once that was safe and in the oven I could switch over and concentrate on the mushrooms. I had read something on Salon about sauteing mushrooms, so I poured a bunch of oil in a pan and started to cook the mushrooms.

A minute or two in to the process, I did something that was very new for me while cooking: I realized that sauteing was not working. The mushrooms were gigantic, almost an inch thick, and they were just soaking up all the oil. They weren’t going to get cooked at all. Even a few years ago I would have just kept cooking them, hoping for the best. Instead I calmly determined my worst case scenario: I would have a gratin but no mushrooms on top. So it wouldn’t be the end of the world.

I think this realization (we’ll always have gratin) helped take the pressure off and freed me to do the second new thing, which was try to figure out, OK, how should I cook these things then? And quickly? The next logical choice seemed to be to try broiling them. That would be fast, and would cook them all the way through. Of course this solution led to yet another problem, because of course my gratin was already in the oven. Turning on the broiler would burn it. Still not panicking (well ok a little), thought, heh heh, it’s a casserole so I can just take it out for a few minutes, then put it back in when I’m done broiling. The dish will keep it warm.

Deciding this was the best option, I removed the mushrooms from the pan and put them on foil, then removed the casserole from the oven and put it on the still-warm spot on the stove (which would warm it better than nothing). I put in the portobellos, turned on the broiler, and prayed. As they cooked, it looked like the mushrooms were burning, but the calmer part of my brain realized they were just giving off steam.

A short few minutes later the shrooms seemed cooked, I served everything, and it was delicious.

I don’t think I’ll get to the point where I can create blends of spices and flavors of my own invention — half the time I suggest an idea Char looks at me like I’m crazy and explains that this doesn’t go at all with that. But I can make small permutations and adaptations, and I can mostly get something right the first time, and that’s better than I ever thought I’d do.

“premium services”

The New York Times reports on, as Atrios says, the plight of the not quite rich enough. But I noticed this bit of End-Of-Empire detail:

Across the country, elected officials in privileged communities are caught between a fervor to hold down taxes and a fervor to maintain good schools, well-paved streets, an ample police force, generous library hours and other premium public services that set a community like Bronxville apart.

Good schools, well-paved streets, an ample police force, and generous library hours: these are what constitute “premium services” in America, 2011.