Category: Technical

How to watch post-season baseball without wanting to stab yourself in the face

You want to watch the post season, but you can’t stand the awful, brain-dead commentary. What do you do? Well, if you use linux, you can spend 7 innings out of nine trying to get jack audio to work — which, when it finally does work, does the job well:

Using jack to delay audio

For those who aren’t familiar with JACK, it’s a “professional”-quality audio library that’s designed for low-latency audio routing and mixing. It’s great if you want to easily pipe audio from one application to another. In this case, I want to take the baseball AM radio broadcast, feed it into my laptop, delay it 7 seconds, and then output it.

So, you can see in the “Connections” window (using the program qjackctl) that the system input is tied to a program called jack-rack, and then jack-rack is fed back to the system output. Jack-rack is a program that supports many interesting audio plugins, like echos, flanges, pitch changes… and delays. Here, I have a seven second delay turned on.

And that’s it. All the difficulty with JACK is getting it to work without stuttering or skipping, which required that I install a “realtime”-quality kernel, edit files in /etc/security, and tweak all sorts of stupid options. As I say, it took me 7 innings of Game 6.

But now it’s working, and I can enjoy game 7 with good commentary from the very beginning!

(It occurs to me I should have gotten this working sooner in the season, but I had forgotten just how bad the national announcers were)

edit: Ah yes, why the delay at all? Because the HD feed is delayed by 7 seconds compared to the radio broadcast. I probably should have mentioned that.

Avid, LEAVE MY LEVELS ALOOOOOOOOOONNNNEE!

Avid has posted a little tutorial on how to export and import quicktimes “correctly.” After reading it several times, I noticed that the author doesn’t cover what I think is the most important use-case: I would like to export video and not touch the levels at all. And then I would like to import it, and not touch the levels at all. Leave my levels alone! Don’t clip them, don’t make them colorsafe. My job as the online editor is to make the show broadcast-safe, and I don’t need the “help”. When Avid screws with my levels, it makes it impossible to roundtrip between Avid and Shake so I can do vfx work.

Thanks to this article, however, I think I finally understand what the Color Levels options in Avid mean:

  • “RGB”: This material I am importing or exporting is EVIL RGB, and needs to be fixed to proper broadcast safety. Please Avid, I am incapable of using a color corrector, won’t you squish (RGB option) or clip (RGB source) my levels for me?
  • “601/709”: LEAVE MY LEVELS ALOOOOONNNEEE. I’ll do my own correction, thanks!

If you select 601/709 everywhere you can, Avid won’t touch your levels and will preserve the full range of the image. I have confirmed this by exporting a dozen files with all sorts of settings. I was able to make the process work two ways:

  • Avid codecs using Format Settings / Video Settings / Options / Color Input: 601/709, and Color Levels: 601/709, then importing 601/709
  • Animation codec with Color Levels: 601/709 and importing 601/709

With the Avid codecs, selecting Color Input: RGB clips data off the top and bottom, and selecting Color Levels: RGB squishes the levels to broadcast safe without clipping.

I have been exporting and importing files incorrectly for years. Along with alpha channels, importing and exporting in Avid is insanely complex. Having backwards alpha channels doesn’t help. This needs to be fixed. Here is how these options should read:

  • Color Levels: Maintain color levels
  • Color Levels: Import as broadcast-safe

3 for 3.0, and Project Sandbox

There has been a lot of discussion recently about the future of GNOME and the stasis that the project seems to have reached ((I dislike the term “decadence,” because it seems to imply abundant wealth going to waste. GNOME will never be the wealthy-person’s OS of choice. That would be OS X)). I think that stasis is not a horrible place to be. As others have said, having a desktop that is stable, useful, and predictable is a good thing. Developers have worked hard to get the desktop where it is, so let’s give them all a pat on the back before we discount all that’s been achieved.

But, as an application developer, I see some shortcomings to GNOME that is preventing further progress in user interface design. There are technical limitations, but there are also social limitations working against progress. On the technical side, GTK needs some changes to allow more developers to take user interaction to the next level. On the community side, there needs to be an official GNOME-sponsored forum in which to experiment free from criticism.

To enable developers to try new things, GTK may need to break ABI stability and move to version 3. I call it “3 for 3.0” — the three features GTK needs to move forward:

  1. Multi-input from the ground up: Right now GTK reacts to one event at a time. It’s possible to make an application look like the user is doing many things at once, but true multi-touch and multi-user interaction is not really possible — or at least it’s too difficult for moderately-skilled developers (like me) to achieve.
  2. First-class Animation: GTK needs to perform animation by default. There are ways to make widgets spin, slide, and fade, but they are all hacks. I should be able to fire off an animation and perform other functions while it is animating. I should get a signal when the animation is complete. Built-in state management would be a key feature. There should be a standard library of basic transitions and special effects that anyone can use with minimal code (like fade, push, wipe).
  3. 3D-awareness: GTK doesn’t need to be 3D itself, but it should understand and be ready for 3D. 2D apps will never disappear, but there will be a need for a bridge between 2D and 3D. This could mean that GTK would support a z-buffer, or perhaps it would have a blessed 3d-canvas like Clutter. Or perhaps it could have access to OpenGL to provide various compositing and shader effects.

Perhaps some of these effects seem only useful for pointless flourishes that will slow down interactivity and increase processor overhead. I would argue that what GNOME needs right now is some stupid slow-ass eye-candy. Look at Compiz Fusion. It has dozens of gaudy effects, half of which are useless and most of which have way too many settings. But I love playing with it. It’s been a fertile sandbox for developers to go in and see what works. Maybe the “fire” transition is a waste of time, but there are a few Compiz features that are genuinely useful and I use all the time. The “enhanced zoom” feature, for instance, is a perfect way to blow up a youtube video without making it full-screen.

Every now and then I see a screencast from a GNOME developer working on a little pet project, and some of those demos have been amazing. Whenever I go to the GNOME Summit in Boston, there’s always some guy with his laptop, and he says, “take a look at this –” and proceeds to blow everyone away with some awesome thing he’s been working on. It’s rare, though, for those hacks to escape from that single laptop onto anyone else’s.

GNOME needs a Project Sandbox — an official, gory-edge (it’s past “bleeding”), parallel installable set of libraries and programs (“Toys,” perhaps? ((The exact terminology isn’t important, I’m just keeping the Sandbox metaphor going))) with all of the crazy hacks developers have been trying. It should be housed in a distributed SCM, so developers can push and pull from each other, mashing features and screwing around with GTK “3”, Clutter, Pyro, and whatever other toys people come up with.

The Sandbox should have two rules:

  1. No Kicking Sand ((alternate title: “No Pooping in the Sandbox”)) — ie, no stop energy. Anything goes, no matter how hacky. Developers should be able to prototype ideas quickly, no matter how cracked up they may be. The good ideas will stick and can be re-written cleanly. Nobody, not Apple, not Microsoft, not Nokia, knows what will really be useful in the future with multi-tap and 3D. Apple has a head start, but that doesn’t mean they have all the answers.
  2. Anyone Can Play: If I pull from your tree, I should be able to build and install what you’ve made. It does no good to have a toy if it only works in your corner of the sandbox. There might be some cases where a feature requires a certain video card, but developers should make a good faith effort to make code build and install on systems other than their own. Setting up a development environment is Hard, but I wouldn’t care if I needed 4 hacked copies of GTK each with different .so-names. Disk space is cheap! Computers are fast!

GNOME has done a good job of reining in developer craziness and promoting consistency and uniformity across the desktop. That was good, and was necessary while the desktop was maturing. Now it’s mature, and those reins need to be lifted, or at least relaxed. The stable desktop can plod forward steadily, but developers need a place to relax, rip off every feature from the iphone and Vista, and more importantly make that code public without fear of attack. What’s worse than a flame on Planet Gnome in response to a crazy feature? The feature that doesn’t get written for fear of being flamed.

Converting Varicam 48fps to 24p

Warning, technical video post production post ahead.

A few weeks back we needed to shoot some greenscreen for a show that is being delivered at 23.98fps (aka “24p”). I’d had problems pulling a key in the past with motion blur at that slow framerate (I prefer 30 for TV work), so I suggested we increase the shutter speed in the camera. The DP seemed more comfortable adjusting the framerate, so he suggested we shoot it at 48 and only use every other frame of the footage. I figured I could write a script to do the conversion later.

We shot the footage, and the next week I sat down to write a python program to convert the material to 24 fps. This could have probably been done with final cut or something, but I don’t know that program so I did it the easy way: play around with the frames themselves using an image sequence (ie, a big folder of numbered tif files, each of which represents a frame of footage).

Normally this would be easy, just take every odd-numbered image. But in this case, although we shot at 48fps, the footage is at 60fps. Why? Varicam works like this: the front of the camera (the shutter and CCD) shoot whatever framerate you choose, but the tape recorder in the camera always records 60fps. Even if you shoot 6fps, the varicam writes 60 frames to the tape — 10 of frame 1, 10 of frame 2, etc. So when we shoot 48fps, there are 60 frames per second on the tape, 48 of which are unique and 12 of which are duplicates.

If I am going to convert the footage to 24p, I need to first remove the duplicate frames (60 -> 48 fps), then remove every other frame (48 -> 24). By analyzing the footage frame by frame, I determined that when the varicam shoots 48fps, it uses the following pattern:

0000100001000010001

Where 0 represents a unique frame, and 1 represents a duplicated frame (to fill out to 60fps). This pattern is repeated over and over again throughout the footage. (I only just noticed that the pattern is 19 frames long, not 20 like I’d expect, but looking at the footage that’s what it is.)

My python program goes through an image sequence, finds the pattern of duplicate frames, and then copies every other file that is not a duplicate to a new folder as a new image sequence. It makes the following assumptions: the files are .tif files, and “duplicate frame” means “exactly the same size” (not a bad assumption with digital media and tifs). It’s a little hacky, but looking at the resulting 24fps image sequences I don’t see any stutter or dropped frames.

There are some basic checks in the code so it hopefully won’t overwrite precious data, but I make no guarantees.

Code: 48-to-24.py

Grading a short film


I had the good fortune of being able to grade (color-correct) a graduate student’s thesis project last weekend. It’s called Mel’s Hole, dir. Kenji Miwa. It was my first narrative project, and my first using Apple’s Color. I usually do documentary work, where the highest priority is to make the footage look “good” and consistent. Also, I’m used to the Avid color corrector, which is not very good for matted secondary color corrections (“brighten his face here”) so it would have been hard to do the sort of aggressive grading that the director wanted.

He’s given me permission to show some before-and-after shots from the film, showing off some of the more fun corrections I got to do. Mouse over the images to see the uncorrected versions. (Shot on a Panasonic HVX200 with a lens adapter for low depth-of-field.)

(Note, these images sometimes appear much too bright on a mac. Set your monitor’s gamma to PC / video standard (2.2) to see the night-time shots correctly.)



The above shot represents the basic look for the film, which is a desaturated “bleach bypass” look. It’s high contrast, with substantial crushing of the blacks and whites. In this shot, we had to knock down the colors of the blanket, which was still too saturated even after we applied the look.



In this scene, the character walks into the woods, which were supposed to be dark and foreboding. By really crushing the blacks we were able to make the woods look deeper and more mysterious. This darkening caused the character to be somewhat lost in the busy-ness of the image, so I put a small tracked oval (the shot pans up) over the character to draw attention to him.



This scene takes place in the middle of the night, and I was instructed to make it very very dark, with a silvery-blue cast. Although the lefthand venetian blind did not have any light behind it, I was able to put one in, which serves to illuminate the character’s face (even though a light back there would really just silhouette him). There are still some bright highlights visible in the blinds, but I wasn’t able to get rid of them.



This shot was actually a last-minute idea. It is paired with another night-time shot, so we decided this shot should also be in night-time. I was able to do a good day-for-night, including drawing in the spill of the light at the bottom of the stairs.

I had a lot of fun doing this grade, and I really liked Apple Color — which makes sense, I doubt they would have bought a company that made a bad color correction program. I do have to say that the keyframing in that program absolutely blows, and the tracker isn’t great either. It also crashed immediately after finishing a render once. But on the whole, it was good at disappearing and letting me work.

The director and DP were great too. We hadn’t really done a lot with aggressive grading before, but once they saw what was possible they were able to direct me better and make requests that were creative but also doable.

World premiere is on May 2nd. The details are on Facebook.

(ps, just shoot me an email if you want me to grade your film — the first job is free!)

PenguinTV’s third platform: maemo

This weekend I hosted a little maemo hackfest with my friend Mike with my goal being to get PenguinTV to launch in the maemo development environment. This meant building python support for mozilla embedding, which was hard. I’m used to checking out code and then building it, but the process of building the microb-engine packages was incredibly obtuse, and had no documentation. But I tried one thing, then another, then another, and finally it worked.

Once the gtkmozembed python module was finally built, porting the application was relatively easy. GTK is really wonderful because you can extract widgets, move them around, and plug them back in where you need them. I can take a toolbar that normally goes here and plug it in there, I can take a preferences window and throw in a scroll bar, and I can turn off that menu item or that button if it doesn’t apply. Sprinkle “if RUNNING_HILDON:” checks where necessary, and:

PenguinTV on Maemo

I always have to do some extra gymnastics because my program requires a lot of special startup initialization and shutdown finalization, but other than that the port was nice and straight-forward.

I’m now working on using all of the maemo-specific features like hardware buttons, network connection, and system messages. It should just be a matter of hooking up to some basic signals to existing functions.

My only big concern about is the “state-saving” feature that allow hildon to completely shut down a program when the user switches away, then restored when the user switches back. Because of all of my threads, database connections, and caches, my startup and shutdown are very slow. I’m not sure if it’s going to be possible to save fast enough. I also don’t know if it’s possible to save the scroll position of a mozilla widget, because when the user switches back they’ll expect to have everything scrolled how they left it. Since gtkmozembed is a black box, it’s just going to start scrolled at the top, which is not acceptable.

Along with olpc and the desktop, PenguinTV now runs on three different platforms using a single code-base.

Back in Business

This Monday morning, I started up my laptop to check the weather report like usual. Although it booted, it quickly crashed. I tried again, and it froze again. I tried memtest86, and that froze too. And so began My Time Without Half My Brain. Something in my thinkpad’s motherboard fried so I had to send it in to IBM to get repaired. Although they assured me the average turnaround is 5-7 days, I knew from experience that, with shipping time, this would be more like two weeks.

This really sucked because after 6 straight weeks of finishing shows and this week being my last, 20th show of the year, I was planning to take much-needed next week off and do some hacking on the OLPC version of PenguinTV. Without my laptop it would be next to impossible to find another linux machine and get it to the point where I’d feed productive on it.

On Tuesday, I got the ez-ship box from DHL and shipped off the poor little laptop Taking care, of course, to remove my not-officially-supported-harddrive with not-officially-supported-OS and insert my original-low-capacity harddrive for them to reimage. DHL: you rock for calling ahead and confirming my address which IBM had wrong.

With my minidock naked and empty, I salvaged some tomboy notes and bookmarks from my backup drive and uploaded them to a VMWare Fusion instance on the mac on my desk By the way, VMWare Fusion is totally awesome and I recommend it highly. I had a fully-functional Ubuntu desktop in the time it takes to download the pre-made image. You can even drag and drop files from the host OS to the guest OS desktop!. This gave me a workable solution for checking email and such, but I still felt slow and stupid without my various array of calendars, bookmarks, and email archives.

Today, I was presented with a package from DHL. My laptop is completely fixed and working again. This is the fastest turnaround time I’ve ever experienced for a laptop repair, and after popping back in my own harddrive everything is just as I left it. And because I got a three-year warranty, it was free.

Now, my hacking plans for next week are back on, and even better I also got an email that I’ve been accepted into the Nokia n810 developer program, so I’m soon going to have two separate mobile platforms for which to develop PenguinTV.

IBM / lenovo / whoever it was: you rock. I’m sorry I ever thought about getting my next laptop from HP.

Avid Symphony Nitris – hope you like progress bars

It’s one thing putting up with Avid because of their kludgy made-it-up-as-they-went interface. I’ll grant that video editors are not particularly technical people, so the cost of modernizing the UI wouldn’t be worth the retraining costs.

But if we pay 90,000$ for the best Avid editing station, which includes 4,000$ for a quad-core Xeon workstation, it should bloody well use more than one processor at once when rendering effects:

If your code is so old and krufty that you can’t support multiple processors for something as simple as effects or video encoding, it’s time for a rewrite.