Avid, LEAVE MY LEVELS ALOOOOOOOOOONNNNEE!

Avid has posted a little tutorial on how to export and import quicktimes “correctly.” After reading it several times, I noticed that the author doesn’t cover what I think is the most important use-case: I would like to export video and not touch the levels at all. And then I would like to import it, and not touch the levels at all. Leave my levels alone! Don’t clip them, don’t make them colorsafe. My job as the online editor is to make the show broadcast-safe, and I don’t need the “help”. When Avid screws with my levels, it makes it impossible to roundtrip between Avid and Shake so I can do vfx work.

Thanks to this article, however, I think I finally understand what the Color Levels options in Avid mean:

  • “RGB”: This material I am importing or exporting is EVIL RGB, and needs to be fixed to proper broadcast safety. Please Avid, I am incapable of using a color corrector, won’t you squish (RGB option) or clip (RGB source) my levels for me?
  • “601/709”: LEAVE MY LEVELS ALOOOOONNNEEE. I’ll do my own correction, thanks!

If you select 601/709 everywhere you can, Avid won’t touch your levels and will preserve the full range of the image. I have confirmed this by exporting a dozen files with all sorts of settings. I was able to make the process work two ways:

  • Avid codecs using Format Settings / Video Settings / Options / Color Input: 601/709, and Color Levels: 601/709, then importing 601/709
  • Animation codec with Color Levels: 601/709 and importing 601/709

With the Avid codecs, selecting Color Input: RGB clips data off the top and bottom, and selecting Color Levels: RGB squishes the levels to broadcast safe without clipping.

I have been exporting and importing files incorrectly for years. Along with alpha channels, importing and exporting in Avid is insanely complex. Having backwards alpha channels doesn’t help. This needs to be fixed. Here is how these options should read:

  • Color Levels: Maintain color levels
  • Color Levels: Import as broadcast-safe

3 for 3.0, and Project Sandbox

There has been a lot of discussion recently about the future of GNOME and the stasis that the project seems to have reached ((I dislike the term “decadence,” because it seems to imply abundant wealth going to waste. GNOME will never be the wealthy-person’s OS of choice. That would be OS X)). I think that stasis is not a horrible place to be. As others have said, having a desktop that is stable, useful, and predictable is a good thing. Developers have worked hard to get the desktop where it is, so let’s give them all a pat on the back before we discount all that’s been achieved.

But, as an application developer, I see some shortcomings to GNOME that is preventing further progress in user interface design. There are technical limitations, but there are also social limitations working against progress. On the technical side, GTK needs some changes to allow more developers to take user interaction to the next level. On the community side, there needs to be an official GNOME-sponsored forum in which to experiment free from criticism.

To enable developers to try new things, GTK may need to break ABI stability and move to version 3. I call it “3 for 3.0” — the three features GTK needs to move forward:

  1. Multi-input from the ground up: Right now GTK reacts to one event at a time. It’s possible to make an application look like the user is doing many things at once, but true multi-touch and multi-user interaction is not really possible — or at least it’s too difficult for moderately-skilled developers (like me) to achieve.
  2. First-class Animation: GTK needs to perform animation by default. There are ways to make widgets spin, slide, and fade, but they are all hacks. I should be able to fire off an animation and perform other functions while it is animating. I should get a signal when the animation is complete. Built-in state management would be a key feature. There should be a standard library of basic transitions and special effects that anyone can use with minimal code (like fade, push, wipe).
  3. 3D-awareness: GTK doesn’t need to be 3D itself, but it should understand and be ready for 3D. 2D apps will never disappear, but there will be a need for a bridge between 2D and 3D. This could mean that GTK would support a z-buffer, or perhaps it would have a blessed 3d-canvas like Clutter. Or perhaps it could have access to OpenGL to provide various compositing and shader effects.

Perhaps some of these effects seem only useful for pointless flourishes that will slow down interactivity and increase processor overhead. I would argue that what GNOME needs right now is some stupid slow-ass eye-candy. Look at Compiz Fusion. It has dozens of gaudy effects, half of which are useless and most of which have way too many settings. But I love playing with it. It’s been a fertile sandbox for developers to go in and see what works. Maybe the “fire” transition is a waste of time, but there are a few Compiz features that are genuinely useful and I use all the time. The “enhanced zoom” feature, for instance, is a perfect way to blow up a youtube video without making it full-screen.

Every now and then I see a screencast from a GNOME developer working on a little pet project, and some of those demos have been amazing. Whenever I go to the GNOME Summit in Boston, there’s always some guy with his laptop, and he says, “take a look at this –” and proceeds to blow everyone away with some awesome thing he’s been working on. It’s rare, though, for those hacks to escape from that single laptop onto anyone else’s.

GNOME needs a Project Sandbox — an official, gory-edge (it’s past “bleeding”), parallel installable set of libraries and programs (“Toys,” perhaps? ((The exact terminology isn’t important, I’m just keeping the Sandbox metaphor going))) with all of the crazy hacks developers have been trying. It should be housed in a distributed SCM, so developers can push and pull from each other, mashing features and screwing around with GTK “3”, Clutter, Pyro, and whatever other toys people come up with.

The Sandbox should have two rules:

  1. No Kicking Sand ((alternate title: “No Pooping in the Sandbox”)) — ie, no stop energy. Anything goes, no matter how hacky. Developers should be able to prototype ideas quickly, no matter how cracked up they may be. The good ideas will stick and can be re-written cleanly. Nobody, not Apple, not Microsoft, not Nokia, knows what will really be useful in the future with multi-tap and 3D. Apple has a head start, but that doesn’t mean they have all the answers.
  2. Anyone Can Play: If I pull from your tree, I should be able to build and install what you’ve made. It does no good to have a toy if it only works in your corner of the sandbox. There might be some cases where a feature requires a certain video card, but developers should make a good faith effort to make code build and install on systems other than their own. Setting up a development environment is Hard, but I wouldn’t care if I needed 4 hacked copies of GTK each with different .so-names. Disk space is cheap! Computers are fast!

GNOME has done a good job of reining in developer craziness and promoting consistency and uniformity across the desktop. That was good, and was necessary while the desktop was maturing. Now it’s mature, and those reins need to be lifted, or at least relaxed. The stable desktop can plod forward steadily, but developers need a place to relax, rip off every feature from the iphone and Vista, and more importantly make that code public without fear of attack. What’s worse than a flame on Planet Gnome in response to a crazy feature? The feature that doesn’t get written for fear of being flamed.

Converting Varicam 48fps to 24p

Warning, technical video post production post ahead.

A few weeks back we needed to shoot some greenscreen for a show that is being delivered at 23.98fps (aka “24p”). I’d had problems pulling a key in the past with motion blur at that slow framerate (I prefer 30 for TV work), so I suggested we increase the shutter speed in the camera. The DP seemed more comfortable adjusting the framerate, so he suggested we shoot it at 48 and only use every other frame of the footage. I figured I could write a script to do the conversion later.

We shot the footage, and the next week I sat down to write a python program to convert the material to 24 fps. This could have probably been done with final cut or something, but I don’t know that program so I did it the easy way: play around with the frames themselves using an image sequence (ie, a big folder of numbered tif files, each of which represents a frame of footage).

Normally this would be easy, just take every odd-numbered image. But in this case, although we shot at 48fps, the footage is at 60fps. Why? Varicam works like this: the front of the camera (the shutter and CCD) shoot whatever framerate you choose, but the tape recorder in the camera always records 60fps. Even if you shoot 6fps, the varicam writes 60 frames to the tape — 10 of frame 1, 10 of frame 2, etc. So when we shoot 48fps, there are 60 frames per second on the tape, 48 of which are unique and 12 of which are duplicates.

If I am going to convert the footage to 24p, I need to first remove the duplicate frames (60 -> 48 fps), then remove every other frame (48 -> 24). By analyzing the footage frame by frame, I determined that when the varicam shoots 48fps, it uses the following pattern:

0000100001000010001

Where 0 represents a unique frame, and 1 represents a duplicated frame (to fill out to 60fps). This pattern is repeated over and over again throughout the footage. (I only just noticed that the pattern is 19 frames long, not 20 like I’d expect, but looking at the footage that’s what it is.)

My python program goes through an image sequence, finds the pattern of duplicate frames, and then copies every other file that is not a duplicate to a new folder as a new image sequence. It makes the following assumptions: the files are .tif files, and “duplicate frame” means “exactly the same size” (not a bad assumption with digital media and tifs). It’s a little hacky, but looking at the resulting 24fps image sequences I don’t see any stutter or dropped frames.

There are some basic checks in the code so it hopefully won’t overwrite precious data, but I make no guarantees.

Code: 48-to-24.py

Grading a short film


I had the good fortune of being able to grade (color-correct) a graduate student’s thesis project last weekend. It’s called Mel’s Hole, dir. Kenji Miwa. It was my first narrative project, and my first using Apple’s Color. I usually do documentary work, where the highest priority is to make the footage look “good” and consistent. Also, I’m used to the Avid color corrector, which is not very good for matted secondary color corrections (“brighten his face here”) so it would have been hard to do the sort of aggressive grading that the director wanted.

He’s given me permission to show some before-and-after shots from the film, showing off some of the more fun corrections I got to do. Mouse over the images to see the uncorrected versions. (Shot on a Panasonic HVX200 with a lens adapter for low depth-of-field.)

(Note, these images sometimes appear much too bright on a mac. Set your monitor’s gamma to PC / video standard (2.2) to see the night-time shots correctly.)



The above shot represents the basic look for the film, which is a desaturated “bleach bypass” look. It’s high contrast, with substantial crushing of the blacks and whites. In this shot, we had to knock down the colors of the blanket, which was still too saturated even after we applied the look.



In this scene, the character walks into the woods, which were supposed to be dark and foreboding. By really crushing the blacks we were able to make the woods look deeper and more mysterious. This darkening caused the character to be somewhat lost in the busy-ness of the image, so I put a small tracked oval (the shot pans up) over the character to draw attention to him.



This scene takes place in the middle of the night, and I was instructed to make it very very dark, with a silvery-blue cast. Although the lefthand venetian blind did not have any light behind it, I was able to put one in, which serves to illuminate the character’s face (even though a light back there would really just silhouette him). There are still some bright highlights visible in the blinds, but I wasn’t able to get rid of them.



This shot was actually a last-minute idea. It is paired with another night-time shot, so we decided this shot should also be in night-time. I was able to do a good day-for-night, including drawing in the spill of the light at the bottom of the stairs.

I had a lot of fun doing this grade, and I really liked Apple Color — which makes sense, I doubt they would have bought a company that made a bad color correction program. I do have to say that the keyframing in that program absolutely blows, and the tracker isn’t great either. It also crashed immediately after finishing a render once. But on the whole, it was good at disappearing and letting me work.

The director and DP were great too. We hadn’t really done a lot with aggressive grading before, but once they saw what was possible they were able to direct me better and make requests that were creative but also doable.

World premiere is on May 2nd. The details are on Facebook.

(ps, just shoot me an email if you want me to grade your film — the first job is free!)

Starting a user-controlled streaming radio station

The dream:
I want a streaming radio station for my company that itunes can tune in to, where anyone at work can upload tracks and add them to the playlist.

The ingredients:

Explanation:
MusicPD is an awesome music server that I’ve talked about before, and it would be perfect for setting up my little radio station. Users would be able to add files to the playlist from any browser, and mpd now supports shoutcast, which should make turning it into a streaming station easy.

However, right now mpd only supports shoutcast streaming in OGG Vorbis format, and iTunes can’t play OGG. Since everyone in the office is on iTunes, I need some way of getting mpd to spit out mp3 to icecast.

Enter JACK! Jack is a “pro-audio” subsystem for connecting applications, kind of like shell pipes for audio. It can be kind of daunting to work with, but in this case it’s best to just think of it as glue connecting one program with another.

For instance, mpd has a JACK output plugin, and there’s another program, Darkice, that has a JACK input and sends audio to icecast. I can link mpd and darkice with jack, and that should solve my mp3 problem.

So, to review:
mpd (links via JACK to) darkice (which encodes mp3s for) icecast (which streams to) itunes

The Wrinkle:
Make it work on a mac server. At work we don’t have a recent machine running linux that’s up all the time and that isn’t mission-critical, so I have to make due with one of the mac workstations. Luckily macports will do a lot of the heavy lifting.

A Note:
This took a lot of building, installing, and configuration that I don’t have space to cover here. I’m trying to record everything I did that was abnormal or not obvious. If you’re comfortable installing software on unix and you know how to configure a program based on the examples and READMEs, there should be enough information here to reproduce my work.

Getting the software:

  1. Install macports on the mac
  2. Install the following ports: apache2 faac faad2 flac id3lib mad lame libid3tag libvorbis speex
  3. Install this port: php5 +apache2 +macosx +pear
  4. Install the following by source: curl, jack (do not use “jackosx”, build it from source), icecast, darkice, mpd
  5. Apply my patch to mpd/src/audioOutputs/audioOutput_jack.c to make it work on mac osx.

Details:

JACK:
Built with these options: ./configure --prefix=/opt/local --with-default-tmpdir=/tmp
I just start JACK with jackd -d dummy -r 44100

If I don’t specify a default tmpdir, jack crashes complaining it can’t stat /dev/shm. I don’t need to use a real audio device in jack because I’m not outputting the audio to the speakers.

Icecast:
built with ./configure --prefix=/opt/local
No special configuration needed other than what’s specific to my site.

Darkice:
Built with ./configure --prefix=/opt/local/ --with-lame-prefix=/opt/local/ --with-vorbis-prefix=/opt/local/ --with-faac-prefix=/opt/local/ --with-jack-prefix=/opt/local/
I probably didn’t need all those options, but heck, it works.
I set up the configuration file so that the audio device is jack (no quotes).

MusicPD:
built with ./configure --prefix=/opt/local --with-faad=/opt/local --with-libFLAC=/opt/local --with-mad=/opt/local --with-libvorbis=/opt/local/ --with-id3tag=/opt/local
I set up the configuration file to use jack as a device. I need to tell mpd to connect to darkice when it starts up, but darkice picks its port names based on its pid, so it’s always different. What I did is create mpd.conf.in, with this as the device config:

audio_output {
type "jack"
name "MPDJack"
ports "darkice-REPLACEME:left,darkice-REPLACEME:right"
}

Then I run a script to actually start mpd:
#!/bin/bash
pid=`jack_lsp | grep darkice | cut -d '-' -f 2 | cut -d ':' -f 1 | uniq`
cat mpd.conf.in | eval sed -e 's/REPLACEME/$pid/g;' > mpd.conf
mpd ./mpd.conf

Putting it all together:
Once all of those pieces are in place, I can start the radio station by starting the various programs in this order:

  • Jack first,
  • then icecast,
  • then darkice, (-v 10 for debugging)
  • and finally mpd. (--verbose --stdout --no-daemon for debugging)

That’s it. I can control the mpd server and it plays music. If I run jack_lsp -c -p I can see mpd is connected to darkice. I can tune in to the icecast stream from itunes and hear it. With simple mac file sharing, users can connect to the server and drop music in the database. Then mpd can either rescan every hour or so with cron, or when the user clicks “update db” in pitchfork.

Setting up pitchfork is left as an exercise for the reader.

Linux Tip: How to clear some room

I’ve been running out of disk space on my laptop. Multiple development environments, vmware images, my music collection, and especially my camera RAW originals take a heavy toll. Most data on a computer is already highly-compressed, but the whole idea behind RAW files is that they are not compressed — they represent the original image sensor data. So, here’s a one-liner I used for freeing up some disk space. (I have an Olympus camera, whose raw file format has the ORF extension.)

find ~/Documents/images/photos/raw -name "*.ORF" | xargs -n 1 bzip2

Just using standard bzip compression reduces each file by about 50%. And, since I had 10 gigs of raw files, that translates into 5 gigs of space I’ve freed up.

PenguinTV’s third platform: maemo

This weekend I hosted a little maemo hackfest with my friend Mike with my goal being to get PenguinTV to launch in the maemo development environment. This meant building python support for mozilla embedding, which was hard. I’m used to checking out code and then building it, but the process of building the microb-engine packages was incredibly obtuse, and had no documentation. But I tried one thing, then another, then another, and finally it worked.

Once the gtkmozembed python module was finally built, porting the application was relatively easy. GTK is really wonderful because you can extract widgets, move them around, and plug them back in where you need them. I can take a toolbar that normally goes here and plug it in there, I can take a preferences window and throw in a scroll bar, and I can turn off that menu item or that button if it doesn’t apply. Sprinkle “if RUNNING_HILDON:” checks where necessary, and:

PenguinTV on Maemo

I always have to do some extra gymnastics because my program requires a lot of special startup initialization and shutdown finalization, but other than that the port was nice and straight-forward.

I’m now working on using all of the maemo-specific features like hardware buttons, network connection, and system messages. It should just be a matter of hooking up to some basic signals to existing functions.

My only big concern about is the “state-saving” feature that allow hildon to completely shut down a program when the user switches away, then restored when the user switches back. Because of all of my threads, database connections, and caches, my startup and shutdown are very slow. I’m not sure if it’s going to be possible to save fast enough. I also don’t know if it’s possible to save the scroll position of a mozilla widget, because when the user switches back they’ll expect to have everything scrolled how they left it. Since gtkmozembed is a black box, it’s just going to start scrolled at the top, which is not acceptable.

Along with olpc and the desktop, PenguinTV now runs on three different platforms using a single code-base.

Back in Business

This Monday morning, I started up my laptop to check the weather report like usual. Although it booted, it quickly crashed. I tried again, and it froze again. I tried memtest86, and that froze too. And so began My Time Without Half My Brain. Something in my thinkpad’s motherboard fried so I had to send it in to IBM to get repaired. Although they assured me the average turnaround is 5-7 days, I knew from experience that, with shipping time, this would be more like two weeks.

This really sucked because after 6 straight weeks of finishing shows and this week being my last, 20th show of the year, I was planning to take much-needed next week off and do some hacking on the OLPC version of PenguinTV. Without my laptop it would be next to impossible to find another linux machine and get it to the point where I’d feed productive on it.

On Tuesday, I got the ez-ship box from DHL and shipped off the poor little laptop Taking care, of course, to remove my not-officially-supported-harddrive with not-officially-supported-OS and insert my original-low-capacity harddrive for them to reimage. DHL: you rock for calling ahead and confirming my address which IBM had wrong.

With my minidock naked and empty, I salvaged some tomboy notes and bookmarks from my backup drive and uploaded them to a VMWare Fusion instance on the mac on my desk By the way, VMWare Fusion is totally awesome and I recommend it highly. I had a fully-functional Ubuntu desktop in the time it takes to download the pre-made image. You can even drag and drop files from the host OS to the guest OS desktop!. This gave me a workable solution for checking email and such, but I still felt slow and stupid without my various array of calendars, bookmarks, and email archives.

Today, I was presented with a package from DHL. My laptop is completely fixed and working again. This is the fastest turnaround time I’ve ever experienced for a laptop repair, and after popping back in my own harddrive everything is just as I left it. And because I got a three-year warranty, it was free.

Now, my hacking plans for next week are back on, and even better I also got an email that I’ve been accepted into the Nokia n810 developer program, so I’m soon going to have two separate mobile platforms for which to develop PenguinTV.

IBM / lenovo / whoever it was: you rock. I’m sorry I ever thought about getting my next laptop from HP.

iphone analogies

Mark Pilgrim has written a now-dugg post on the iphone, and how people should learn to stop buying products from Apple in the hopes that they will get a clue and truly open them up.

Although the blog post is nice, I prefer the simplicity of his delicious comment:
“Doctor, it hurts when I buy Apple products.”

The punchline, for those that don’t recognize the joke: “Then don’t do that.”

Or take the digg comments when Steve Jobs took X+200$ from people for an iphone, then gave them 100$ back so they can spend it at the Apple store. Commenters were falling over themselves praising Apple for doing the right thing, even though customers have still spent X+200$ at Apple, and the only thing Apple loses is the equivalent of a few months iphone subscription revenue. My response: “Thank you sir may I have another!”

Similarly, I often describe buying a Mac as a deal with the devil. You get the best-available computing experience money can buy, but you’ll have to put up with Steven P. Jobs’ odd whims — some of which cost money18-month 129$ charges for OS upgrades, plus money for simple utilities that should be free, others of which just make you mad“Why do I have to buy the expensive laptop to get it in black?”, some of which screw you over“you mean this device that worked with my ipod two days ago doesn’t work now because it doesn’t have a chip from apple?”.

But I think the most apt analogy for Apple and its users is the abusive relationship. Apple will slap you around and beat you up, but it’ll always come back with a bouquet of flowers and promise that this time things’ll be different.

Update:

Now apparently Apple is saying “I’ll buy you a bouquet of flowers, and you can choose the flowers, but only if I think they go with the drapes.”