Gesture is an animation generated by custom video processing software that seeks to unite the movements of dance and drawing by distilling the dancer’s poses into a sequence fluid lines. As a spotlight follows the dancer across the new stage, indications of movement are left behind by soft erased lines.
The music is “Glory” by The Acorn.
If you’re interested in a technical description of the code, read on. The main hurdle to getting this completed was implementing a fast “nearest neighbor” system. From the chroma-key footage (”green screen” as it’s often called), I threshold it (which means turn it to black and white). Next, I perform edge detection which leaves just the outline to the dancer. By altering the edge detector’s “kernel,” a mathematical term for a table of numbers, I can degrade the outline so it is a series of dots rather than one solid line. Next, I go through the image and make note of the locations of all the points. All that’s left to do is to connect them with a continuous line. However, if you simply connect the dots in the order they were found (top to bottom, left to right), you will get a zig-zag line that fills the area of the dancer. The next logical step is to take a starting point and ask for it’s nearest neighbor and connect the two. Then continue the line to the nearest neighbor after that, and so on. Unfortunately, this naive approach does not scale well and takes an enormous amount of time to compute. Enter the KD Tree, a method of storing the points for fast retrieval. I had started to write my own implementation last year and got distracted. Thankfully, I found a simple implementation for Java which worked perfectly. It’s beautiful to see a series of techniques converge to produce something entirely new.
The source code is provided in two files: dancers_outlined.pde and Tree.java for reference. Please be warned–it is very rough and not intended for public consumption, although I tried to comment as much as possible.
