Imagined Systems is an interactive sound installation creating a virtual space filled with imaginary machines and objects. As visitors passed under spotlights in particular ways, for example in a counter clockwise motion for a large turbine wheel, the virtual devices would respond with appropriate sound. Drawing on everyday sounds collected over the years—a potato hissing in the microwave, unintelligible radio babble from a cab in Queens, an alarm at the airport, oil dripping in the pan of a cooling engine—the sounds are remixed into a scene that cannot be placed, perhaps vaguely reminiscent of Pittsburgh’s industrial past.

The work was created for 2011 Pittsburgh Biennial at the Pittsburgh Center for the Arts. The visitor is presented with a darkened, empty room illuminated by three spots of light on the floor. Hidden 3D cameras fed video to computers running custom computer vision, physics, and sound synthesis programs driving a four-channel speaker system.

Below is a brief sample of the audio from one point during the piece’s lifetime.
Imagined Systems by ukle

I was recently featured on Berlin Reboot FM discussing some of my sound work, the episode is embedded above for listening. The episode’s page is here.

Below is a video clip showing the setup and sound produced.

Sonification of Ants from Luke Loeffler on Vimeo.

The first clip takes us to the campus of Carnegie Mellon University this winter. While in the arts building I heard a the beautiful and haunting sound of a girl practicing the marimbas.  Contrast the solitary sound with he cacophony of many music practice rooms mixing.  Later in the week, some dispatcher on a radio in a cab ride in Queens was noteworthy for its strangely rhythmic musicality.  As if LaGuardia wasn’t bad enough already, this 20-minute alarm at the airport made me vow to go through JFK next time.

Back home, leaving the garage, I noticed the beautiful clicks and pops of an engine cooling down. While walking to work: pigeons under the bridge. The terrifying creaky stairs at the old warehouse in which my studio is located. It’s just a matter of time before they break.  Next, the sound of relays chattering in a little mechanical feedback noisemaker a made.  Lastly, I will leave you with the sound of the death throes of an exhaust fan in the bathroom of the warehouse. It always manages to screech like some kind of killer primate when you’re least expecting it.

Mark making can be broken into three categories: addition of material, removal of material, and the modification of a material (although there is a fair amount of overlap between the categories).  Examples of addition were to clamp a bead onto a passing string, to add a blob of glue, or to add a drop of water. Removal of material could mean cutting away, notching, or burning a substrate as it passes. Modification could be knotting the material as it passes, staining it, raising the temperature, etc.

Most media strive to be precise and permanent. By soaking portions of a passing rope with saline, these regions can be detected as they pass “listeners” further down the line. Due to the capillarity of the rope, however, the saline spreads out in several directions which causes slight conductivity and more gently ramping (as opposed to digital) signal. Due to evaporation, the signals get weaker and weaker as they progress through the system yielding unexpected results. Looping everything back around will allow one continuous rope to be used.

Additional possible methods could use a heating element to apply varying amounts of heat to a passing wire. As the heat spreads, the crispness of the signal is diminished and bleeds into adjoining signals. As the heat is transferred to the environment, the overall signal diminishes and returns the wire to its original state.

Saline on Jute by ukle

The Sound of Vapor from Luke Loeffler on Vimeo.

Since I’ve been learning Puredata, I thought it would be a fun exercise to attempt to recreate the software Ablinger wrote to translate speech to midi notes. The secondary purpose was to turn my oft-idle digital piano into an interactive sound piece, translating sound from another part of the house into music downstairs. The result isn’t perfect, but I think it still achieves the same ambiguous result where you are able to hear the voice once you see the transcript. The biggest difference is obviously that I’m using a digital piano, not a mechanically-actuated analog piano. However, the Roland has a fairly sophisticated physical model with things like dampening and string resonance, so it’s better than nothing.

Below are the two components to the software. Clicking the image will link to the pd file if you’d like to experiment yourself. You can either load in a pre-recorded wave file and play it back, or set the gain to the adc~ to 1 and use a microphone to drive it in real time (although I set up a delay of about 3 seconds so I could evaluate the results without hearing my own voice). delread~ passes the data into fiddle~ which does all the hard work of Fourier analysis. A metronome set to 15 ms samples the outputs of the individual sine components and creates midi notes. The blocks that create the actual notes are partial_key.  The highest key on the piano is midi 108, which corresponds to 4186 Hz, so I added a low pass filter to remove frequencies that couldn’t be reproduced.

partial_key.pd creates the midi notes which are sent to the piano. The signal makenote_b is received from the metronome, which causes the note to be made. Additionally, no note is sent if the midi key number is higher than 108, the limit of my piano, or if the amplitude is too small (< 0.01).

Here is a sample of the result, speaking the following: “these are very profound words, which is why they are being spoken by a piano. I hope you are forever moved by these profound words.”

I’d love any feedback from Puredata or DSP gurus on how the software could be improved. I’m not quite sure what sorts of additional analysis and synthesis steps are being taken in the Quadraturen software as it is not available.

This is how I’d like to imagine the first Christmas–even the wind crying out. And doing so subtly, not unlike the whispering symbolic language of babies and cattle stalls.

The idea came about from the desire to manipulate various parts of the built environment to create music as the wind is blowing. Maybe change the orientation of a sheet of metal, the size of some opening, etc. With a feedback loop, the system could progress through a melody once it detected that the desired notes of the sequence had been played. Although a composer could control the content, the weather would determine the tempo. A listener may have to wait weeks or months for the piece to complete without sufficient wind.

In the meantime, I created the above synthetic version of what I imagine it could sound like (though the song above would probably have to be recorded somewhere in Antarctica). The synthesized version uses a custom patch written in Pure Data to shape the frequency of white noise to follow a midi file of choice.

Over the years I’ve played with a number of tools for audio processing: Matlab, jMusic, a Java library for algorithmic composition, Nyquist, a Lisp-based synthesis/analysis environment, Beads, another Java library for synthesis and analysis, and Supercollider, another synth/analysis environment with smalltalk-like syntax. All of these are powerful tools, but aren’t as engaging in terms of interactivity. Having been forced to use LabVIEW in the past, another dataflow language, I was initially reluctant to pick up another, but for audio work, it’s been great. It is so easy to try new ideas without any need to recompile. It’s a lot like playing with a running circuit.

So far I’ve used to to analyze sound and control some lighting panels to create a reactive environment, synthesize tones for my invisible chimes project, and do some other synth experiments. This brief subtractive synth test uses filters to shape pink noise into hazy tones forming a chord. synth2 tinkers with sample playback and ring modulation. Next up, granular synthesis to build some instrumental Christmas music?

Other Useful Audio Software

Jack is a great tool for routing inputs and outputs on your system. It has made it really simple, for example, to send the output of iTunes to Pure Data, which allows me to sample chunks, process them, and mix it back into songs playing. I made a small program that samples chunks of the last song and then injects them into the new song when it detects beats. It also supports plugins, so you could use Pure Data as a signal processor for other programs like Logic.

Wiretap Studio is really useful for capturing any sound source on your system, doing basic waveform editing, fades, effects, and exporting to any other sound file format.

RjDj for the iPhone is a program that lets you download (and create, using Pd) “scenes” that generate music or process environmental sound and play it through the headphones. For example, one might identify that fan humming along at 300 Hz and re-inject overtones to change its timbre.

The applet has been posted along with the source, although you may have to run it locally in processing to work properly.  You will need a video camera attached to your computer.  Listen to a sound clip from the synth.

An experimental, interactive audio synthesizer in which independent generators combine to create an ambient soundscape as they try to reach a state of equilibrium.

Click to listen to a sample clip, play with the applet, or view the source.