Tag Archives: puredata

Visualizations for new Display

by · · 6 comments

IMG_1463Over the last couple months I’ve had the chance to develop a series of “paintings” for the fiberoptic tapestry, a new display system developed by artists Ligorano/Reese. It is essentially a hand-woven canvas of optic fibers, each illuminated by an electronically-controlled LED. The result is incredibly beautiful and expressive, producing a painterly effect. I was mesmerized by the gently-glowing canvases the first time I saw them and was very excited to get the chance to create content for them.

My work has been developing visualizations that utilize the new physical display in different ways. The first visualizes air traffic over four major airports in the US, drawing moving lines to indicate aircraft taking off and landing. The second, entitled Order/Disorder, displayed below aggregates a number of sources for natural disaster information and draws the disasters as “tears in the fabric” of the world. Lastly, I am working to create software that reacts to the environment and responds by flashing a series of animations developed by Marshall Reese.

A program in Processing communicates with the hardware and serves as a hub accepting drawing commands from other software via OSC. A number of scripts in Python perform data aggregation, scraping, parsing and animation for the flight tracker. A patch in Pd performs sound analysis and drives the canvases through OSC.

“Order/Disorder is a software visualization of destructive and restorative forces in the world. The software runs on an electronically-controlled tapestry of woven fiberoptic threads created by New York-based artists Ligorano/Reese. Two computer programs, named order and disorder, modify the tapestry throughout the day in response to natural and man-caused events such as earthquakes, biohazards, and aircraft accidents. Order seeks to weave a rainbow-gradated pattern representing peace and wholeness while Disorder seeks to destroy and unravel the tapestry by “tearing” the fabric and weaving in aberrant threads.”

Reverse Engineering the Speaking Piano

by ·

I was really intrigued by Peter Ablinger’s Speaking Piano (officially titled Quadraturen, auf Deutsch)–a system that takes human speech and translates it to a sequence of notes to be played on a piano by a bunch of solenoids, or “mechanical fingers.”

Since I’ve been learning Puredata, I thought it would be a fun exercise to attempt to recreate the software Ablinger wrote to translate speech to midi notes. The secondary purpose was to turn my oft-idle digital piano into an interactive sound piece, translating sound from another part of the house into music downstairs. The result isn’t perfect, but I think it still achieves the same ambiguous result where you are able to hear the voice once you see the transcript. The biggest difference is obviously that I’m using a digital piano, not a mechanically-actuated analog piano. However, the Roland has a fairly sophisticated physical model with things like dampening and string resonance, so it’s better than nothing.

Below are the two components to the software. Clicking the image will link to the pd file if you’d like to experiment yourself. You can either load in a pre-recorded wave file and play it back, or set the gain to the adc~ to 1 and use a microphone to drive it in real time (although I set up a delay of about 3 seconds so I could evaluate the results without hearing my own voice). delread~ passes the data into fiddle~ which does all the hard work of Fourier analysis. A metronome set to 15 ms samples the outputs of the individual sine components and creates midi notes. The blocks that create the actual notes are partial_key.  The highest key on the piano is midi 108, which corresponds to 4186 Hz, so I added a low pass filter to remove frequencies that couldn’t be reproduced.

talking_piano.pd

partial_key.pd creates the midi notes which are sent to the piano. The signal makenote_b is received from the metronome, which causes the note to be made. Additionally, no note is sent if the midi key number is higher than 108, the limit of my piano, or if the amplitude is too small (< 0.01).

partial_key.pd

Here is a sample of the result, speaking the following: “these are very profound words, which is why they are being spoken by a piano. I hope you are forever moved by these profound words.”

I’d love any feedback from Puredata or DSP gurus on how the software could be improved. I’m not quite sure what sorts of additional analysis and synthesis steps are being taken in the Quadraturen software as it is not available.