Tag Archives: music

Reverse Engineering the Speaking Piano

by ·

I was really intrigued by Peter Ablinger’s Speaking Piano (officially titled Quadraturen, auf Deutsch)–a system that takes human speech and translates it to a sequence of notes to be played on a piano by a bunch of solenoids, or “mechanical fingers.”

Since I’ve been learning Puredata, I thought it would be a fun exercise to attempt to recreate the software Ablinger wrote to translate speech to midi notes. The secondary purpose was to turn my oft-idle digital piano into an interactive sound piece, translating sound from another part of the house into music downstairs. The result isn’t perfect, but I think it still achieves the same ambiguous result where you are able to hear the voice once you see the transcript. The biggest difference is obviously that I’m using a digital piano, not a mechanically-actuated analog piano. However, the Roland has a fairly sophisticated physical model with things like dampening and string resonance, so it’s better than nothing.

Below are the two components to the software. Clicking the image will link to the pd file if you’d like to experiment yourself. You can either load in a pre-recorded wave file and play it back, or set the gain to the adc~ to 1 and use a microphone to drive it in real time (although I set up a delay of about 3 seconds so I could evaluate the results without hearing my own voice). delread~ passes the data into fiddle~ which does all the hard work of Fourier analysis. A metronome set to 15 ms samples the outputs of the individual sine components and creates midi notes. The blocks that create the actual notes are partial_key.  The highest key on the piano is midi 108, which corresponds to 4186 Hz, so I added a low pass filter to remove frequencies that couldn’t be reproduced.

talking_piano.pd

partial_key.pd creates the midi notes which are sent to the piano. The signal makenote_b is received from the metronome, which causes the note to be made. Additionally, no note is sent if the midi key number is higher than 108, the limit of my piano, or if the amplitude is too small (< 0.01).

partial_key.pd

Here is a sample of the result, speaking the following: “these are very profound words, which is why they are being spoken by a piano. I hope you are forever moved by these profound words.”

I’d love any feedback from Puredata or DSP gurus on how the software could be improved. I’m not quite sure what sorts of additional analysis and synthesis steps are being taken in the Quadraturen software as it is not available.

Wind Carol

by · · 1 comment

It’s that time of year again. Christmas is coming, snow is falling outside my window, and the wind is whistling.  Maybe it’s late, or my mind is playing tricks on me, but the wind seems a bit more coherent than usual… Musical, even. Creepy.

This is how I’d like to imagine the first Christmas–even the wind crying out. And doing so subtly, not unlike the whispering symbolic language of babies and cattle stalls.

The idea came about from the desire to manipulate various parts of the built environment to create music as the wind is blowing. Maybe change the orientation of a sheet of metal, the size of some opening, etc. With a feedback loop, the system could progress through a melody once it detected that the desired notes of the sequence had been played. Although a composer could control the content, the weather would determine the tempo. A listener may have to wait weeks or months for the piece to complete without sufficient wind.

In the meantime, I created the above synthetic version of what I imagine it could sound like (though the song above would probably have to be recorded somewhere in Antarctica). The synthesized version uses a custom patch written in Pure Data to shape the frequency of white noise to follow a midi file of choice.