http://versatilesolutionsformodernliving.com is an interactive site that showers the entire catalog of IKEA products into a pile, removed from its typical carefully-curated context of showroom, catalog, or home. The user is then free to shape the pile as raw material making tunnels, totems, or any anything imaginable. The construction may then be saved to a gallery which rotates on the website.
A Photoshop special effects filter that secretly uses real smoke and water
The Really Real filter system consists of a custom-written Photoshop filter plugin which sends the image to a remote computer which displays it on its screen, creates a physical, visible effect over the screen, snaps a photo, and sends it back to the user, none the wiser. The three effects demonstrated in the video are ripples, perspective shift, and fog.
Stemming from an interest in replacing computer-generated effects with natural ones, I created an “unnatural natural” system that hides behind the veil of the familiar Photoshop user interface. As the beach ball spins and the user waits for the computer to finish, it is actually triggering physical movement and reactions in a remote location to “compute” the resulting effect.
Brain Blur is an OS X application that runs in the background and adjusts the sharpness of the screen based on the user’s mental state as measured by an Electroencephalogram (EEG) device.
As the users mental activity increases due to computer usage, the screen blurs to make it harder to see and use. By meditating and closing the eyes, alpha waves increase and the screen becomes sharp again. The system creates a frustrating feedback loop that prevents normal computer use.
The documentation video below is a screen recording of me using the program while attempting to create a presentation about Brain Blur.
An invisible machine that can be heard but not seen and touched but not felt
Imagined Systems is an interactive sound installation creating a virtual space filled with imaginary machines and objects. As visitors passed under spotlights in particular ways, for example in a counter clockwise motion for a large turbine wheel, the virtual devices would respond with appropriate sound. Drawing on everyday sounds collected over the years—a potato hissing in the microwave, unintelligible radio babble from a cab in Queens, an alarm at the airport, oil dripping in the pan of a cooling engine—the sounds are remixed into a scene that cannot be placed, perhaps vaguely reminiscent of Pittsburgh’s industrial past.
The work was created for 2011 Pittsburgh Biennial at the Pittsburgh Center for the Arts. The visitor is presented with a darkened, empty room illuminated by three spots of light on the floor. Hidden 3D cameras fed video to computers running custom computer vision, physics, and sound synthesis programs driving a four-channel speaker system.
Below is a brief sample of the audio from one point during the piece’s lifetime. Imagined Systems by ukle
Lately I have been exploring the idea of invisible, intangible interactive systems and how they would be experienced. So much of everyday interaction deals with sight and touch. We rely on visual and tactile feedback to manipulate and understand physical systems. How difficult is it to understand and know a system when these two components are removed leaving only sound? Everyone has had the experience of stumbling through a dark house at night to get a glass of water, but relied primarily on the sense of touch to get by. What happens when Newton’s 3rd law no longer guides us and the glass we grasp doesn’t press back on our hand? The closest experience I can think of is the theremin, although the underlying system is limited in potential complexity.
The first piece I have made to explore this concept features invisible wind chimes. When a person walks through the space occupied by the virtual chimes, they begin to ring. Should the actor stop to explore the source of the sound, he will uncover a fine-grained, predictable, knowable system. Using computer vision techniques to detect movement in a space, real, physical objects are able to interact with virtual, “physical” ones. Although not concrete, they are still physical because a physics simulation ensures the individual chimes still hang behave in ways we know and expect. They hang from the ceiling by strings, collide with each other (causing them to ring), swing and respond to gravity by eventually settling back to rest, and respond to touch by a soft body (muting any ringing).
Invisible Chimes is an intangible, interactive system that prompts us to consider how we acquire knowledge of complex systems from limited experiential data. The chimes, though installed in a space, cannot be seen, and while they can be touched, they cannot be felt. The only indication of their presence is the sound they make. Nevertheless, the system is still very much a “physical” one, in that the chimes are governed by the laws of physics. Although they can be appreciated by simply walking through them, further interaction reveals that the individual chimes can be separated, muted, and lifted, and that they behave in a predictable, understandable way. The inset video reveals the physical model the custom software uses to create the interactive system.
This is a screenshot of one of the resulting paintings from an experiment using trackmate to create a tangible drawing interface. By placing special coded labels on the bottom of miscellaneous objects I had lying around my desk, I was able to assign a different function to each object. For example, a green candle drew green lines and a yellow one drew in yellow. Another one was a “universal” brush that took on the color determined by the rotation of a fourth object. Another object only erased, and one final one controlled the opacity of the brushes by its position.
The trackmate system examines an image (in this case coming from a cheap webcam) and looks for special coded labels. Each label contains a unique ID and also encodes its rotational direction. When trackmate identifies a label, it sends the id, position, and rotation to my software in processing which in turn draws lines.
The screenshot was taking during an interactive session where one person was controlling the color, the other was controlling the opacity, and the third was moving the brush.
The lines in the drawing are broken up due to the low resolution of the camera (this could have possibly been solved with interpolation). I need to calibrate the camera again (and really, buy a cam with a res higher than 640×480), and find a better surface that will filter some of the ‘noise’ from the scene. Still, trackmate behaved remarkably well given the circumstances.
The trackmate website has good tutorials on how to get a system up and running. My source code is provided to help get you started if you want to play around with the system, although I accidentally lost the final revision, so the code provided may not function as expected.
This project aims to create sound from movement. In this processing sketch, the system finds three areas of a particular size and assigns each a continuous tone according to its size associating larger pieces with lower tones. The colors indicate the three sound-generating regions.
The applet has been posted along with the source, although you may have to run it locally in processing to work properly. You will need a video camera attached to your computer. Listen to a sound clip from the synth.
Your Browser does not support iframes, so you cannot view the embedded movie. You may still view a version of the movie without sound.
In this interactive visualization, a musician watches the screen while performing on a keyboard. As notes are played, ‘raindrops’ begin to fall. Louder notes fall faster than softer ones and continue to accelerate as the note is sustained. Eventually, notes will blacken if they have been falling for an extended period, and will slowly dissipate unless cleared away by fresh rain from above.
The real-time feedback causes the musician to alter what is being played in order to generate particular audio and visual outcomes simultaneously.