— M i r r o r F u g u e —

Playing the piano as a child, I was always entranced by the reflection of my hands on the lacquered surface behind the keys. A few years ago, I had a thought of how lovely it would be to play a duet with this reflection—a pianist from afar or from another time, illustrious virtuoso or simply my own childhood self. By augmenting a Yamaha Disklavier player piano with life-sized projections, MirrorFugue evokes the illusion of the a pianist playing the physically moving keys.

For most of human history, music could only have been experienced live, channeled through the bodies of performers, felt in the bodies of audiences. Due to the proliferation of audio recording, today's music comes most often as disembodied sound severed from physical origins. We experience music more through devices than live, the human dimension increasingly filtered away. When learning, we have become obsessed with hitting the right notes, the expressive dimensions of playing music falling by the wayside.

Musical expression is rooted in the human body. The physicality of a musical performance bears undeniable potency and poignance—as audience, we feel the energy in the performer's every breath and gesture. When learning to play, knowledge of the body is indispensable for both expression and technique. MirrorFugue returns the human dimension in how music is captured and re-experienced, enabling rich musical experiences for both learning and enjoyment—whether watching a master play right in your living room or playing a duet with your favorite improviser (and maybe learn a riff or two).

Alisa Ishii playing with her digital double

Myself in a mirror

JJ Abrams with MirrorFugue (photo by Joi Ito)

Talks and Performances
TEDxBoston, June 2013
Aspen Ideas Festival, 6.13
BIF9, 9.13
Intersect by Lexus Concert, 9.13

P3+ at The Historic New Orleans Collection
New Interfaces for Musical Expression 2014

These works mainly explore MirrorFugue's potential as a tool for piano pedagogy, especially over distance. By rendering the physical movements of the pianist, MirrorFugue can aid the understanding of both technique and expression, especially breathing, phrasing and projecting character.

NIME 2013 full paper
TEI 2011 full paper
ITS 2012 demo
CHI 2011 poster

Press and Awards
BBC (7.14)
CreativeApplications.net (1.9.14)
NHK "Number One On This Earth" (12.6.12)
Schnitzer Prize in the Visual Arts at MIT, Honorable Mention

Thanks to
Hiroshi Ishii - advisor
Paula Aguilera and Jonathan Williams - consultation on video production
Lauren Gust - fabrication of a prototype
Donal Fox - insights on piano performance
Marvin Minsky, Alisa Ishii, Donal Fox, Ryuichi Sakamoto, Nick Sanders, Vijay Iyer - recordings of performances