Death and the Powers: A Robot Pageant

Technology  and Innovation

For Death and the Powers, a team of faculty, staff, and graduate and undergraduate students at the MIT Media Lab has brought a host of innovative technologies to the stage. From robots to visuals to sound-producing Hyperinstruments like the giant Chandelier, more than 40 computers are required to run the production, all backed by extensive wired and wireless networks. These computers run a broad range of distributed control systems that were developed for the production, in which each component can share information with any other in order to create a synchronized and unified presence of Simon in The System.

The Chorus of Operabots and three large bookshelf periaktoi are controlled centrally using software developed specifically for choreographic robots onstage. This software includes a 3D visualization for monitoring and authoring the animation of robotic movement and lighting. If need be, puppeteers above the stage can assume manual control of any parameters of a robot using a typical video game controller. An absolute position tracking system monitors the location of robots and actors onstage to help the robots navigate, as well as affect sound and visuals.

After Simon Powers enters The System, the singer portraying him exits the stage, though he continues to sing and act as if he were onstage. In a new technique called Disembodied Performance, gestural and physiological sensors, as well as voiceanalysis, capture the singer’s offstage performance, which is then used to generate in real time the visual representation of Simon Powers in the bookshelf displays and other aspects of the production. Mapping software was created that can connect sound, robots, and visuals to the singer’s performance. A custom graphics environment allows these live performance parameters to generate expressive graphic representations of Simon in The System.

Another method of representing The System’s omnipresence is through sound. Over 140 speakers are used to create a unique sonic environment. Two formats of surround sound are used in the production. Wave Field Synthesis uses an array of tiny speakers across the front of the stage to create the impression of a sound emanating from any point on the stage. Ambisonics technology is used to move sound all around the theater. Software and plug-ins for common audio packages were engineered to allow the hundreds of streams of audio to be processed in real time.

All of this technology–although complex–is designed to work invisibly behind the scenes, helping to draw audiences into the unusual, mysterious, and “animate” world of Simon and The System.

by Peter Torpey, Elena Jessop, and Ben Bloomberg

Read More