The Virtual Mise-en-Scène
At the core of the Telematic Theater is the Visual Stage, written in Cycling ’74’s Max/MSP/Jitter visual programming environment. It provides the environment for compositing live remote performers, spherical imagery, textured stage platforms, moving images, and lighting into a shared mise-en-scène.
Up to four concurrent performers can be positioned within this world. Their live feeds currently arrive through LiveToAir via NDI, then undergo chroma keying and cropping for clean placement within the scene. External sources can also enter through SDI capture, NDI, and Syphon, extending the stage to additional cameras, computers, and visual systems.
A 24-track mixer shapes the layered composition of performers, video material, platforms, and external sources. Compositing blends establish relationships among layers, while track effects—including color control, duotone, and video granulation—allow the imagery to be processed and transformed. Multichannel audio routing carries sound from video clips into the studio’s mixing and spatialization environments.
The Visual Stage has been implemented as a fully fledged theatrical environment through stage and independent performer lighting, scenic textures, spherical backgrounds, and the arrangement and animation of its performers. A preset system stores configurations for recall and interpolated transition, including performer positions, scenic elements, and lighting for both the overall stage environment and for individual performers. The Preset Grid allows for the composition and sequencing of scenic and performer presets that are repeatable, cueable, and interpolated for transitions.
Choreographing Space
The 3D Panner connects visual staging with acoustic composition. It controls the positions of performers in the virtual environment while communicating their corresponding locations to the spatial audio system. A body’s movement across the stage environment can therefore be heard as well as seen.
Top, front, and split views reveal the spatial relationships among performers along with a virtual camera. The panner’s view can also be rotated to examine the spatial arrangement from different angles, supporting the design of depth, proximity, and perspective.
The Path Creator extends this control into repeatable, designed choreography. Movement paths can be drawn, animated, saved, and recalled. Each of the four performers and the virtual camera can follow an independent path, allowing the composition to unfold through both the movement of its inhabitants and changes in the viewpoint from which they are encountered. This choreographic design synthesizes attributes from film, theater, and video games into a unique online performance technique.
The theatrical environment thus becomes a space in motion: figures approach or recede, relationships shift, and the audience’s perspective is an ever-changing immersive experience in a virtual environment choreographed in space and sound.