How a group of visitors can interact, individually and as a group, with an immersive room. Our interactive scenario is described, along with part of our creative process.
A experience with collective interaction
As a follow-up to the Immersive Vitamin project, we at the Lab148 Cooperative have teamed up with OASIS immersion, la Piscine and Mon Relief to develop an interactive experience for a group of visitors. This experience is designed to support the group’s sense of connection and shared purpose.
But let’s talk about OASIS immersion first: this is a permanent immersive space where inspiring and resolutely optimistic exhibits are presented, thanks to state-of-the-art equipment, operated through industry best-practice methods. A wonderful playground!
We developed this experience in the largest room, called “Panorama”. The large room is equipped with a live tracking system that allows us to locate group members, and thus build an interactive grammar at the service of the group experience.
Interactive storytelling
The principle is simple: the group of participants enters a room whose walls are hidden by large slabs, revealing a landscape behind them.
The appearance of red dots on the ground marks the acknowledgement of the presence of one or more people, and if two people are relatively close, then a connexion is activated between them and indicated with an animated line drawn between them.
Participants have to produce the longest series of connections to make the slabs fall and discover the whole landscape.
In the following video shows the complete sequence revealing the 4 walls one after the other.
Hiding physical space
For this project, we experimented with a method for limiting the perception of real physical space, in favor of virtual space. A well-known trick is to adjust the position of the virtual space rendering viewpoint (which can be likened to the focal point) to the user’s position. This technique works particularly well when only one user is present in the space, and is someting implemented in CAVE.
In our case, however, we consider a group of users. It is not possible to adjust the point of view for each of them, but we can come close. To stack the odds in our favor, the interaction scenario favors proximity between users. Under normal conditions of use, users should not be scattered all over the physical space.
Finally, by using the average position of users as a focal point, we can adjust the projection to reduce the distortion of virtual space by physical space. While this solution isn’t perfect, since some distortion remains, in terms of results it’s pretty close to what you’d get in an immersive space where the focal point is fixed, as in a dome, for example.
Sound production, composition of the “lent” piece
For the sound part, the “lent” piece was specially composed to make use of the 30+ speakers, their spacing, and to suit the positive, tranquil atmosphere of the experience.
The initial idea was to generate an arbitrary number of interpretations following a predetermined musical sequence, by imitation.
The aim is to have a wide variation of sequences, which continue to work together in the same way as the initial musical idea.
The pipeline above illustrates the steps in the composition process. Initially, instrument extracts are selected to give the piece a sound aesthetic (“timbre” stems). Next, an existing assembly of instruments whose musical cohesion will be reused (“structure and harmony” stems). For the generation of variations, the techniques used retain the initial temporal positioning and harmonies, thus providing sufficient material to produce a spatialized piece.
This audio file presents several stages of the pipeline:
- 0:00 extract of an instrument from the stems Audio (timbre)
- 0:07 extract of the initial audio piece stems Audio (Structure, harmony)
- 0:25 new piece generated by pattern association
- 0:43 new piece augmented by layers of synthesis
A work in progress
The current version of the experiment is a proof of concept that required several sessions of experimentation and fine-tuning. In terms of collective interactivity, the large OASIS immersion room has demonstrated its ability to support real-time displays on the walls and floor, enabling the production of fully interactive content. In terms of sound, beyond the impressive acoustic treatment for this room, the precision and flexibility of the sound system enabled us to produce a rich and appropriate spatialized soundtrack.