Although the first Dutch Virtual Reality film Ashes to Ashes premiered last year, now that it has been nominated for a Gouden Kalf, the Dutch equivalent of an Oscar, it is back in the spotlights. Ashes to Ashes was nominated in the category ‘Best Interactive Project’. Big Orange was in charge of sound for this unique film.
The Gouden Kalf nomination is perfect for explaining what makes Virtual Reality (VR) so special, both in visuals and sound. What challenges and opportunities does VR present? A virtual world has depth. Objects and people actually appear close by and far away, at a distance from the viewer. The desire for visual depth has existed for years. Earlier, stereoscopic imaging techniques requiring a 3D Viewmaster provided the solution. With a circular photo card and a handle to switch pictures on the side, viewers saw depth. This was achieved by taking pictures with two cameras that had lenses set at pupil distance. Allowing each eye to see only the single corresponding image creates the illusion of depth.
The same goes for sound. Two ears allow us to hear location and direction, but also distance. We can record that effect, too, by using two microphones to record sound at the same distance as our eardrums are to each other. When the recording is played strictly separated (using headphones, for example), listeners feel they are surrounded by the sound.
It is an incredible experience to reproduce something like this, both recording and listening to it. When recording sound for a 360° film, the challenge is to make sure that no recording equipment or sound technician is in the picture. That means no sound man on the set. Actually, we should return to the age of film making when microphones were hidden in plants or other props on the set. Fortunately, nowadays there are wireless microphones that are easy to conceal. The important thing is to ensure that the sound is effectively ‘captured’ in various ways, for all viewing directions.
The next challenge is editing the sound. How do you know where each actor is standing and the related angle the sound has to come from? Mixing sound wearing a VR headset is impossible, and on an ordinary television set you cannot see the positions of the actors and the distance between them. Our solution is to use a ceiling camera to record the actors’ lines of movement. This allows us to see exactly where each actor is standing in relation to the 360° camera. Next, we copied the lines of movement for each actor and all of the sound (mood, effects, etc.) in the Audiopost. An algorithm then calculates how it should sound.
Screening a VR film is complex, too. Every user views it in their own way,and every viewing direction has a different sound or tone. Consequently, instead of a single soundtrack there are several. This includes a datatrack containing information about the location of the sound. The device used to play the film (in many cases a telephone) must calculate in real time how it should sound. Some platforms still have a ways to go in terms of sound quality. Ashes to Ashes sounds best on a Samsung Gear VR. We were amazed by how accurate the positional sound was reproduced. The future is very promising!