Some Design Principles for Developing Audio-First VR

“Real Time Whenever Possible”

(Title sourced from VR researcher Ge Wang) The modern wave of VR and the cryptocurrency boom rapidly increased GPU development. Stunning 3D graphics can be rendered in realtime on consumer grade computers. Fixed media should no longer be used for technical reasons, it is only to be used in the event that real-time realization of a work drastically exceeds current technical limitations. A recent example is the portal scene in the motion picture Big Hero 6, where 3D fractals were rendered in the form of volumetric clouds, an impossibly resource-intensive render.

You are likely creating something more humble than this. If your simulation is just slightly too detailed to render in real time, simplifying the graphics and making it real time is almost always a more strategic decision.

Interface Balance

Just…no…

There are two dominant and polar tendencies for music based VR developers. One is to simply post existing instrumental paradigms directly into the VR realm, like a virtual string quartet. Another is to wipe the slate clean, like John Cage musing about the potential uses of the then-novel Theremin, and explore in an unbounded manner. An overly traditional approach yields boring results, and an overly progressive approach creates artworks incomprehensible without lengthy textual explanation. Every artist must find their own space in the grey area between these two poles. Both honor existing instrumental paradigms, but also take advantage of the novel VR space being played with.

Video Game Paradigm Awareness

The most recent wave of VR hardware is most commonly associated with commercial video games. As with mediums yoked to other commercial enterprises, there are wide ranging expectations as to how the medium is to be used. The artist can reject these norms but they should only do so if they have a clearly articulatable reason for doing so, as going abasing them will raise questions that will need to be answered in the artwork or paratext.

Collaborate

One developer cannot possibly consider all of the connections between audio and VR in a given artistic project. Collaborating at a nascent stage of development allows insight into more possible connections to the core concept of the work that will allow it to resonate more perfectly with the audience.

Use Idiomatic Stories

Some existing stories are already historically intertwined with VR. Mining them for your own artistic purposes allows you to bring in a whole world of associations without having to construct everything from the ground up. Allow culture to fill in the gaps.

Situate Yourself in the History of VR

Be keenly aware of the history of VR before integrating it into an audio based project. Hard won knowledge discovered decades ago by figures such as Scott Fisher and Jaron Lanier are still largely applicable. Ignoring things such as the minimum acceptable latency between motion and rendering in VR risks making your audience physically ill.

Cellular Growth of Ideas

Begin with a concept and allow it to branch outwards when developing for audio and VR. Permeate as many features of both mediums as possible with the small initial concept. Allowing this small idea to flourish in all of its different manifestations achieves a high level of recursion at different levels of scale, which is one of the features that demonstrates high quality in a musical work.

Simplify Interactions – Essentialism

What are the most valuable interactions out of all of the possibilities. Distilling ten interactions between audio and VR down to one meaningful and well designed feature will not only enhance the development focus on that one feature, it will be more understandable by the audience. This “peek behind the curtain” allows the audience to partially demystify the process, but also allows them to fill in the unknown variables with their own perceptions.

Leave a comment

Your email address will not be published. Required fields are marked *