Specification of technical aspects
Last updated
Last updated
With cinema, that little hole you see through, I give you 20% as director and you figure out the other 80%. That’s the dialectic. With virtual reality I give you 360 degrees. In this, ironically, you have the control. I give you the will, with light and sound and all, but you act unilaterally. It reveals who you are.
(~Alejandro González Iñárritu)
You have to think on the the mise-on-scene:
What will be shown? e.g. choose your location, lightning, colors, objects
Who is in your scene? e.g. choose your characters, clothing, positioning in space, the movement of the characters towards the camera
How do you design your 360° space? Make use of quadrants
How do you experience your space? create an atmosphere
Think about your location, lightning, colors, objects
Who is in your scene? e.g. choose your characters, clothing, positioning in space, the movement of the characters towards the camera, non-verbal language
How do you design your 360° space? Make use of quadrants.
How do you experience your space? Create an atmosphere.
How do you show your experience?
Integration: How are spatial territories defined and integrated?
Flow: What obstacles and openings are placed?
Choice: What choice do users have?
What information will be shown and how to use texst? Today, we find ourselves at the brink of yet another typographical transformation: virtual reality. People tend to be drawn to a headset proving them with a 360° experience. An experience in which text is being presented in a 3D environment. Typefaces, however, are developed to work in a two dimensional setting. How do they act in a three dimensional habitat? Overall, VR cases tend to minimize the use of text.
8 rules for working with legible text an type in a VR environment:
typeface: opt for a legible sans serif typeface
font-size: opt for larger that 28px, preferably +32px
line length: opt for less than 55 characters per line
leading: opt for 115% up to 150%
paragraph length: opt for less than 9 lines per paragraph
colour & contrast : opt for high level contrast
position in space: when using text on a fixed locatoin: opt for rotation
distance: opt for 3 to 4.5m
How do we move into the scenes? E.g. are temporal and spatial transitions seamless?
The standard cut in film editing is a hard cut which is utilised where one cut from the end of one clip is pasted to the beginning of another. In 360-degree experiences, this hard cut brings the spectator back to reality. As an opposite, we invented a soft cut technique in which at any time by measuring the headset’s rotation, we know what and where the spectator is looking at. By recording or making every scene multiple times, but with little changes in one half of the 360-degree view, the scene is changed accordingly, i.e. depending on where the spectator is looking at. Because of the minimal latency of updating the 3D world in a virtual reality device, scenes can switch without the viewer's notification.
How do you navigate/help your viewer? e.g. arrows, info messages, navigation signs, audio, …
Focus on crucial details to optimize the immersive experience
Are all the sounds in the story world that are also perceived by the characters (An example of this is the ringing of a phone or a conversation between two people )
Are not perceived by the characters but by the audience. (Examples include background music and voice-overs.)
Who is the teller?
Voice-over (comments)
Other characters
dialogue
monologue
inner monologue (=thoughts)
What technology do you use to show your VR application:
Devices: smartphone, tablet, laptop,
VR-goggles: Cardboard, Oculus Quest, Playstation VR, ..
Mobile app: Google Cardboard
Tracking movement: controller, haptic suit,
Physical context
Where are you going to show your VR ?
Agency
VR developers and storytellers see agency as VR's greatest promise. At the same time, they also call it one of the biggest challenges.. Agency refers to the freedom given to a VR user. That freedom translates, for example, into the ability to take actions, manipulate objects and navigate through space If VR developers and storytellers want to uphold agency, they encounter a narrative paradox. This means that the VR users' freedom of choice conflicts with the pre-programmed narrative.
“I think this storytree is a very hard one to do but it is the most beautiful VR there is. When a viewer can interact in the story and when he even has influence on the story, that’s when the magic happens.”
“It’s very difficult to tell your story and keep the game interactive. I think for us that’s one of the things that has been the hardest. It’s been a hold up point because we have a great story and we also want to make the game interactive.” (Participant 5)
“I am always juggling between storytelling and techniques to use for it. It seems like you can’t have a lot of a narrative when you make it interactive and vica versa.”
VR developers and storytellers attempt to resolve this paradox through a diversity of storytelling techniques: (1) user identification, (2) interaction opportunities and (3) illusion of agency.
VR developers and storytellers repeatedly indicated that it is important to first identify the target audience. In doing so, they ask themselves two questions:
(1) Who is the VR user?
(2) What is the role of the VR user?
A good picture of VR users is necessary to respond appropriately to certain interests or characteristics. After all, not everyone likes the same stories and storytelling techniques.
“So, when I became more experienced, I’m more conscious of the fact that as a content creator I have to make sure I’m asking myself the questions: for who am I making this experience […] [sic] And you also have to keep in mind that not everyone likes VR the same.”
The interests and/or characteristics can be found out in different ways such as, for example, with a small analysis or a (group) interview.
Clarifying the role of the audience is another important step. VR developers and storytellers want to clarify at the beginning of the story whether VR users will experience the story passively or interactively. After all, it is easier for the VR users to be immediately included in the story if movement options, manipulation options and context are clear in advance.
“The person has to know who he is and where he is. So, you need to prepare them very good. [sic] […] It just has to be clear in the beginning of the movie or the game.”
"Plot a clear purpose in your story. Viewers need background information, especially with VR, to get into the story."
Different techniques can make the role clear. For example, another character can inform VR users of the context of the story. That character can then also introduce the role to the viewer. Another example is a short tutorial text that appears on the screen at the beginning. The creator can also choose to first show a short 2D intro movie where a story, which clarifies the context, is shown.
VR developers and storytellers state that a specific point of view (POV) or perspective is a good technique for making the role of VR users clear. When there are few or no interaction opportunities, a third-person perspective is usually chosen.
When interaction opportunities do exist, a first-person perspective is preferred. VR users may then also receive a virtual body (embodiment). With a third-person perspective, VR users know that their role is rather neutral. Participants also name this as fly on the wall or passive VR. With an embodiment perspective, the VR users know that their role is not neutral; this can increase empathy, according to the participants.”
Interaction
According to the type of VR, there are different interaction opportunities:
(1) characters
(2) objects
(3) Movement
characters
In addition to the VR user, the other, often non-playable characters (NPCs), are also important. VR developers and storytellers can choose the degree to which VR users can interact with other characters. A first level is that NPCs acknowledge VR users by, for example, staring or approaching. A second level is that NPCs ask the VR users questions, with no response options. A third level is that communication is possible between NPCs and VR users. For example, VR users can respond to questions by nodding, indicating an answer, or making a movement with a controller.
Objecten
VR developers and storytellers can also choose to what extent interaction with other objects is possible. For example, VR users can sometimes take problem-solving actions. Examples include grabbing or throwing objects and using weapons.
Movement
VR users are sometimes given the opportunity to move around in the world. This must be done slowly and unforced to avoid simulation sickness. VR developers and storytellers therefore often opt for teleportation. Sometimes there is also the ability to step around freely in the virtual world (free locomotion).
"Patrick Swayze plays a character in the film Ghost who has a presence in space, but no palpable relationship with reality. Sam Wheat, the main character in the film, shouts from all the rooftops, 'I'm here, I'm here,' but no one notices him. With a lot of frustration as a result. You also notice the latter with VR viewers who are not given a role in the story."
To avoid these frustrations, VR developers and storytellers employ techniques that create illusion of agency. VR users are directed without realizing it. This keeps the sense of agency high. The following techniques are discussed: (1) guidance, (2) visual effects, (3) audio, (4) cues.
Guidance
Other characters can be used to guide or influence the viewer. This guiding character is also called the agent. For example, the agent can point, look up startled, turn toward something, step toward something, address the viewer, call out, present and narrate.
“When I think about techniques that make the viewer look at a certain point, then I think about a technique with another actor or person in the world. When you make these actors look behind the viewer and make their facial expressions surprised, yes, that works with us. The viewer turns his head and is looking what is happening behind him.”
Visual effects
VR developers and storytellers use visual effects to direct the attention of VR users. Examples of visual effects include making the view cloudy if the player looks to the wrong side, using red colors in dangerous situations, making objects dark and lighting up routes.
“We had one of the scenarios in a huge cave and we shine light on places that are important.
Audio
VR developers and storytellers see audio as the most efficient way to trigger VR users (naudio = 19). Audio effects consist of non-diegetic sounds such as a voice-over, on the one hand, and diegetic sounds such as an inner-voice or spatial audio, on the other. Spatial audio is a technique where sound can be played at a particular location in virtual space.
“Spatial audio is the most important thing for us. We showed for example an elephant, but when it is walking behind you, you can hear the footsteps and the breathing behind you. The most natural reflex is to turn around and look what is walking behind you. So, audio cues are a natural way without forcing someone.”
Cues
Cues are another way to direct the VR user. These are objects such as arrows, objects, text and drawings. These can convey a strong message. A few participants (n = 4) feel that cues force the viewers and that it comes across as unnatural, making presence disappear.
"Sometimes you really have to go very specific and put arrows on the ground. [...] You can also go with text but the more explicit it becomes the less natural it is. That no longer has anything to do with discovery, exploration and freedom. A lot of tension is taken away then."
VR developers and storytellers are clearly looking for new ways to encourage immersion and presence. In this, they see the development of a new "language" as an important step. However, the need to reinvent storytelling techniques is present with any new medium. For example, filmmakers who wanted to get started with the first forms of cinematography faced the same quest (Rouse, 2016).
They tried to shape films with techniques from their own disciplines. Thus, cinematography was influenced by drama, sculpture, architecture, literature and drawing, among others. This cross-pollination can also be seen in VR: there is an influence from different genres and media such as animation, theater, games and film, among others. However, it is in this interdisciplinarity that new techniques emerge (Rouse, 2016). This can be framed within Bolter and Grusin's (1999) concept of remediations, which implies that people often adopt aspects of old media into new media in order to get a better grip on it.
Furthermore, spatiality is as an important aspect in this new "language. They call for a departure from the linear storyboard used in traditional media. Yet spatiality is not a new way of thinking. It is, as described earlier in the literature review, also found in video games and attractions where space is equally used.
Is there a need for new storytelling techniques?
These new storytelling techniques need to be adapted to the medium on the one hand and meet the narrative paradox on the other. Indeed, VR developers and storytellers focus heavily on agency, or the freedom of the VR user in the virtual world. This freedom of choice is a major challenge for VR developers and storytellers: after all, it conflicts with the pre-programmed narrative. This narrative paradox is not new. It has long been the subject of debates in other interactive media. One example is discussions of video games as narratives, where supporters of ludology go to war with supporters of narratology. (Jenkins, 2003; Murray, 2005). Juul (1998) does not see games as narrative media because the player's freedom of choice ensures great unpredictability. In contrast, Murray (2005) argues that games, despite the freedom of choice, are just as strong narrative media. After all, players are unconsciously directed by the rules, structures and boundaries of the game. This process influences how players act and how stories proceed.
In this way, game developers try to accommodate the narrative paradox (Bogost, 2008). They are increasingly trying to employ storytelling techniques that create the illusion of agency. Here, individual choices made by VR users nevertheless lead to the same storyline. The illusion of agency is reflected in various discussed storytelling techniques such as guides, spatial audio, visual effects, cues.