Creating compelling films and games for virtual reality is difficult. You need to create coherent narratives in a relatively new story-telling medium while avoiding some of the technical problems that risk making viewers throw up.
Those are just some of the biggest nuances that coders and designers that make creating virtual reality different than traditional movies and television, Google (GOOG) chief game designer Noah Falstein explained this week at the Virtual Reality Developers Conference in San Francisco.
Falstein likened today’s era of virtual reality to the film industry’s early years and the rise of computer video games in the 1980s. Developers, designers, and writers for virtual reality are entering uncharted territory, and it’s up to them to experiment with new kinds of storytelling and coding tricks.
Get Data Sheet, Coins2Day’s technology newsletter.
Consider a movie with a scene in which a woman in a burning house screams from a window. The image then quickly jumps to another scene showing fire fighters getting the call at a fire department before the camera jumps back to the terrified woman.
In the early days of cinema, some filmmakers thought such a rapid succession of scenes would confuse viewers, explained Falstein, who previously led development for some of the biggest video games created by LucasArts, the former gaming studio of film company Lucasfilm, producer of franchises like Star Wars and Indiana Jones. But as film caught on, they got used to it.
In virtual reality, filmmakers face the risk of viewers turning their gaze in the virtual environment away from where the director intended, Falstein pointed out. His advice was to give viewers cues that hint at where they should focus.
“Don’t be afraid of that,” Falstein said about the possible confusion. Instead use “light and sound to direct them to where you want them to look.”
Another wrinkle in virtual reality is the strong reaction by viewers to characters in the action staring at them. It’s an issue that the creators of the Emmy-award winning virtual reality animated short Henry, starring a lonely hedgehog , had to grapple with.
During screenings in the film’s early development, viewers felt insulted by the hedgehog, who failed to engage with them. Instead, he would ignore their presence because the film’s creators didn’t initially take viewers into account.
“You are there and he’s not reacting to you,” Falstein said about the feeling of rejection viewers felt. By testing the film with audiences prior to its premiere in 2015, according the Los Angeles Times, Henry‘s filmmakers were presumably able to fix the problem along with other issues.
Another common problem faced by virtual reality film makers is audiences sometimes get motion sickness because of what’s known as latency. For example, when people move their heads in virtual reality, there may be a slight delay while the software tries to keep up with the appropriate images in their field of vision. This delay can disorient users, because what they perceive is not matching to the movement of their bodies.
Falstein said that designers can do a couple of things to limit the unpleasantness.
In real life, if an object like a baseball is hurled in your direction, the baseball should appear bigger as it moves closer. But if thrown in a poorly designed virtual world, its size may not change, causing viewers’ brains to be confused and creating a sense of nausea.
It’s not an easy task, however. Designers also have to take in account that viewers have two eyes, and each eye must see the same object from its own slightly different perspective. In the case of the virtual baseball, the designers must make sure that a person’s right eye sees more of the right side of the baseball and that their left eye sees more of its left side. Any slight distortion from how people would see the baseball in real life could risk making them feel sick.
Likewise, virtual reality developers must ensure that background scenery like a mountain must not change in appearance as frequently as objects in the foreground while viewers move through virtual world. The difference mimics how humans perceive depth in the real world and failing to adhere to the rule could make viewers sick.
For more about virtual reality, watch:
Ultimately, in a carefully built virtual world, viewers can have a profound emotional experience. Falstein used the example of the virtual reality film Pearl, which put viewers in the passenger seat of a car watching a father driving interacting with the daughter in the back seat.
Watching the film in 2-D was charming enough, he said. But seeing it through virtual reality was “more intense” because he felt as if he was witnessing them up close.
“I was moved to tears,” Falstein said.
