‘Choose your own adventure’: VR journalism gives audience control

Sign up for The Media Today, CJR’s daily newsletter.

With cheaper and more accessible technology, the production of 3D content will soon become commonplace in newsrooms around the world. Immersive content will change how journalists tell stories, and how people consume news. A new study suggests that virtual reality offers an opportunity to engage readers more fully, but with costs.

Some colleagues and I at the Associated Press set out to measure the effects of immersive storytelling on the brain. In addition to interviewing over 50 experts in virtual reality journalism, the research team collaborated with neuroscientists at Multimer, an MIT Media Lab spinoff, to analyze overall levels of attention and relaxation. The study looks at biological feedback from motion-capture suits, EEG brainwave sensors, and heart-rate monitors to gather concrete evidence of how the brain and body react to VR journalism. You can read the full report here.

By connecting VR users to bio-sensors, the AP determined that engagement levels are at their highest when people consume fully immersive experiences in “room-scale VR.” This refers to a virtual environment in which people walk through scenes and touch 3D objects, as opposed to passively consuming multimedia content on devices without the ability to interact.

As with other nascent technologies, there are significant hurdles: cost, training, and profitability. Not only does immersive media require specialized editorial training for the journalists, it is also more demanding for sources, more challenging to distribute, and potentially emotionally taxing for consumers.

Emerging technologies can now produce vibrant 3D models that give us an even greater sense of visual richness. A technology called volumetric capture is enabling journalists to create stories where participants are able to touch objects and interact with others. “We experience the world with our whole bodies, so why shouldn’t we experience stories with our whole bodies?” asks Nonny De La Peña, founder of Emblematic Group, an immersive journalism studio in Santa Monica, California.

Our research suggests that individuals had increased levels of “open-mindedness”—being attracted to a topic, but not alarmed—during room-scale VR experiences, when compared to cardboards and regular VR headsets, which do not allow for movement or digital touch. High engagement was also driven by better image quality in a room-scale experience and the ability to freely move around the virtual space.

Audiences will be able to choose different story paths—a “choose your own adventure” version of journalism.

“When you include more interactivity by impacting someone’s senses as well as giving them six degrees of freedom [forward/backward, up/down, left/right], it means there’s a stronger reaction, a stronger commitment, and ultimately a stronger memory of the experience,” says Gabo Arora, documentary filmmaker and founder of the United Nations’s VR unit.

More user immersion can also mean less editorial control. As the technology powering 3D models gets more advanced, journalists will be able to develop multiple storylines in a single environment. Audiences will no longer be guided in a linear progression, but will be able to choose different story paths as they freely explore the virtual space—a “choose your own adventure” version of journalism.

For example, a VR rendition of a protest on raising the minimum wage could allow a participant to choose to follow those marching in favor, or decide to stand with those against reform.

“Like other interactive media, VR allows for non-linear exploration and peripheral engagement with news stories. Design permitting, people can explore the parts of a news story they find the most interesting, as well as jump between subject perspectives at different points in time,” says Jonathan Albright, research director at the Tow Center at Columbia Journalism School.

This also means that news organizations relinquish a degree of control over the narrative. In traditional storytelling, journalists can steer the camera, decide where to focus, and even suggest where the audience should direct its attention. In virtual reality, participants can choose where to look when, and which questions they want answered, by inspecting the virtual environment at their own pace and from multiple perspectives.

With higher engagement, VR can also be used to elicit specific emotions. Data from the AP study suggests that war zone reporting in VR drives participant “stimulation,” while science and environment stories build open-mindedness. Metrics associated with “power and intensity”—related to the lasting impact of the experience—were also highest for stories about war and conflict when compared to other news types.

Journalists who want to use VR have significant challenges ahead, including understanding the psychological effect stories have on news consumers. But these concerns are familiar territory for journalism. As Shaza Nessa, global head of visuals at The Wall Street Journal, says, “A great story told well is a great story, no matter what the format. With VR, lots of perennial questions have reemerged, as they do with all new frontiers … around journalism ethics and standards, newsroom workflows, technology, skills, and user adoption.”

Has America ever needed a media defender more than now? Help us by joining CJR today.

Francesco Marconi and Taylor Nakagawa are the authors of this article. Francesco Marconi is the manager of strategic planning and development at the Associated Press and an innovation fellow at the Tow Center for Digital Journalism at Columbia University. Taylor Nakagawa is an immersive media fellow at the Associated Press, a master’s candidate at the Missouri School of Journalism, and a recipient of the Scripps Howard Foundation Scholarship.