VR Production 101: What Stays and What Goes as Virtual Reality and Augmented Reality Offer New Canvas to Content Creators

• Producing VR content brings new challenges to video content creators, but many of their traditional skills still apply.
• In the VR world, viewers must discover the message through their own voyage of discovery in the environment: they should not be dictated to in the storytelling process. 
• There is a shortage of tools for producing VR content, but game engine technology is maturing, and proving to be invaluable for VR productions.

Video content producers are increasingly being called on to produce content for virtual reality environments and are eagerly embracing this new medium, but it presents some challenges: both to their creativity and from a technology perspective.

In virtual reality, content creators can no longer dictate exactly what the viewer will experience. Because commercial offerings to support VR/AR productions are only just coming on the market, artists have to find new ways to tell stories and need to cobble together the technology to do this.

Boo Wong, Group Director of Emerging Technology at The Mill

So says Boo Wong, Group Director of Emerging Technology at The Mill, with whom we caught up to learn more about the artistic and technological challenges in immersive media, and how video content producers are rising to meet them.

 [Editor’s Note: In September of 2015, Technicolor acquired London-based The Mill, the world’s largest visual effects and content creation studio for the advertising industry.]

Q: Boo, before we delve into the conversation tell me a little about The Mill and what it means to be Group Director of Emerging Technology.

Wong: The Mill is a visual effects and content creation studio that collaborates on content for the advertising, gaming, and music industries. For 26 years we have worked at the frontiers of visual narrative, creating content for screens of every type and size.

In the old days that meant television. Then we moved into web content. Now, we produce content for large-scale interactive surfaces, as well as for VR and AR.The area I look after is emerging technology. I like to describe it as ‘constantly looking forward’.

My responsibility is to identify the new things that are upon us now and that are coming up: to find them, scout them out, look to see how we can bring them to the premium short form content we make, and then roll them in. Today, that means we are working with VR and AR, in their experiential spaces. But who knows what it will be in five years time.

Q: As The Mill look at these new areas from the perspective of more than two decade’s experience, how do you think differently about content creation as it is applied to persuasive storytelling?

Wong:  We have so much experience in telling a linear story. We know exactly where to put the viewer, exactly how to time things out, how to frame things, who to put on the screen to convey whatever message we want to tell.

When it comes to the type of content we make — short form content for commercials, for instance — we have to be super efficient. We have to get our message across very quickly, right on point. But now we are moving into newer spaces, newer screens that are not all of a set size and are not even screens in some cases, just interactive spaces, hanging in the air.

Taking all that into consideration we have to find new ways to tell the story. We have to understand that the story can exist all around the viewer. We have to give up the ‘dictatorship of attention’. We can no longer tell people what to do, or where to look.

That is the starting point for us when we create this type of content. The viewer has so much more control. We have to put them into a world that allows them to discover what we want them to see, but without saying to them “Hey you have to look here, because that is the only choice you have.”

Q: That must have an impact on the amount of time you need to produce deliverables. It’s not like having a 30-second spot where people are looking at exactly what you want them to look at. Do you need more time to produce these projects, or can you work within the form factors that traditional advertising and commercial production offer?

Wong: It takes a bit longer than 30 seconds for someone to explore content in this new format. However, it is in its early days. When audiences explore a piece of immersive content, their tolerance is still under the three-minute mark. We have to keep that in mind at all times, but we can certainly tell a short story in the time frame allotted. That is our history at the Mill.

A lot of the work we do these days is in the real time arena - I like to compare it with what game developers do. They have 20 to 40 hours to tell a story over the course of a game. We have three minutes to tell the same story and really make an impact. That level of efficiency in storytelling is something that is well honed in the area we work in.

Q: How many of the skills have been developed over the years in what is now thought of as “traditional mediums” pour over into the areas you are now exploring?

Wong: We are finding a lot of them do apply. My background is in visual effects, and when we are building this new experience, we are basically building onto what we already know. It is just that instead of building content for a rectangular space, we are adapting for content that is around us.

A lot of the things do we for premium level filmmaking, for visual effects, for storytelling, all come into play in this new area. We are finding a great deal of synergy. Creatively, it is a really exciting space to work in.

But there are challenges that show up in early discussions about how these new forms of content need to be created. For instance, we will often see scripts that have been written for a short film or a traditional commercial. You will see instructions like “start wide, then pan left, then cut to close up.”

That direction absolutely does not work with VR. You can’t tell people where to look or what to look at. Audiences are going to look wherever they want. We see a lot of low level mistakes like that.

The other thing people don’t realize is how much time it can take to explore interactive content. You have to understand the user’s point of view extremely well, because you are not telling them what to do. You have to try and understand what they might innately want to do for themselves.

That requires a whole new way of thinking from a content development perspective. It is not something that traditional filmmakers or traditional advertising clients might think about if they have never worked in this medium before.

However, it is not hard to do, once you have that mindset. Here at The Mill we get it. We know how to tell a good story, and now we have added on this additional dimension.

Q: What about tools of the trade? There is a long history of creating workflows and management tools for artists who work with flat images. What is the state of things in the VR world to help tell those multi-dimensional stories and immersive persuasive engagements?

Wong: We are in the very early stages of VR. Some people say it is year two, because it is the second year that VR headsets have been available. But in some ways it is only year one. The first room-scale VR is shipping this year.

If you look at everything from the way you make content for VR to the way you deliver VR, the technology is still young and immature. Where it is going remains to be seen, but it is super exciting.

There are currently very few tools for making VR content. In many cases, we have had to cobble together existing tools to do what we want. If we are shooting a live action VR experience, like a 360-degree video, people are strapping together Sony A7 digital cameras, GoPro rigs, in some cases with tape. Literally.

However, this year you’ve seen the first Google Jump camera and there are rigs being built for this.

In the visual effects, post production side of VR, many of the normal production things we do – editorial, color, visual effects – all still apply.

The crew I lead to work on VR jobs is the same crew that I have been working with to produce visual effects for 20 years. It is just that now, the crew is working in 360-degree environments. Some things -like rendering - take longer, but it many ways, it is basically business as usual.

There are challenges. For instance, as yet, there are no automated solutions for stitching live action content in VR that work 100 percent effectively, but the technologies are getting better.

One technology that is very mature – and that traditional visual effects companies are moving into – is real time game engine development. You have to work in game engines for a lot of interactive VR content, and game developers have been using these for years.

We are new to this technology, but it is very close to what we have been doing on the computer-generated visual effects side. The technology is quite developed with many of great features. Tapping into this and using it to tell our stories in VR is awesome.

In summary, there are some areas where we are taking baby steps and others where everything is ready to go.