"The Three Musketeers." That's what Trent Spivey calls the first cohort of residents in the Immersive Storytelling Residency Program in the Media and Emerging Technology Lab at the University of North Carolina School of the Arts.
“Our skill sets really complemented each other in an interesting way,” Spivey said of the project that he, Alex Moro and Fernando Goyret worked on during their residency.
At the end of the six-month residency, the residents created “BonsAI,” an animated virtual reality narrative experience. It was inspired by the COVID-19 pandemic.
Spivey was the project’s screenwriter and producer. Moro was the software developer and Goyret was the 3-D technical artist.
“In this interactive virtual experience, you embody the apprentice to a top botanist who has been called away during the rise of a catastrophic plant virus,” Spivey, a UNCSA alumnus, said. “Operating on the fringes of biology and artificial intelligence, you take up the mantle to orchestrate a solution while trying to avoid the clutches of a nefarious agribusiness ...”
The Immersive Storytelling Residency Program is just one aspect of the Media and Emerging Technology Lab founded by the School of Filmmaking at UNCSA in 2017.
The goal was “to define and shape the impending evolution of how immersive story content will be created, consumed and told — in close collaboration with leading academic institutions and companies of North Carolina,” UNCSA stated about METL.
Ryan Schmaltz came up with the idea for the Immersive Storytelling Residency and has been the director of METL since its start.
“Basically, what we have observed in the last 30 years is we’ve gone through several revolutions in terms of technology,” Schmaltz said.
The first was personal computers followed by the internet and then smartphones.
“What we are on the cusp of here and what we are poised and sort of ready for is the next revolution, which is immersive,” he said.
He uses the word “immersive” as an umbrella term for virtual reality and augmented reality.
“Virtual reality is when you put on a head set and you are taken to an entirely different place, and augmented reality is when you overlay your surroundings with digital content,” Schmaltz said.
He said the world will eventually be one in which people will no longer consume digital information in square formats (screens). Instead, there will be the ability for digital content to be immersed around them in a variety of ways.
Before METL’s debut, Susan Ruskin, former dean of the UNCSA School of Filmmaking, noticed that major film festivals had sections dedicated to VR films that were being created and VR installations, Schmaltz said.
The lab was started, he said, to help the film school take advantage of these immerging technologies as storytelling mediums and offer new tools for filmmakers to tell stories in new ways.
“It’s the first of its kind type of lab within the film school,” he said.
Initially, METL officials worked on building awareness, establishing a mission and vision for the lab, and forming partnerships within UNCSA and in the community. Phase 2 focused on technology and the building of an industry-grade motion-capture stage, as well as creating partnerships with companies that have hardware or software.
In 2018, METL held its first Future of Reality Summit, a bi-annual event, and kept the number of participants — local, state, national and international — to about 150. Speakers included Zeina Abi Assy of Tribeca Film Institute, Ted Schilowitz of Paramount Pictures and Yelena Rachitsky of Oculus.
“We were supposed to do it this year but due to COVID, we’ve had to postpone it,” Schmaltz said.
The summit was a transitional moment for the lab.
“That’s really what put us out there,” Schmaltz said.
Looking ahead, METL officials have established a corporate membership program “to make it easier for this conduit between industry and academia to work together,” he said.
He declined to give names but said they are starting to take on their first members and are kicking off some new partnerships with Wake Forest Baptist Health.
Schmaltz said METL is working with Wake Forest Baptist Health and Innovation Quarter to combine arts and sciences in new ways, including the establishment of a virtual visitation pilot.
METL officials have also gotten aggressive in procuring grants.
The lab recently received a $25,000 Epic MegaGrant from Epic Games to continue its Immersive Storytelling Residency Program; and a grant from the Small Business & Technology Development Center in Greensboro — about $20,000 so far to support METL programs with $13,000 being directed to the residency program.
When Schmaltz came up with the idea for the Immersive Storytelling Residency, he saw there were limited budgets for budding filmmakers to create immersive content, and he saw the gaming and film industries colliding.
“What doesn’t exist is really a vernacular between these two highly creative industries,” he said.
He said the gaming industry is highly technical in terms of things such as software development, and the film industry has an established way of working.
“I created the residency program to address these two fundamental problems with the goal of having an interesting immersive experience at the end of the residency,” Schmaltz said.
The residents were asked to create a story, artwork, the coding and make their project work on a mainstream VR headset. They chose Oculus Quest. Their two parameters were to make the experience 8 to 20 minutes and do something that would have a positive impact.
“They definitely did that,” Schmaltz said. “I’m really happy with the outcome.”
Spivey, Moro and Goyret, who had never met before the residency, worked together for two weeks in March before the lockdown in North Carolina because of the pandemic.
“We were using a piece of software that allowed us to save all of our changes, upload it into a server and then redownload it from our home offices,” Spivey said.
The men spent about three months working from home.
“Our routine was talking in the morning, working throughout the day in our home offices then checking in again in the evening,” Spivey said.
But by June, they were tired of working from home. After making a request to school officials, the residents got to go back to working in the Center for Design Innovation in Innovation Quarter.
“They came up with some very good guidelines to keep everybody healthy and safe,” Spivey said.
The residents had the opportunity to work with voiceover actors, including Hannah Myers, a UNCSA alumna.
At the end of August, the residency completed. Now, as friends, Spivey, Moro and Goyret plan to run the film festival circuits with “BonsAI,” starting with Sundance Film Festival.
“It was a great opportunity for just three people to be in a room together and walk out with something on the other side,” Spivey said of the residency program. “Because we were the very first cohorts, we got to give a lot of feedback about the program itself and help build and polish it for the next round.”
Schmaltz said he has managed both small and large teams of people.
“I think first and foremost them (residents) working together with mutual respect and engaging in the exchange of ideas and everyone pulling their weight, that doesn’t always happen,” Schmaltz said. “I think the three individuals who participated in the program, definitely left as a unit, as a team. That was great to see.”
Stay up-to-date on what's happening
Receive the latest in local entertainment news in your inbox weekly!