Future Interfaces, Science Fiction and Idea Transmission – Can Our Imagination Outpace Reality?

Gotta Hand it To Ya

What’s in a digit?

VR is a short acronym with a lot of history, a lot of baggage and a lot of dreams, be they manifested or broken. Whether it comes from its constant dance with the military-industrial complex; broken promises of holodecks and anti-gravity skateboards from ‘90s futurism; the many dystopian visions afforded us by Lynch (Wild Palms), Cronenberg (eXistenz) and Leonard (Lawnmower Man), among countless others’ or the entries that assume that it will come and form the foundation of our virtual beingness like Stephenson’s Snow Crash, Vinge’s Rainbow’s End, Kline’s Ready Player One and Jeff Noon’s VURT, William Shatner’s TV series TekWar (1994)* or Kathryn Bigelow’s Strange Days (1995), the exploration of life with VR never comes in moderation. It is depicted as a pandemic, a soma, a panacea, a global construct.

As we see it appear more frequently and in larger-scale productions, what is so fascinating is the amount of imagination required to stay ahead of reality. From Altered Carbon and Philip K. Dick’s Electric Dreams to Black Mirror and Star Trek, Virtual and Augmented Reality are increasingly depicted in high quality, big budget mainstream fare.

The other day (Q1 2020), I was at a projection mapping enthusiasts’ meetup in Los Angeles and ran into Todd A. Marks from imagesonscreen.com, who for over a quarter-century has been handling customized, story-specific playback graphics that are designed to mesh with the visual style of the project, while providing an authentic “look and feel”. This has led to managing and working with collaborators on the computer screen UIs for films like The Net, Deep Impact, Soderbergh’s Solaris, Anchorman, Constantine, Venom and Star Trek: Nemesis among many others. The conversation led to advancements in playback technology, like moving from CRTs stowed behind smoked plexiglass to the argument for 4K LCD monitors. But more importantly, we discussed the challenges and creativity required to devise what is actually on the screen.

This serendipity was kind of amazing because the YouTube algo had just thrown before me a takedown of Hackers in Movies a few days prior. So I could tell I was in a flow state freq.

Todd told me “One of the things that I do on a film/production is help into introduce them to new display technologies and help them think about different methods to interact and interface with them.”

Mise En Place

You see, though it is easy to do a takedown on of the realism of computer screens operated by movie hackers as opposed to real-life Linux consoles, the audience needs to understand what is actually being explained in a visual manner that advances the narrative. Marks calls it VIPAL: “Visually Interesting, Plausible and Logical.”

I found this notion exhilarating. How do you strike the balance between reality and fiction? Between getting the point across and being literal. Heck, the dripping green digits in The Matrix – that presumably contained all the code for reality as we know it — turned out to be little more than a Japanese food menu. How can you implement the ideal narrative interface in situ in immersive contexts? This will require substantial play and experimentation, a liberation of the imagination and the capability to deploy it smoothly.

Vox.com took a look at this ideation and production pipeline in the following video:

Get On Board

About a year ago, I took my lifelong casual interest in board game design much more seriously. I spend a lot of time on my own in Hollywood, as many do, and was tired of looking at the social media scrolling down my 6-inch screen. I wanted to give my eyes a break, do something with my hands, get my ideas on the table in front of me, analyze game mechanics, systems, UI and the solo automata for running campaigns with paper and dice “AI.” It has proven to be an exceedingly rewarding and educational journey.

A board game is an interesting conundrum – immutable in its organic form of printed and molded materials, yet constantly interpreted, reinterpreted, modulated and reconsidered by its users. It has to be color-blind-proof, language-proof, easy to pick up and organize, easy to set up, quick to learn, but rich in emergent play and complexity. Something that can be understood by sight, touch, voice, silence, relational dynamics, deduction and risk-taking.

What does this have to do with computer screens in movies? Maybe it is becoming clearer. How we interface with data is more than just design and science, more than just practicality and narrative. It is a shortcut for the transmission of complex ideas, even more concise and precise than storytelling.

How can we use these examinations to design better-spatialized interfaces? What I call diegetic UI – in other words, things that feel intrinsic to their contexts and as readily available as the same?

This holds whether we are figuring out the scanners on a Tie Fighter that was designed in the 1970s to the star charts on the USS Enterprise in 2020. How can we retain the style and cultural effects of an interface while also considering its longevity and ability to transcend linguistic frameworks, all the while managing the widest subset of possible scenarios and data?

Digital Divide

In the immersive industry this question looms larger than ever as the very nature of its interface is under design. Some interesting experiments are being done now that Leap Motion’s mature hand tracking technology is now essentially incorporated into the very popular and tetherless Oculus Quest head mounted display device.

Behold just some of the early experiments taking place in the side-loaded, handtracking gallery at SideQuest:

While I could explore this subject in a million different vectors radiating out from this question, I will leave it as food for thought: How do we free our imagination and expand it to contain not only the data the future will provide us, but the new ways we’ll engage with it?

*full disclosure: in the series I play the role of an underground VR hacker/dealer named Mustapha.

About the Author:
Keram Malicki-Sanchez is the executive director of the VRTO Virtual & Augmented Reality World Conference & Expo and the FIVARS Festival of International Virtual & Augmented Reality Stories. He is an essayist who recently published in the Handbook of Research on the Global Impacts and Roles of Immersive Media from IGI Global.

Special thanks to Matthew Johnson for his assistance with this article.