Typing in VR: the unsolved problem of input

Virtual reality is a game-changing technology. And games are not the only industry that will be affected. We are living in the very early days of VR. Most content is currently being created by independent developers or small studios, and being consumed as a passive experience. Despite all these immersive experiences, 360 videos, and games are blazing the trail for VR; I predict the largest impact of this technology will be on the web.

Within a year, the number of total active smartphones on the planet will grow to 2 billion. Almost all of these devices have WebGL enabled browsers, and are capable of becoming true VR headsets for just a few dollars thanks to things like Google Cardboard (image below). Both Mozilla and Google have realized this and already have experimental WebVR features built into their nightly browser releases. This means over the next few years we will be undergoing a major transition on the web - the transition into 3 dimensions.

This is an exciting new technical and creative challenge for developers around the world. What will the 3d version of a dropdown look like? Or an input, button, carousel, modal, etc? Not to mention all of the UI elements that could not exist in 2 dimensions. There is a huge opportunity for designers to define the next generation of UIs. However, amidst all of this excitement and potential is one serious obstacle that no one has managed to successfully overcome: input.

We all know input is a hot topic in the VR space. After announcing their CV1 Q1 release, Oculus also mentioned that they will start discussing input leading up to E3 this year. What could their solution will be? My worry is that it will be another gearVR-like gamepad. Why is that a problem? You can't type a long email on a gamepad. Not as fast or easily as you can type on a keyboard anyways, and the average user will not sacrifice that convenience - no matter how pretty the visuals are. I fear a lack of intuitive, user-friendly input devices will jeopardize the revolutionary potential of this new technology, and risk it becoming another niche gamer and hobbyist peripheral. Something that is cool to own, but not a necessity - as opposed to something that is so necessary you don't even remember a world in which it didn't exist.

The truth is, the mouse and keyboard won't work in mobile VR. They are as bound to the 2D dimensional screens they serve, as the flat surfaces they lay on. But the need to type quickly, efficiently, and ergonomically will never disappear. With brain scanning devices like the Emotiv Epoch are still at least a decade away from being ready for mass markets, (Feel free to comment if you think I am wrong.) what will the device look like that can carry us along until then? Here are my thoughts and predictions. I would love to hear yours.

The input devices of VR will generally have two major form factors/use cases: immersion and productivity.

Immersion

These devices are all about the experience of VR, and include products like leap motion, prioVR, perception neuron, and many more. They aim to further suspend our disbelief and make us feel like we are truly in the environments that our headsets are trying to convince us we are in. I think these have an incredibly exciting and optimistic future, especially when you start thinking about how they can be used in AR as well. (Let's be honest, who doesn't want their own Minority Report workstation?) Eventually I think AR and VR will be blended together in a single device - but that's a topic for another day.

Productivity

This is the less flashy type of input device, but in my opinion the kind that is much more important and critical to the adoption and success of VR in general. If I can't type a google search while using my VR device, I will find myself taking it off pretty quickly. Here are two specific products you can buy today that I think are at least on the right track.

The first is a device called the TREWGrip keyboard. Their product was targeted at medical professionals who are constantly on the go and want to be able to take notes while maintaining eye contact with patients. The pro here is you don't really have to relearn a keyboard layout, as it is already very similar to many ergonomic keyboards. The con is the size (not really portable) and the fact that there is no trackball or trackpad. The cursor is controlled by the device's orientation. Eventually when HMDs have great eye tracking support perhaps this issue will be moot. Until then a VR input device must have a great mouse replacement.

The next device I found, which I think is further ahead in terms of its form factor, is the alpha grip. I have actually purchased this to try, and I have to say it does feel good to hold. The con (and it's a big one) is the fact that you have to relearn a completely new keyboard layout. It's not querty, or anything like it really. Before you immediately write this off as a deal breaker, keep in mind the following thought. A shift as radical from a 24'' monitor to full immersive VR will surely necessitate a shift just as radical in terms of input. I believe the best input device for VR will look nothing like a QUERTY keyboard (Feel free to comment if you disagree.) Although the alpha grip is not perfect, it has the right idea. I should just be able to hold the device comfortably in my hands for hours (like a gamepad) and be able to type an email at 60+ words per minute. Creating a device that can meet this goal will be a tremendous design challenge, but there is a HUGE reward for the first company that manages to achieve it. We all know that VR isn't going away, it's just going to keep getting bigger. Much bigger.

So what about now?

In the off chance that Oculus does announce a gamepad as their input solution in the coming months, I have played around with what I think is the best UI for typing with a gamepad. Steam spent a lot of time designing their 'Daisywheel' UI for their 'Big Picture' mode, and Chris Dolphin did a great job porting it to HTML/CSS. I further expanded on that and created a simple WebVR demo you can try out here. (You can read my post about that) You must visit the site from a WebVR enabled browser of course, so either Firefox Nightly or Chromium. Once the site loads, just connect your gamepad, press any button, and then click on both input elements in the left and right views. Then view the window on your oculus in full screen (F11) and see what it feels like to use the daisywheel. It's not bad; with some practice you can approach smartphone typing speed.

Like I said, I would be disappointed if Oculus announces a gamepad as their input device. But if they do, thanks to Valve, we at least have a starting point. Websites that want to take advantage of WebVR technologies can use this or something similar for user input until a better device is created.

We are living in new and exciting times. We have a chance to redefine how we interact with our computers. I imagine in the not-too-distant future, we all have just one device - our glasses. Things like PCs, laptops, tablets, mobile phones, and even smart watches will seem antiquated and archaic. Usurping the long and well-deserved reign of the keyboard and mouse will not be easy, nor will it be painless.

Let's just make sure it is well worth it.