Table of Contents
A closer observation of our typing habits could show that we don’t type in static movements. There is a flow in movement and in pace, and there is no one way that is followed by everyone. Irrespective of whether we are using a computer or a smartphone, the typing experience could make or break our overall experience with a product.
As the world of tech grows leaps and bounds, introducing and acclimatizing us to newer ways of experiencing life, immersive technologies like Virtual Reality (VR) and Augmented Reality (AR) are no longer a thing of sci-fi. They have already found their place in our lives, inducing much curiosity and excitement. With these technologies influencing more practical spheres of life, it also means that typing experiences are no longer limited to the ways we are used to. Typing on smartphones and keyboards in a way we are used to our entire lives, only to learn an entirely different way of typing in AR or VR, is easier said than done.
Why new modes of typing tend to fail
It’s not always that a new way of doing something appeals so much to its users that it becomes mainstream. Microinteractions, which play a pivotal role in forming digital habits, can provide more context in this regard. It is key to acknowledge the importance of these small and simple interactions that we barely notice until they’re not there but make all the difference. Successful microinteractions make user experience engaging by providing necessary feedback and stable by always following the rules- the user knows what to expect whether they are using it for the very first time or been using it for long. Predictability, in this case, is a good thing.
This also explains why new layouts tend to fail, as they contradict the user’s long established patterns of behavior, and users on the other hand don’t like to learn a new layout.
Take Pinch Typing for example. Developed by Facebook Reality Labs to address the text entry problem in VR and AR settings, this mode of typing assigns each finger a group of letters from the standard QWERTY layout. A particular letter group is entered when you pinch that finger against your thumb. It’s a language algorithm that determines what word you were intending to type, based on each letter group and previous context. Not surprisingly, this makes typing an extremely slow process- in comparison to an average of 120 WPM for a QWERTY keyboard or 80 WPM for a modern smartphone using the Fleksy keyboard, Pinch Typing allows for an average of just over 12 WPM 🙄!
Consider some other modes of text input. Typing mid-air lacks haptic feedback. Controller based typing becomes tiresome- navigating from letter to letter, also makes it slow. VR punch keyboards can cause physical discomfort including strain on arms and shoulders. Besides this, there are also alternatives to the QWERTY layout for devices with limited number of buttons, that does help make typing faster but still is a completely new type of interaction for those unversed and hence involves a steeper learning curve. These comparisons make it easier to understand why typing with just the movements of a finger is the only way in which people will be able to type in an ergonomic way.
Typing experiences in Virtual (VR) and Augmented Realities (AR)
Fundamentally, AR uses real-world settings while VR uses completely virtual or fictional settings. VR is a completely immersive experience that transports you into an altogether different reality, virtually. You don’t see your surroundings as they are completely blocked out. It doesn’t matter where you are physically- while in a game you could be sitting down, steering a fighter jet or in an app, you could be touring a virtual landscape.
Augmented Reality, on the other hand, adds or augments your surroundings instead of replacing them. AR glasses are smart glasses allowing you to see through, designed for free movement, simultaneously projecting an overlay of information including images, over whatever you choose to look at. This concept has started with smartphones and this concept is popularly extended to smartphones through AR apps (IKEA Place) and games(Pokemon Go), using the phone camera to track your surroundings.
As VR interactions can mostly be done sitting down, it is safe to say that VR offers a better case scenario for a keyboard typing experience without breaking the virtual immersion. It’s also relatively straight forward- the glasses can represent the physical keyboard in the virtual space with the same layout. This also ticks the box for ergonomics. Typing in this case, can be further enhanced by features like next word predictions, that are available at the user’s disposition, enabling them to just pick words using their fingers on the virtual keyboard.
Typing gets trickier in AR, where you are not sitting down but moving around.
Swipe input is the way to go in AR
Apart from being an intuitive way of typing in AR, swipe typing allows users to move around freely without having to carry around another device for text input. Most importantly, it is ergonomic and sustainable in the long run as users don’t have to learn their way through new, unfamiliar layouts but can be comfortable with their interactions- physically and cognitively.
There is a reason why keyboards are used umpteen number of times in a day- convenience, reliability, ergonomics and the list goes on. Everything that exists to make virtual keyboard and smartphone experiences a seamless part of everyday life should be present in immersive technologies too. Thanks to its immense potential to get this right, the future sure looks bright for swipe input text entry in the Metaverse.
Other modes of input vs Swipe input
What are some of the ways by which you can input text in augmented reality? To begin with, traditional ways of typing that involve a regular physical keyboard is restrictive of movement in an AR environment, and is also not ergonomic. To input text using eye tracking, the user moves their eyes across the keyboard and rests their gaze, by staring or blinking, at the character they want to enter. Though this offers a hands-free experience, prolonged use could put tremendous strain on the eyes and tire you out. Speech-to-text or voice input, though intuitive, is not always reliable as of now. It is prone to accuracy issues owing to peculiarities in accents and variations in dictation, ultimately requiring editing. Moreover, the user will have to dictate the words out loud to input text every time, making it less of a private experience.
In the near future, when AR interactions are independent of smartphones and typing capabilities entirely move to the AR glasses itself, we would find ourselves typing on a projected keyboard while on the go. Imagine a scenario in augmented reality, where you go about your day in the city with your AR glasses on and have to text someone. Since you don’t have anything physical to type against, it can be difficult to type quickly.
In this scenario, swiping to enter text can offer a more natural and effortless experience as it enables a flow between keys on a virtual keyboard. “Swipe typing” enables the user to type by touching the first letter of a word and continue “swiping” or moving their finger from one letter to the next. Several decoding candidates ordered by score are shown to the user once the user lifts their finger. This score is calculated by a spatial probability model which takes into account each point of the swipe. A beam search algorithm further filters out invalid search paths to arrive at the words that are the best fit to code. An invalid search path could be one with a decoding probability lower than limit calculated from real data from users. Several dictionaries of valid words are used to build these search paths in what’s called a “Directed Acyclic Graph” or DAG optimized data structure.
Fleksy is an award-winning virtual keyboard technology provider. The Fleksy Software Development Kits (SDKs) enable developers and companies to overcome the challenge of building a top-notch typing experience across various platforms. For more information, visit www.fleksy.com or get in touch with our team
