Haptics for VR and AR - Where Are We ... Really?

I’ve been getting one or two “cease and desist” DMs/e-mails recently relating to my apparent negativity towards current-generation haptics feedback technologies and the applications the developers are claiming they are revolutionising. Well, having invented one of the first (if not the first) haptic/tactile feedback gloves, Teletact, back in 1990, not to mention having experimented with vibrotactile, electromechanical, exoskeletal and many other devices over the years (see: https://pdfs.semanticscholar.org/b614/b201e4790b9890469b993099ef70f1736298.pdf), I feel totally justified in making these comments and make no apologies for doing so.

As I explain to the many visitors we host who believe what they’re seeing online, once again we’re witnessing a period of over-hyped claims of what exoskeletal/glove-like wearables, fingertip transducers (or "tactors") and ultrasonics (to mention but three examples) are, in reality, capable of delivering. Don’t get me wrong, I think these technologies, from a research prototype perspective, are great fun and they’re great PR when shown on videos and at exhibitions. But are they really at a level of maturity where they can be used reliably, to enhance simulated experiences for automotive design or safety-critical applications like surgery (to mention but two)? 

Just look at the videos. Think about how many takes the video sequences may have required before a realistic/believable contact or manipulation was made by the end user. Is the user actually experiencing an accurate or realistic tactile/force experience with an object or just a simple additional cue to having collided with that object? Look at or consider how these devices compromise the user’s performance in a virtual or augmented reality environment - forcing that user to adopt unrealistic postures, move slowly, often unnaturally (especially when interacting with virtual controls, as shown in vehicle interior design). The same is true with many examples of telerobotics or telepresence systems. Question the overall usability and wearability of the devices – can the user really sustain reaches, grasps and other manual interactions for any length of time – is the design of the system likely to interfere, physically, with other nearby tech? In some recent videos the wearer hardly moves his/her hand at all, whilst other haptic feedback devices are used to complete the main tasks! 

Why? Well, if there’s one thing that the Teletact project (and other not-so-successful VR/AR experiences) taught me, all those years ago, it was this. When contemplating the use of interactive hardware and software technologies in VR, AR and, today MR, especially those where you need to provide some form of guarantee to the sponsor that your solution will transfer positive skills from the virtual to the real, it is absolutely vital that you understand the user’s task thoroughly from a human factors perspective and then consider how best to achieve the key perceptual and motor elements of that task by delivering the most appropriate content and interactive technologies. Credible, believable, reliable and affordable haptics technologies in VR and AR, are, I’m afraid, many years away, which is why we are pressing forward with our MR research in the defence healthcare arena (at least) where we demonstrate the importance of blending the best of the real with the best of the virtual, thereby enhancing end user believability during training (see images below). I worked on automotive design and VR many years ago too, where we helped to design a low-cost seating buck that provided all the believable haptics the headset-bedecked user actually needed. And the experience was realistic enough so that they were able to exploit the support provided by a wooden steering wheel and other interior objects, rather than dangle their arms in mid-air, with the resulting fatigue issues that would have undoubtedly brought. 

It’s a bit like the incessant claims of “full immersion” and “”Holodeck” achievements” we see on LinkedIn and elsewhere – we are decades away from achieving a wearable-free realistic multisensory interactive environment of the sort depicted on Star Trek. I certainly won’t be around when that happens, neither will many LinkedIn readers – we’ll be in our own metaverse, no doubt! It’s a similar situation with haptics. When you understand the challenges faced by developers to satisfy the many – and physiologically unique – sensors distributed within the skin of the human, not to mention the muscular and tendon-based subsystems governing kinaesthesis and proprioception, and, of course, the importance of ensuring a seamless integration of visual, cognitive and motor features when designing simulated tasks requiring accurate object and tissue manipulation, then you can appreciate the challenges the haptics community really faces. So this is not to deter researchers and companies from developing novel approaches to haptics … far from it. But, for potential end users of today’s haptics tech (and for the foreseeable future), please just think about the task you’re hoping to simulate, the importance of the skills you’re trying to foster, and the possible negative impact of adopting the wrong technology, especially for safety-critical task training. And, most importantly (as I always say in my talks and lectures), do not believe everything you read or see online!