• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer

Electrical Engineering News and Products

Electronics Engineering Resources, Articles, Forums, Tear Down Videos and Technical Electronics How-To's

  • Products / Components
    • Analog ICs
    • Connectors
    • Microcontrollers
    • Power Electronics
    • Sensors
    • Test and Measurement
    • Wire / Cable
  • Applications
    • Automotive/Transportation
    • Industrial
    • IoT
    • Medical
    • Telecommunications
    • Wearables
    • Wireless
  • Resources
    • DesignFast
    • Digital Issues
    • Engineering Week
    • Oscilloscope Product Finder
    • Podcasts
    • Webinars / Digital Events
    • White Papers
    • Women in Engineering
  • Videos
    • Teschler’s Teardown Videos
    • EE Videos and Interviews
  • Learning Center
    • EE Classrooms
    • Design Guides
      • WiFi & the IOT Design Guide
      • Microcontrollers Design Guide
      • State of the Art Inductors Design Guide
    • FAQs
    • Ebooks / Tech Tips
  • EE Forums
    • EDABoard.com
    • Electro-Tech-Online.com
  • 5G

Augmenting touchless gesture recognition with haptic feedback

September 21, 2016 By Aimee Kalnoskas 1 Comment

By Vicky Messer

Ever since the development of the trackball in the 1940s and the consumer release of the computer mouse in the 1980s, engineers have sought to push the limitations of computer-human interaction in an attempt to achieve a more intuitive and natural user experience. An obvious avenue to explore has been the gestures used in face-to-face communications, and the market has come a long way especially when considering developments in sophisticated audio and visual capabilities. The importance of the human sense of touch has also been championed with haptic feedback added to many consumer devices. Traditionally, however, the two markets have been distinct entities.

According to market analyst firm GIA, the global market for gesture recognition[1] will exceed $12.7 billion in 2020, driven by expanding applications in gaming, healthcare, automation, consumer electronics and automotive sectors. Consumer electronics (including gaming) currently accounts for approximately 99% of the gesture recognition market. A second report[2] predicts the market for both gesture recognition and touchless sensing technologies will have a CAGR of 28% over the next few years and is predicted to be worth $24 billion worldwide by 2020.

Touching and Touchless

Controlling devices with human gestures is, of course, now commonplace: the ability to swipe, pinch, push virtual buttons and type on touchscreens is provided by almost every type of electronic equipment. However, almost all require the physical touching of controls or surfaces. Clearly, this has limitations such as wear and tear of electrical switches and vulnerability to contamination – the classic example being spilt coffee on a computer keyboard. Some less obvious examples include the possibility of cross-infection from medical equipment controls or public toilets, or possible injury from hot or cold surfaces or sharp or electrically charged objects.

ultrahaptics
Touchless haptic hob control

Controlling equipment via human gestures without touching has become a growing area of interest to major technology companies over the past few years. The enormous global gaming market, in particular, has driven demand for the detection of body movements at a distance. However, a fundamental weakness of touchless controls is that they do not offer the desired level of precision and tactile feedback necessary to provide the best user experience. Users will prefer to feel a control or different textures and surfaces, and ideally also have audible confirmation. Effective simulation of tactile feedback – haptics – is, therefore, being seen as the key to unlocking the potential of touchless gesture recognition.

Tactile Feedback

The accuracy and usefulness of human gesture recognition is advancing rapidly and will continue to do so for decades to come. The issues associated with image-based systems are a focus of development efforts, including adaptability to varying lighting conditions and backgrounds and managing unwanted objects in the field of view. However, only seeing and hearing limits human perception and the ability to effectively control computers, which can be a significant problem particularly in safety critical applications. Simply recognising human gestures does not necessarily provide the tactile feedback experienced from physical controls such as switches and dials.

For example, even with the use of predictive text, there is the need for the user to check the screen to ensure the correct key has been pressed when using the virtual keyboard of a tablet, phone or touchscreen. Adding the third sense of touch provides feedback that a key, button or switch has been activated correctly and substantially enhances perception and control. Tactile feedback is even more important in some other applications such as vibrating the steering wheel to simulate the effect of driving over a rumble strip, which warns a distracted driver that their vehicle is drifting from its road lane. In some scenarios, tactile feedback will be an absolute necessity for the public acceptance of touchless gesture recognition.

Haptics

Haptic feedback is the sensation of touch and is already well known in computer gaming to provide a more immersive environment, and especially in various applications such as aircraft control where user attention is critical. Even flat-panel touchscreens can use programmed interference patterns between several tiny vibrators around the edge of the glass display to produce haptic feedback at the fingertips and simulate different surface textures.

Many new consumer products now feature haptic feedback – for example, major smart phone makers are now implementing vibrational haptic feedback in some of their latest models – generating lots of interest for numerous other applications worldwide. Some standard APIs (Application Programming Interfaces) are now available for research in the field of haptics, and several of these are being used to produce tactile sensations in simulations and training for remote and keyhole surgery such as laparoscopy and other medical applications.

Although tactile feedback provides a much more complete user experience – it closes the loop to provide helpful non-visual cues – almost all these approaches are limited by the need to be in physical contact with the haptic system. Therefore, they cannot be used in applications based on touchless gesture recognition at a distance.

Evaluation Kit

Providing touchless haptic feedback has been impossible until very recently. Ultrahaptics has developed a range of focused ultrasound techniques to augment recognition and control systems for hand gestures with touchless or mid-air haptic feedback. The company’s haptics evaluation kit – the only solution for touchless gesture recognition on the market today – includes a ‘leap motion’ camera and gesture recognition system. The kit allows developers to experiment and generate novel haptic sensations in touchless gesture control interfaces for a range of devices including computers, consumer electronics, domestic appliances and automotive dashboards.

ultrahaptics-figure-2
Touchless haptic feedback evaluation kit from Ultrahaptics

Recognizing human gestures and using them to control computers and equipment is well proven given the existing market of billions of smart consumer device and other electronic equipment. The advantages of not touching equipment are valuable, if not vital, in many use cases. Touchless gesture recognition has therefore become a well-established market sector and an area of academic and intense commercial research by major global corporations.

There is little doubt that many challenges remain in perfecting the user experience, including maximizing the range and field of view, and particularly in the accuracy of detection – not only of position and movement – but perhaps more importantly in the correct understanding of a user’s intention. In addition to incremental improvements from enhanced software algorithms, plus more accurate lower-cost and lower-power camera and sensor hardware, tactile feedback adds another new dimension to the accuracy of touchless gesture recognition in interpreting a user’s commands and confirming the correct action.

 

[1] Global Industry Analysts Inc. – Gesture Recognition, June 2015

[2] Markets and Markets – Gesture Recognition & Touchless Sensing Market, June 2015

Vicky Messer is the Technical Marketing Manager at Ultrahaptics

DesignFast Banner version: 03e1dd97

Filed Under: Aerospace & Defense, Applications, Automotive/Transportation, Consumer, Featured, Industrial, IoT, Medical, Sensors, Wearables Tagged With: ultrahaptics

Reader Interactions

Comments

  1. JacobJones says

    December 7, 2018 at 11:53 pm

    Great blog! Thanks for sharing the Information on gesture recognition haptic.. Please keep sharing, I look forward to read more..

    Log in to Reply

Leave a Reply Cancel reply

You must be logged in to post a comment.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

EE Training Center Classrooms

EE Classrooms

Featured Resources

  • EE World Online Learning Center
  • CUI Devices – CUI Insights Blog
  • EE Classroom: Power Delivery
  • EE Classroom: Building Automation
  • EE Classroom: Aerospace & Defense
  • EE Classroom: Grid Infrastructure
Search Millions of Parts from Thousands of Suppliers.

Search Now!
design fast globle

R&D World Podcasts

R&D 100 Episode 7
See More >

Current Digital Issue

April 2022 Special Edition: Internet of Things Handbook

How to turn off a smart meter the hard way Potential cyber attacks have a lot of people worried thanks to the recent conflict in Ukraine. So it might be appropriate to review what happened when cybersecurity fi rm FireEye’s Mandiant team demonstrated how to infiltrate the network of a North American utility. During this…

Digital Edition Back Issues

Sponsored Content

Positioning in 5G NR – A look at the technology and related test aspects

Radar, NFC, UV Sensors, and Weather Kits are Some of the New RAKwireless Products for IoT

5G Connectors: Enabling the global 5G vision

Control EMI with I-PEX ZenShield™ Connectors

Speed-up time-to-tapeout with the Aprisa digital place-and-route system and Solido Characterization Suite

Siemens Analogue IC Design Simulation Flow

More Sponsored Content >>

RSS Current EDABoard.com discussions

  • Photovoltaic MOSFET Drivers - Voltage Rating
  • UCC28070A controller ramp circuit implemented incorrectly?
  • Timing question on RX code
  • Frequency of FM transmitter not changing
  • Altium Routing: through vias being split into two blind vias. Additional question about blind via costs / benifits

RSS Current Electro-Tech-Online.com Discussions

  • software PWM
  • 200mv pulse to 12v lock
  • Background of Members Here
  • UCC28070A ramp circuit is wrong?
  • Treadmill board component burn repair

Oscilloscopes Product Finder

Footer

EE World Online

EE WORLD ONLINE NETWORK

  • 5G Technology World
  • Analog IC Tips
  • Battery Power Tips
  • Connector Tips
  • DesignFast
  • EDABoard Forums
  • Electro-Tech-Online Forums
  • Engineer's Garage
  • Microcontroller Tips
  • Power Electronic Tips
  • Sensor Tips
  • Test and Measurement Tips
  • Wire & Cable Tips

EE WORLD ONLINE

  • Subscribe to our newsletter
  • Lee's teardown videos
  • Advertise with us
  • Contact us
  • About Us
Follow us on TwitterAdd us on FacebookConnect with us on LinkedIn Follow us on YouTube Add us on Instagram

Copyright © 2022 · WTWH Media LLC and its licensors. All rights reserved.
The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media.

Privacy Policy