Research & Development World

  • Home Page
  • Topics
    • Aerospace
    • Archeology
    • Automotive
    • Biotech
    • Chemistry
    • COVID-19
    • Environment
    • Energy
    • Life Science
    • Material Science
    • R&D Market Pulse
    • R&D Management
    • Physics
  • Technology
    • 3D Printing
    • A.I./Robotics
    • Battery Technology
    • Controlled Environments
      • Cleanrooms
      • Graphene
      • Lasers
      • Regulations/Standards
      • Sensors
    • Imaging
    • Nanotechnology
    • Scientific Computing
      • Big Data
      • HPC/Supercomputing
      • Informatics
      • Security
      • Software
    • Semiconductors
  • 2021 R&D 100 Award Winners
    • R&D 100 Awards
    • 2020 Winners
    • Winner Archive
  • Resources
    • Digital Issues
    • Podcasts
    • Subscribe
  • Global Funding Forecast
  • Webinars

New Technology Uses Mouth Gestures to Interact in Virtual Reality

By Binghamton University | October 6, 2017

Binghamton University Professor of Computer Science Lijun Yin and his team created a new framework that interprets mouth gestures as a medium for interaction within virtual reality in real-time.

Researchers at Binghamton University, State University of New York have developed a new technology that allows users to interact in a virtual reality environment using only mouth gestures.

The proliferation of affordable virtual reality head-mounted displays provides users with realistic immersive visual experiences. However, head-mounted displays occlude the upper half of a user’s face and prevent facial action recognition from the entire face. To combat this issue, Binghamton University Professor of Computer Science Lijun Yin and his team created a new framework that interprets mouth gestures as a medium for interaction within virtual reality in real-time.

Yin’s team tested the application on a group of graduate students. Once a user put on a head-mounted display, they were presented with a simplistic game; the objective of the game was to guide the player’s avatar around a forest and eat as many cakes as possible. Players had to select their movement direction using head rotation, move using mouth gestures and could only eat the cake by smiling. The system was able to describe and classify the user’s mouth movements, and it achieved high correct recognition rates. The system has also been demonstrated and validated through a real-time virtual reality application.

“We hope to make this applicable to more than one person, maybe two. Think Skype interviews and communication,” said Yin. “Imagine if it felt like you were in the same geometric space, face to face, and the computer program can efficiently depict your facial expressions and replicate them so it looks real.”

Though the tech is still in the prototype phase, Yin believes his technology is applicable to a plethora of fields.

“The virtual world isn’t only for entertainment. For instance, health care uses VR to help disabled patients,” said Yin. “Medical professionals or even military personal can go through training exercises that may not be possible to experience in real life. This technology allows the experience to be more realistic.”

Students Umur Aybars Ciftci and Xing Zhang contributed to this research.

The paper, “Partially occluded facial action recognition and interaction in virtual reality applications,” was presented at the 2017 IEEE International Conference on Multimedia and Expo.

Related Articles Read More >

Glowing rocks, a new sport, and what we can learn about different approaches
R&D winner of the day: Monolithic Fiber Array Launcher
R&D collaborations looking to build expertise, in this week’s R&D power index
Invention addresses the problems of running a red light at traffic intersections
2021 R&D Global Funding Forecast

Need R&D World news in a minute?

We Deliver!
R&D World Enewsletters get you caught up on all the mission critical news you need in research and development. Sign up today.
Enews Signup

R&D World Digital Issues

February 2020 issue

Browse the most current issue of R&D World and back issues in an easy to use high quality format. Clip, share and download with the leading R& magazine today.

Research & Development World
  • Subscribe to R&D World Magazine
  • Enews Sign Up
  • Contact Us
  • About Us
  • Drug Discovery & Development
  • Pharmaceutical Processing
  • 2022 Global Funding Forecast

Copyright © 2022 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search R&D World

  • Home Page
  • Topics
    • Aerospace
    • Archeology
    • Automotive
    • Biotech
    • Chemistry
    • COVID-19
    • Environment
    • Energy
    • Life Science
    • Material Science
    • R&D Market Pulse
    • R&D Management
    • Physics
  • Technology
    • 3D Printing
    • A.I./Robotics
    • Battery Technology
    • Controlled Environments
      • Cleanrooms
      • Graphene
      • Lasers
      • Regulations/Standards
      • Sensors
    • Imaging
    • Nanotechnology
    • Scientific Computing
      • Big Data
      • HPC/Supercomputing
      • Informatics
      • Security
      • Software
    • Semiconductors
  • 2021 R&D 100 Award Winners
    • R&D 100 Awards
    • 2020 Winners
    • Winner Archive
  • Resources
    • Digital Issues
    • Podcasts
    • Subscribe
  • Global Funding Forecast
  • Webinars