Self-guided Haptic Body Search Virtual Reality Training System

In collaboration with HTX Human Factors & Simulation Centre of Expertise (HFS CoE), this jointly-developed Virtual Reality (VR) application integrates haptic feedback gloves and Natural Language Processing (NLP), to provide a more immersive, interactive and effective learning experience for trainees undergoing training in the proper conduct of body search procedures

Haptics create an experience of touch by applying forces, vibrations or motions to the user

Team members

Chan Yi Xiang (EPD), Darren Chow De Jian (EPD), Hayden Ang Wei En (CSD), Katragadda Sahana (CSD), Mandis Loh Zhi Cheng (DAI), Sitoh Chin Weng Bryan (CSD), Sharryl Seto Shau Ru (CSD)

Instructors:

Lim Kwan Hui

Writing Instructors:

Delfinn Tan Sweimay

Why Self-guided Haptic Body Search Virtual Reality Training System?

Address Manpower & Logistics

A limited number of trainers having to conduct training sessions for an ever-increasing number of trainees

Reduce Human Subjectivity

Procedures are usually done from trainee to trainee. Different trainers might have different evaluations of the point of contact applied by the trainees

Provide Training Review

It is difficult and time-consuming for trainers to track or review mistakes done by the trainees

Our solution aims to address manpower & logistics issues, reduce human subjectivity and provide training reviews to achieve higher training productivity and better results for trainees learning the body search procedure

Comparison of our VR Training Against the Original Training Process

How we achieved this

How to use our Self-guided Haptic Body Search Virtual Reality Training System

Main Features

HAPTIC GLOVES

Our Customised Lucid VR Gloves integrated several subsystems such as ESP32 microcontroller finger tracking, finger pulling, buttons, and controllers

WHISPER AI (NLP)

The Whisper AI (NLP) transcribes spoken phrases and detects keywords. We then used this to adjust the digital avatar's movements accordingly, all without requiring an internet connection

VIRTUAL ANIMATION & HAND PHYSICS

The digital avatar mimics natural human movements. The VR hands emulate the user's hand movements. This realism then enhances the immersive experience

GRADES & RECORDINGS

Structured learning sessions are recorded, enabling trainees to review their mistakes on the computer for enhanced self-improvement

TRAINER INTERVENTION

See trainee's POV, track progress via virtual checklist on the computer, and guide trainee by marking completed tasks as necessary

Conclusion

Our VR application introduces an innovative system that blends the key aspects of body search procedures with an additional grading system feature. Combining the expertise of CSD, DAI, and EPD students, the design embodies a multi-disciplinary approach that considers the functionality, user comfort, operational requirements, and the needs of HTX to provide an efficient learning experience for trainees.

In partnership with :

Acknowledgements

Firstly, we would like to give special thanks to our Capstone mentor Mr. Lim Kwan Hui for the time and effort he provided throughout the project. Your advice and suggestions were really helpful to us during the project’s progress, as well as your guidance every week. We are very honored to have you as our mentor.

Secondly, we are incredibly grateful to Home Team Science and Technology Agency (HTX). You have been crucial to our project’s direction and progress and your valuable insights and feedback were key in refining the project for our clients.

Last but not least, thank you to the staff at the Capstone office and Dr Delfinn Tan from the Centre for Writing and Rhetoric (CWR). The wisdom given to us during consults on important reports and presentations has been invaluable. We have had such a smooth and fun learning experience.

We would like to thank everyone who has contributed to our project in one way or another, for all the time and effort you have put in for the whole duration of this project. Without you, we could not have made it thus far.

Days

Hours

Minutes

Seconds

Vote the project now!

Vote for our project at the exhibition! Your support is vital in recognizing our creativity. Join us in celebrating innovation and contributing to our success. Thank you for being part of our journey!