CrossTalk Secure

Bridging language gaps in Southeast Asia with secure speech translation

Introducing CrossTalk Secure

CrossTalk Secure is a compact, portable offline speech-to-speech translation device designed to deliver accurate, low-latency translations while ensuring user privacy and security. Tailored for security personnel operating in Southeast Asia, it facilitates seamless and secure communication with foreigners, helping to overcome language barriers in an interconnected world where tourism is on the rise.

Team members

Abby Wong Yik Yung (EPD), Caitlin Wong Yu-Lin (ESD), Joel Luke Tan Yi (EPD), Low Wei Kang, Keith (CSD), Luv Singhal (CSD), Yeo Jing Zhe (Yang Jingzhe) (ASD)

Instructors:

  • Lim Kwan Hui

Writing Instructors:

  • Rashmi Kumar

  • Bernard Tan

Problem background

Singapore’s growing tourism industry has led to an increase in tourist influx, including a surge in non-English speaking visitors. This presents challenges in ensuring a smooth and enjoyable visitor experience, as language barriers can create communication gaps between locals and international tourists. Additionally, navigating Singlish—Singapore’s unique blend of English, local slang, and dialects—can be difficult, even for tourists who have some knowledge of English, further complicating interactions and understanding. Addressing these communication challenges is essential to enhancing the overall experience for visitors from around the world.

How might we best allow security personnelto communicate with foreignerseffectively and conveniently?

 
 
 

Key Requirements

Product Information

With careful consideration of the key requirements, CrossTalk Secure was developed to support offline translation for a selection of Southeast Asian languages and those commonly spoken in Singapore, facilitating seamless conversation between a pair of users conversing comfortably in their own languages without jeopardising their privacy.

Overview
Software
Hardware

Through extensive research into various components, we have designed an integrated solution that addresses key market gaps in existing products—specifically, the lack of secure offline translation for Southeast Asian languages. By leveraging AI edge computing, our approach eliminates the need for internet connectivity, ensuring secure and reliable translations for users in the region.

This approach presents a unique challenge: balancing powerful computational requirements with portability. To achieve optimal performance, we carefully considered hardware components such as battery capacity and cooling, along with the AI model pipeline.

When activated, the device delivers real-time transcription of spoken dialogue, text-to-text translation, and audio output of translated text in eight supported languages: English, Mandarin Chinese, Malay, Hindi, Indonesian, Tagalog, Thai, and Vietnamese.

Its split-screen design enables both users to view the transcribed conversation in their chosen language simultaneously, enhancing comprehension and ensuring accurate communication.

Our speech-to-speech translation system uses a step-by-step approach, where each part of the process is handled by a specialised model.

First, the system detects when someone is speaking. Then, it detects the language and transcribes their speech into text. Next, the text is broken into sentences and translated into the target language by sentence. Finally, the translated text is spoken aloud using a voice synthesis model when there is a lull in the conversation. This structured approach ensures lower latency and flexibility compared to a single all-in-one model.

Our customized integrated hardware solution delivers a compact form factor while combining performance, efficiency, and security.

The compute unit handles intensive processing entirely offline and locally, ensuring high security without relying on cloud services or Wi-Fi. The audio board is designed for clear speech capture, enhancing translation accuracy, while the speaker ensures clear audio output. A custom cooling system maintains peak performance under demanding workloads, and the touchscreen provides an intuitive user experience. With a battery capable of 3.5 hours of continuous AI-powered speech translation, our meticulously integrated design offers a reliable, low cost, high-performance solution for seamless communication.

 

Performance

 

To evaluate the translation accuracy of our speech-to-speech system, we used sacreBLEU, an industry-standard metric for assessing machine translation quality. We ran tests on conversational audio data, simulating real-time input as if streamed directly from a microphone. The updated pipeline achieved a 2.25x improvement in BLEU score compared to the baseline, reflecting significantly more accurate translations. This gain is particularly important in conversational settings, where preserving meaning and nuance in real-time dialogue is critical to ensure effective communication.

 

 

 

 

 

 

 

 

Additionally, we achieved a 4.8x reduction in model inference time across the speech-to-speech pipeline, enabling translations to occur much closer to real-time. This improvement is crucial for seamless and efficient communication, especially in fast-paced conversational scenarios where delays can disrupt the natural flow of dialogue.

User Testing Results

During user validation testing, pairs of participants used CrossTalk Secure to facilitate conversations in English and one target language. To assess perceived performance in areas such as audibility, transcription accuracy, translation accuracy, and response speed, each participant rated these factors on a scale from 1 to 5.

User ratings ranged from 3.4 to 4.4 on average, with a mean score of 3.8 out of 5—indicating generally satisfactory perceived performance.

Participants also provided feedback on ease of use and overall satisfaction through a short questionnaire, offering a broader view of subjective usability. This resulted in a mean System Usability Scale (SUS) score of 68.4 out of 100, which falls within the moderate usability range.

In partnership with :

Acknowledgements

We would like to express our sincere gratitude to several individuals who have been instrumental supporting us throughout this ongoing project. First, we extend our deepest thanks to KLASS Engineering & Solutions, our industry mentor, for providing the opportunity to work on this project. Additionally, we would like to thank Toh Si Hui, Terrance Goh, Lu Zheng Hao, Nicholas Chan and Syed Asif Bin Ascar for their invaluable help and guidance throughout the process.

We are also extremely grateful to Prof. Lim Kwan Hui and Prof. Rakesh Nagi for their exceptional support. Their technical insights and guidance were crucial in shaping the project, and their assistance in refining both the report and presentation flow was invaluable.

Additionally, we would like to thank Dr. Bernard Tan and Ms Rashmi Kumar for their meticulous review and keen eye, which helped us significantly improve the presentation and showcase material for this project.

Their collective support and expertise have been fundamental to the success of this project, and we are deeply appreciative of their contributions.

Menu

ornament-menu

Contact the Capstone Office :

+65 6499 4076

8 Somapah Road Singapore 487372

Please fill in your information below and feedback

Contact the Capstone Office :

8 Somapah Road Singapore 487372

8 Somapah Road Singapore
487372

Welcome back!

Log in to your existing account.

Contact the Capstone Office :

+65 6499 4076

8 Somapah Road Singapore 487372

Welcome back!

Log in to your existing account.