Can we trust them? Let's find out...
Motivation
During investigative interviews, investigators need to balance questioning the suspect and verifying the credibility of the suspect’s statement. This results in high mental workload for investigators that can result in missing of information critical to solving the case. Furthermore, current credibility assessment tools are often bulky, invasive and cumbersome, making them impractical for most investigation cases.
Current Investigative Interview Practice
Investigative Interviews are often long, complex and tiring. Here are the 3 main stages.
- Investigators engage with the suspect by asking questions, taking down notes and building rapport
- Investigators review and analyze the notes taken during the interview
- Decide if they would require a follow up interview to confirm some suspicions
- There is no reviewing of the interview in its entirety
- If deemed necessary, follow up interview with the suspect is carried out
- Polygraph test used to determine the suspect's credibility
Key Challenge: Interview Process
During the investigative interview process, investigators face many key challenges. Here are just 2 of the many.
High Mental Workload
Investigators have to juggle multiple tasks during investigative interviews. Such tasks include:
- Questioning the suspect
- Noting down responses
- Observing facial, verbal, and body cues to assess credibility
- Coming up with questions to drive the interview
Uncooperative Behaviour
Sometimes, the suspect may display uncooperative behaviour during interviews. Such behaviour include:
- Refusal to answer questions
- Shouting
- Attempts to deceive and misleading investigators
- Feigning ignorance
Key Challenge: Current Technology
The polygraph is the current tool that is used in investigative interviews to assess truthfulness or deception by measuring physiological changes in a person’s body. The polygraph has its limitations, and of them examples are:
- Imposing and intimidating
- Long training period of up to 6 months to train someone to use the polygraph
- Bulky and combersome, requires a long time to set up
- Probes are in contact with the suspect which is invasive
- Complex graphs generated by the polygraph requires specialized training to interpret
Problem Statement
How might we create an integrated credibility aid that is contactless, discreet and accessible to augment the current common interview practices?​
Key Requirements for our Solution
To better customize our solution to investigators, our solution must be:
Prospero's system and sensors cannot touch the suspect during operation
Prospero should not interrupt the workflow of the investigators during the interview and should not be easily identifiable by the suspect
Prospero's system can be set up seamlessly and the outputs should be easily understandable by investigators
Background: Physiological Indicators
Literature reviews indicate that both facial expressions and speech patterns provide valuable insights into a person’s credibility. Micro-facial expressions can be used to capture facial cues, while speech patterns can be analyzed using Layered Voice Analysis. Let’s dive into them.
Micro-Facial Expressions (MFE)
What are MFEs?
MFEs are quick and subtle facial expressions that last for up to half a second. Due to their spontaneous nature, they are usually expressed when individuals try to hide their genuine emotions. Giving investigators access to MFE data allows them to make inferences from context about what a suspect might be concealing.
Facial Action Coding System (FACS) is a comprehensive and objective method to measure and classify facial movements developed by Paul Ekman. It works by breaking down each expression into specific muscle movements known as Action Units (AUs).
What are AUs?
AUs are the movement of a facial muscle or muscle groups that make up an emotion.Â
They enable the capture of precise descriptions of all expression types.Â
We developed an AI-powered automated MFE detection system which maps images to their corresponding AUs
Layered Voice Analysis (LVA)
LVA is a patented and proprietary tool developed by Nemesysco that analyzes vocal patterns.
Features
- Content and language independent
- Focuses on abormal speech patterns
- Detects peaks and plateaus from voice waveforms
- Generates data on deception risk based on audio analysis
- Generates a coloured bar which displays the perceived level of deception risk shown throughout the interview
Most helpful feature: The Coloured Bar
Different colours correspond to different levels of deception risk, and the most important colours are:
- Orange indicates medium deception risk
- Red indicates high deception risk
Investigators look out for the orange and red coloured segments to help them ascertain whether someone has deceptive intentions. We seek to integrate LVA into our solution.
So, how are we solving the problem?
Hardware Solution: SpyFlask
Software Solution: Prospero Analyser
Introducing: SpyFlask
Now that we know what to indicators to analyse and how we are analysing them, we need a way to get the data in an accurate and optimal way.
Technical Requirements to meet
- MFE requires a clear image of the interviewee’s face
- LVA requires a clean audio signal of only the interviewee, since having the interviewer’s audio could corrupt the analysis
Features
- Video records at 720p, 60fps
- Audio recorded using directional microphones
- Ensures we have noise isolation of only the interviewee for accurate LVA analysis
- Live streaming of audio and video feed via RTSP and WebRTC protocol
- Functions on a Local Area Network
- Computer vision model to identify and track interviewee's face
- Motorized gimbal system to rotate the camera module
- Ensures interviewee's face is centered for accurate MFE analysis
Introducing: Prospero Analyser
Prospero Analyser, our frontend, is a visualization tool to visualize recorded interviews from SpyFlask and analyses from MFE and LVA, empowering interviewers to move beyond raw data.
Start Recording Live Interviews
- Interviewers can select their video and audio capture device as SpyFlask or an integrated/external webcam and microphone
- Starting the feed of live interview from the selected capture device is as simple as pressing a "Start Recording" button
End Recording Live Interviews
- Ending recording is as simple as pressing the "Stop Recording" button
- Doing so will automatically execute MFE and LVA analysis on the recording
- A progress bar on the analysis is clearly displayed
View Interview Analysis Data
- The video of the interviewee is analyzed with our MFE microservice to detect MFEs and its AUs exhibited throughout the interview
- The interviewee's audio is analyzed with the LVA microservice obtain deception risk levels exhibited throughout the interview
- The MFE and LVA data are displayed with intuitive and interactive UI components, giving interviewers an easy way to obtain a second opinion from
Managing All Interview Analyses
- All saved interview recordings and analyses are displayed
- Interviewers can search and filter them for easy referencing
- They can also manage the interview analyses by editing details or deleting them
Overall System Architecture
The video and audio inputs are live streamed from SpyFlask to Prospero Analyser. It is the bridge between our hardware and software. Below, the detailed system architecture shows the technologies we used and interactions between our individual components as well as the data flow path.
Testing and User Reviews
We devised a testing plan aimed to evaluate how well Prospero answered our Problem Statement. Here is what our users had to say.
Enhancement: How does Prospero add value to the current interview practices and post-interview analysis?
Prospero was found to have provided the convenience of a combined analysis as well as a second opinion
Discretion: How disruptive is Prospero during interviews?
Prospero was deemed to be unobtrusive and the typical interview scenario flow was not disrupted
Accessibility: Can investigators use Prospero with minimal training?
Interviewers were able to use Prospero in their interviews with ease and could setup everything on their own
Prospero's Value Proposition
Why choose us?
-
Second opinion
-
Natural questioning
-
Maintains interview procedure
-
Easy setup and use
-
Discreet and sleek
In addition to their initial judgement, investigators can rely on Prospero to give them a second opinion.
The burden of looking out for fleeting facial expressions and listening for vocal cues is offloaded, allowing investigators to engage in conversation with the suspect more naturally.
Prospero does not disrupt current interview processes. It just sits on the table, silently recording information.
Prospero has an easy start up procedure - all we need is a click of a button and in the blink of an eye, the live feed is shown on Prospero Analyzer.
Prospero's recording capabilities are well obscured by the disguise of an everyday object, a bottle.Â
All in all, Prospero brings automatic streaming with a combined analysis that empowers interviewers and most importantly, continues to enable human connection.
