Project S18 – HTX Adaptive Gripper

The Home Team Science and Technology Agency (HTX), a statutory board under Singapore’s Ministry of Home Affairs, drives cutting-edge science and technology to bolster national security.

Within HTX, the Robotics, Automation, and Unmanned Systems (RAUS) Centre of Expertise develops robotic solutions that boost operational efficiency and decision-making.

What is the current problem?

1. Manually pick out items and arrange them neatly on a table

Climb a step ladder to capture a top-down photo of each item
Transfer the photos, then manually pair them with item descriptions and dimensions

Weigh each item and carefully transfer it into a bag before sealing

Over time, this not only slows down productivity but also contributes to fatigue and potential injury.

Our goal is to deeply understand and address these pain points — reducing risk, improving efficiency, and designing a system that supports the user rather than working against them.

How might we streamline the object handling and sorting process — reducing time and labour by 90%?

Categorising items by size into two distinct groups

Recording key attributes of each item, such as dimensions and visual references

Here's where Grip-X comes in

The process is tedious and repetitive easy.

Adaptive Gripper

Reverse Double Four Bar gripper to reliably grip objects of up to 1.75kg, due to high torque offered by a DC motor.

Precise handling of objects is enabled by a position control loop, opening the gripper jaw to a specific width before proceeding to pick up an object, thereby reducing the vertical deviation when the bar linkages rotate. After which, a pressure control mode ensures sufficient force when gripping the object, reducing incidences of slippage.

Grasping Algorithm

A novel algorithm was designed to circumvent the limitations of a single depth camera to generate grasp targets, even for cluttered scenes.

A height-displaced triangle mesh is generated, which valid grasps are generated against.

Size Estimation

With our triangulated camera setup (top + dual front angles), the booth can approximate object sizes through reconciling intrinsic and extrinsic matrices. The intrinsic matrix encodes the camera’s internal parameters, while the extrinsic matrix defines its position and orientation. Together, they map 3D world points to 2D image coordinates, enabling size estimation from images.

AI Labelling

The multi-angle collage is then fed through AI-powered object recognition using OpenAI’s vision model. By customising a prompt to output unstructured descriptions, a rich description can be recorded in the database.