Sign Language Translation Demo

This interactive demo showcases automatic sign language translation from video to text.

The current version demonstrates the processing of American Sign Language (ASL) using modern computer vision and deep learning methods.

How to use the demo

1. Upload a video or select one of the provided examples.
2. Click Submit to process the video.
3. The predicted text will be displayed as the output.

For best results, the signer should be clearly visible, including the upper body, hands, and face.

Note: This is a research demonstrator. Predictions may not always be accurate, especially for videos that differ significantly from the data used during training.

How does it work?

The system consists of two main stages: pose preprocessing and sign language translation.

Pose preprocessing

The input video is first converted into a structured pose representation. Keypoints describing the signer's body, hands, and face are extracted and normalized before being passed to the translation model.

Body keypoints are normalized globally, while hand and facial keypoints are normalized locally to preserve detailed information about their shape and movement.

More information about the preprocessing pipeline is available in the PoseEstimation repository.

Sign language translation

The extracted pose sequence is processed using a model based on Uni-Sign, a unified framework for sign language understanding.

More information about the model and our implementation is available in the Uni-Sign repository.

Resources

• Demo source code
• Pose estimation and preprocessing
• Uni-Sign implementation
• Uni-Sign: Uni-Sign: Toward Unified Sign Language Understanding at Scale, Li et al., ICLR 2025

Acknowledgements

This demonstrator was developed at the University of West Bohemia (ZČU), Department of Cybernetics, Computer Vision group.

Development was supported by the 2026 ZČU internal mini-project programme for the development and wider use of artificial intelligence.

The system builds upon the Uni-Sign framework. We thank its authors and the open-source sign language research community for making their work publicly available.