Hand Gesture Recognition Systems

Derniers papiers

• A Benchmark for Spatially Grounded Gesture Generation

  Anna Deichler, Rishabh Dabral, Fethiye Irmak Dogan, Anindita Ghosh, Jonas Beskow · 5 octobre 2026

  Communication in shared space interweaves verbal and non-verbal signals, and pointing gestures anchor language to the environment: "put the cup on that one" is uninterpretable without the gesture that fixes the referent. Yet no common framework exists for evaluating whether generated gestures indica…

• ReSCUE: Re-translation with Sentence Commitment for Unsegmented Long-Form Simultaneous Sign Language Translation

  Sihan Ren, Gaozheng Li, Yuanshang Quan, Yiming Qin, Fuyi Yang, Chang Liu, Lan Xu, Minye Wu · 5 octobre 2026

  Simultaneous Sign Language Translation (SLT) is critical for real-time communication, yet existing methods remain largely confined to sentence-level, offline settings that assume pre-segmented inputs. These assumptions hinder deployment in realistic scenarios involving continuous, unsegmented video …

• Simultaneous Translation between Sign Languages

  Zetian Wu, Bowen Xie, Stefan Lee, Liang Huang · 30 septembre 2026

  Deaf and hard-of-hearing (DHH) signers cannot converse in real time across different sign languages today: existing sign-to-sign translation systems run offline, requiring the full source clip before any target sign is emitted. Live use cases - e.g. broadcast interpretation and two-way video calls -…

• SignFLIP: A Unified Model for Sign Language Translation and Generation via Stage-wise Alignment at Scale

  Zhaoyi An, Sihan Tan, Youngbae Hwang, Kazuhiro Nakadai, Rei Kawakami · 30 septembre 2026

  Sign language translation and generation share the goal of bidirectional alignment between text and sign representations. However, existing approaches either treat them as isolated tasks or are only verified on limited datasets, limiting effective modeling between modalities. In this paper, we propo…

• OneSign: Unifying Sign Language Understanding Tasks with One Model

  Shiwei Gan, Yafeng Yin, Xiao Liu, Desibieer Tuerdaken, Lei Xie, Sanglu Lu · 29 septembre 2026

  SLU encompasses a diverse set of tasks, including ISLR, CSLR, and SLT. Although these tasks share basic semantic and linguistic foundations, they are typically addressed with task-specific architectures and training pipelines, which hinders knowledge sharing and requires costly pretraining and finet…

• Naturalness-guided Manifold Flow Matching for Sign Language Production

  Jiayi He, Shengeng Tang, Sisi You, Yanbin Hao, Lechao Cheng, Richang Hong · 29 septembre 2026

  Sign Language Production (SLP) aims to generate sign motions from text. Conditional Flow Matching methods have achieved strong performance in SLP by constructing conditional paths that transform a source distribution into a target distribution. However, existing methods construct these paths via lin…

• Seeing Semantic Shift: Difference-Aware Sentence-Level Temporal Segmentation of Sign Language Videos

  Bowen Guo, Shiwei Gan, Yafeng Yin, Xiao Liu, Kuizhuang Liu, Zhiwei Jiang, Lei Xie · 28 septembre 2026

  Recent advances in sign language understanding have achieved impressive success on short, single-sentence videos, yet their performance drops sharply when applied to long, continuous sign language videos. To bridge this gap, we focus on a challenging and realistic setting: Visual-only Sentence-level…

• SignTrace: Describe a Sign, Find the Word

  Zengji Tu, Xingye Zhu, Ningjing Wang, Tingyi Huang, Yangjunfeng Zhu, Dai Wan · 28 septembre 2026

  Identifying an unfamiliar sign is difficult when a learner remembers its movement but does not know its meaning or formal feature codes. SignTrace addresses this longstanding reverse-lookup problem through natural-language access to a Chinese sign-language dictionary. The system integrates LLM-based…

• SignMimic: Robust High-Quality Sign Language Motion Generation via Human-Shape-Oblivious Pose Transfer Guidance

  Zhewen He (New York University Abu Dhabi), Junyi Yu (New York University Abu Dhabi), Haomian Huang (New York University Abu Dhabi), Zhenhua Li (ChatSign Technology), Yi Fang (New York University Abu Dhabi, ChatSign Technology) · 25 septembre 2026

  We study the challenge of sign language video mimicking: given a driving video and a single reference frame, synthesize a video where the target signer reproduces the source motion while preserving identity and linguistic form. Prior pipelines entangle rigid motion, non-rigid deformation, and view-d…

• PHOSA: Photorealistic 3D Sign Avatar Modeling and Benchmark

  Haodong Wang, Hezhen Hu, Wengang Zhou, Houqiang Li · 25 septembre 2026

  In this work, we focus on photorealistic sign avatar modeling, which is crucial for effective communication with the Deaf community and is characterized by complex hand gestures and nuanced facial expressions. To this end, we introduce MVSign, the first multi-view Chinese sign language dataset co-de…

• Isolated Sign Language Recognition for Icelandic Sign Language: Experiments in a Low-resource Setting

  Finnur \'Ag\'ust Ingimundarson, Gu{\dh}n\'y Bj\"ork {\TH}orvaldsd\'ottir, Mathias M\"uller, Sarah Ebling · 23 septembre 2026

  We present the first experiments on isolated sign language recognition (ISLR) for Icelandic Sign Language (\'ITM). We use \'ITM SignWiki, a dataset derived from a bilingual Icelandic--\'ITM online dictionary. It is genuinely low-resource: 1,845 videos cover 849 classes, 86% of which have only two ex…

• Real-Time Hand Gesture Recognition for OpenXR Using Transformer-Based Machine Learning

  Salar Rezayani, Russell Butler · 23 septembre 2026

  Hand gesture recognition is a key component in human-computer interaction (HCI), enabling intuitive interfaces for applications in gaming, virtual reality (VR), robotics, and more. This study integrates transformer-based machine-learning models for real-time hand gesture recognition, using hand-trac…

• Bimanual 3D Hand Motion and Articulation Forecasting in Everyday Images

  Aditya Prakash, Richard Li, David Forsyth, Saurabh Gupta · 22 septembre 2026

  We tackle the problem of forecasting bimanual 3D hand motion and articulation from a single image in everyday settings. To address the lack of 3D hand annotations in diverse settings, we design an annotation pipeline consisting of a diffusion model to lift 2D hand keypoint sequences to 4D hand motio…

• SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generation

  Ronghui Li, Jun Dong, Zhongyuan Hu, Zunnan Xu, Jun Zhou, Liyuan Chen, Shuoling Liu, Jiangpeng Yan, Jie Guo, Xiu Li, Linchao Bao · 21 septembre 2026

  Large language models (LLMs) provide limited support for sign language interaction. Unifying sign language translation (SLT) and generation (SLG) to enable sign language as both input and output can reduce switching between separate models during sign-text interaction. We present SignGPT, a unified,…

• Learning Sign Language Recognition under Label Noise: A Study of Noise-Robust Losses for Isolated and Continuous Settings

  Akihisa Shitara, Yoichi Ochiai · 14 septembre 2026

  In sign language recognition, the isolated (ISLR) classification loss treats a single label as ground truth, as does the frame-level auxiliary classifier over pseudo-labels we add to continuous (CSLR) methods, which lack one. Stylistic variation blurs ISLR annotation and the lack of temporal boundar…

• Investigating Temporal Motion Features for Pose-to-Text Indian Sign Language Translation

  Manav Dhamecha, Praveen Kumar Chandaliya, Pruthwik Mishra · 14 septembre 2026

  We investigate the effect of pretrained T5 model scale and explicit motion features on pose-to-text Indian Sign Language Translation (SLT) for the WSLP 2026 Shared Task. Pose sequences are projected into the embedding space of T5 through a lightweight pose encoder, with the complete model fine-tuned…

• Prompting with Sign Parameters for Low-resource Sign Language Instruction Generation

  Md Tariquzzaman, Md Farhan Ishmam, Saiyma Sittul Muna, Md Kamrul Hasan, Hasan Mahmud · 11 septembre 2026

  Sign Language (SL) enables two-way communication for the deaf and hard-of-hearing community, yet many sign languages remain under-resourced in the AI space. Sign Language Instruction Generation (SLIG) produces step-by-step textual instructions that enable non-SL users to imitate and learn SL gesture…

• RAIDAL: Redundancy-Aware Information Density Active Learning for CTC-Based Continuous Sign Language Recognition

  Rafael A. Diniz Augusto, Gabriel L. Oliveira, Erickson R. Nascimento · 11 septembre 2026

  Continuous sign language recognition (CSLR) is a key technology for accessibility, yet its development remains limited by the high cost of annotating continuous video streams. Active learning offers a path toward mitigating this cost, but standard acquisition functions are not designed for weakly al…

• Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation

  Keren Artiaga, Sabyasachi Kamila, Haithem Afli, Conor Lynch, Mohammed Hasanuzzaman · 9 septembre 2026

  Sign Language Translation has advanced with deep learning, yet evaluations remain largely signer-dependent, with overlapping signers across train/dev/test. This raises concerns about whether models truly generalise or instead rely on signer-specific regularities. We conduct signer-fold cross-validat…

• SeRV: Semantic-Aligned Residual Vector Quantization for American Sign Language Generation

  Hongyu Wu, Xu Wu, Tianhao Wu, Jiawei Yu, Phuc Nguyen, Jian Liu, Yi Wu · 9 septembre 2026

  American Sign Language (ASL) generation remains challenging due to limited paired text-ASL motion data and the difficulty of learning motion representations both precise for reconstruction and predictable from linguistic input. Existing methods rely on motion tokenizers optimized for reconstruction,…

• SignSeek: Learning Transferable Representations for Sign Dictionary Retrieval

  Sobhan Asasi, Ozge Mercanoglu Sincan, Richard Bowden · 4 septembre 2026

  Sign language dictionaries are essential resources for sign language learners, yet automatically retrieving a sign from a dictionary, given only a query video, remains a challenging problem due to the natural variability between signers. Existing sign representation learning methods are built for cl…

• A Reverse Sign Language Dictionary: Open-Vocabulary Sign Recognition from Continuous Signing via Video Captioning and Description Retrieval

  Santiago Poveda-Guti\'errez, Hideki Nakayama, Mayumi Bono · 4 septembre 2026

  Isolated Sign Language Recognition (ISLR) is conventionally cast as closed-set classification over gloss labels, which cannot generalize to signs unseen in training and ties every deployment to a gloss-annotated lexicon. We instead recognize signs extracted from continuous signing by (1) captioning …

• Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

  Oline Ranum, Edward Fish, Simon Hadfield, Richard Bowden · 4 septembre 2026

  BLEU-4 is the standard metric for evaluating sign language translation (SLT), but spoken-language metrics may not adequately reflect sign language proficiency. The multimodal, low-resource context of SLT allows models to exploit spurious correlations and spoken-language priors, rather than learning …

• SignRR: Retrieve and Refine Real Motion for Sign Language Production

  Fidel Omar Tito Cruz, Angie Sanchez Marquina, Summy Farfan, Gissella Bejarano · 31 août 2026

  Sign language production (SLP) aims to generate continuous signing motion from spoken language, often through gloss-to-pose generation. Prior work mainly follows two paradigms. Generative models synthesize motion from a learned prior or from noise, without reference to an observed signing instance, …

• SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting

  Eunjee Choi, JungHoon Sung, Seongwhan Cho, Chu Xin, Younggeun Choi · 27 août 2026

  Continuous sign language recognition (CSLR) aims to recognize gloss sequences from unsegmented sign videos under weak sequence-level supervision. However, existing methods rely on sentence-level gloss annotations, providing limited temporal and semantic guidance for fine-grained representation learn…