Human Pose and Action Recognition

Neueste Paper

• TraceGen: World Modeling in 3D Trace Space Enables Learning from Cross-Embodiment Videos

  Seungjae Lee, Yoonkyo Jung, Inkook Chun, Yao-Chih Lee, Zikui Cai, Hongjia Huang, Aayush Talreja, Tan Dat Dao, Yongyuan Liang, Jia-Bin Huang, Furong Huang · 27. November 2025

  Learning new robot tasks on new platforms and in new scenes from only a handful of demonstrations remains challenging. While videos of other embodiments - humans and different robots - are abundant, differences in embodiment, camera, and environment hinder their direct use. We address the small-data…

• SOAP: Enhancing Spatio-Temporal Relation and Motion Information Capturing for Few-Shot Action Recognition

  Wenbo Huang, Jinghui Zhang, Xuwei Qian, Zhen Wu, Meng Wang, Lei Zhang · 27. November 2025

  High frame-rate (HFR) videos of action recognition improve fine-grained expression while reducing the spatio-temporal relation and motion information density. Thus, large amounts of video samples are continuously required for traditional data-driven training. However, samples are not always sufficie…

• EvRainDrop: HyperGraph-guided Completion for Effective Frame and Event Stream Aggregation

  Futian Wang, Fan Zhang, Xiao Wang, Mengqi Wang, Dexing Huang, Jin Tang · 27. November 2025

  Event cameras produce asynchronous event streams that are spatially sparse yet temporally dense. Mainstream event representation learning algorithms typically use event frames, voxels, or tensors as input. Although these approaches have achieved notable progress, they struggle to address the undersa…

• From Observation to Action: Latent Action-based Primitive Segmentation for VLA Pre-training in Industrial Settings

  Jiajie Zhang, S\"oren Schwertfeger, Alexander Kleiner · 27. November 2025

  We present a novel unsupervised framework to unlock vast unlabeled human demonstration data from continuous industrial video streams for Vision-Language-Action (VLA) model pre-training. Our method first trains a lightweight motion tokenizer to encode motion dynamics, then employs an unsupervised act…

• GRAM: Generalization in Deep RL with a Robust Adaptation Module

  James Queeney, Xiaoyi Cai, Alexander Schperberg, Radu Corcodel, Mouhacine Benosman, Jonathan P. How · 26. November 2025

  The reliable deployment of deep reinforcement learning in real-world settings requires the ability to generalize across a variety of conditions, including both in-distribution scenarios seen during training as well as novel out-of-distribution scenarios. In this work, we present a framework for dyna…

• Evaluating the Performance of Deep Learning Models in Whole-body Dynamic 3D Posture Prediction During Load-reaching Activities

  Seyede Niloofar Hosseini, Ali Mojibi, Mahdi Mohseni, Navid Arjmand, Alireza Taheri · 26. November 2025

  This study aimed to explore the application of deep neural networks for whole-body human posture prediction during dynamic load-reaching activities. Two time-series models were trained using bidirectional long short-term memory (BLSTM) and transformer architectures. The dataset consisted of 3D full-…

• Uplifting Table Tennis: A Robust, Real-World Application for 3D Trajectory and Spin Estimation

  Daniel Kienzle, Katja Ludwig, Julian Lorenz, Shin'ichi Satoh, Rainer Lienhart · 26. November 2025

  Obtaining the precise 3D motion of a table tennis ball from standard monocular videos is a challenging problem, as existing methods trained on synthetic data struggle to generalize to the noisy, imperfect ball and table detections of the real world. This is primarily due to the inherent lack of 3D g…

• Tracking and Segmenting Anything in Any Modality

  Tianlu Zhang, Qiang Zhang, Guiguang Ding, Jungong Han · 26. November 2025

  Tracking and segmentation play essential roles in video understanding, providing basic positional information and temporal association of objects within video sequences. Despite their shared objective, existing approaches often tackle these tasks using specialized architectures or modality-specific …

• M2R2: MultiModal Robotic Representation for Temporal Action Segmentation

  Daniel Sliwowski, Dongheui Lee · 25. November 2025

  Temporal action segmentation (TAS) has long been a key area of research in both robotics and computer vision. In robotics, algorithms have primarily focused on leveraging proprioceptive information to determine skill boundaries, with recent approaches in surgical robotics incorporating vision. In co…

• FERA: Bridging the Semantic Gap in Foil Fencing via Kinematic Pose Recognition and Explainable Rule Reasoning

  Ziwen Chen, Zhong Wang · 25. November 2025

  Foil fencing presents a unique multimedia challenge: actions are extremely fast, interactions are subtle, and the final right-of-way decision relies on complex linguistic rules applied to visual data. We present FERA (Fencing Referee Assistant), a pose-based framework that bridges the gap between pi…

• MambaTAD: When State-Space Models Meet Long-Range Temporal Action Detection

  Hui Lu, Yi Yu, Shijian Lu, Deepu Rajan, Boon Poh Ng, Alex C. Kot, Xudong Jiang · 25. November 2025

  Temporal Action Detection (TAD) aims to identify and localize actions by determining their starting and ending frames within untrimmed videos. Recent Structured State-Space Models such as Mamba have demonstrated potential in TAD due to their long-range modeling capability and linear computational co…

• PriVi: Towards A General-Purpose Video Model For Primate Behavior In The Wild

  Felix B. Mueller, Jan F. Meier, Timo Lueddecke, Richard Vogg, Roger L. Freixanet, Valentin Hassler, Tiffany Bosshard, Elif Karakoc, William J. O'Hearn, Sofia M. Pereira, Sandro Sehner, Kaja Wierucka, Judith Burkart, Claudia Fichtel, Julia Fischer, Alexander Gail, Catherine Hobaiter, Julia Ostner, Liran Samuni, Oliver Sch\"ulke, Neda Shahidi, Erin G. Wessling, Alexander S. Ecker · 14. November 2025

  Non-human primates are our closest living relatives, and analyzing their behavior is central to research in cognition, evolution, and conservation. Computer vision could greatly aid this research, but existing methods often rely on human-centric pretrained models and focus on single datasets, which …

• ViPRA: Video Prediction for Robot Actions

  Sandeep Routray, Hengkai Pan, Unnat Jain, Shikhar Bahl, Deepak Pathak · 12. November 2025

  Can we turn a video prediction model into a robot policy? Videos, including those of humans or teleoperated robots, capture rich physical interactions. However, most of them lack labeled actions, which limits their use in robot learning. We present Video Prediction for Robot Actions (ViPRA), a simpl…

• ImitDiff: Transferring Foundation-Model Priors for Distraction Robust Visuomotor Policy

  Yuhang Dong, Haizhou Ge, Yupei Zeng, Jiangning Zhang, Beiwen Tian, Hongrui Zhu, Yufei Jia, Ruixiang Wang, Zhucun Xue, Guyue Zhou, Longhua Ma, Guanzhong Tian · 11. November 2025

  Visuomotor imitation learning policies enable robots to efficiently acquire manipulation skills from visual demonstrations. However, as scene complexity and visual distractions increase, policies that perform well in simple settings often experience substantial performance degradation. To address th…

• Countering Multi-modal Representation Collapse through Rank-targeted Fusion

  Seulgi Kim, Kiran Kokilepersaud, Mohit Prabhushankar, Ghassan AlRegib · 11. November 2025

  Multi-modal fusion methods often suffer from two types of representation collapse: feature collapse where individual dimensions lose their discriminative power (as measured by eigenspectra), and modality collapse where one dominant modality overwhelms the other. Applications like human action antici…

• Grounding Foundational Vision Models with 3D Human Poses for Robust Action Recognition

  Nicholas Babey, Tiffany Gu, Yiheng Li, Cristian Meo, Kevin Zhu · 11. November 2025

  For embodied agents to effectively understand and interact within the world around them, they require a nuanced comprehension of human actions grounded in physical space. Current action recognition models, often relying on RGB video, learn superficial correlations between patterns and action labels,…

• Co-Evolving Latent Action World Models

  Yucen Wang, Fengming Zhang, De-Chuan Zhan, Li Zhao, Kaixin Wang, Jiang Bian · 31. Oktober 2025

  Adapting pre-trained video generation models into controllable world models via latent actions is a promising step towards creating generalist world models. The dominant paradigm adopts a two-stage approach that trains latent action model (LAM) and the world model separately, resulting in redundant …

• EventFormer: A Node-graph Hierarchical Attention Transformer for Action-centric Video Event Prediction

  Qile Su, Shoutai Zhu, Shuai Zhang, Baoyu Liang, Chao Tong · 28. Oktober 2025

  Script event induction, which aims to predict the subsequent event based on the context, is a challenging task in NLP, achieving remarkable success in practical applications. However, human events are mostly recorded and presented in the form of videos rather than scripts, yet there is a lack of rel…

• BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning

  Hongyi Zhou, Weiran Liao, Xi Huang, Yucheng Tang, Fabian Otto, Xiaogang Jia, Xinkai Jiang, Simon Hilber, Ge Li, Qian Wang, \"Omer Erdin\c{c} Ya\u{g}murlu, Nils Blank, Moritz Reuss, Rudolf Lioutikov · 27. Oktober 2025

• What Do Latent Action Models Actually Learn?

  Chuheng Zhang, Tim Pearce, Pushi Zhang, Kaixin Wang, Xiaoyu Chen, Wei Shen, Li Zhao, Jiang Bian · 27. Oktober 2025

• GoRA: Gradient-driven Adaptive Low Rank Adaptation

  Haonan He, Peng Ye, Yuchen Ren, Yuan Yuan, Luyang Zhou, Shucun Ju, Lei Chen · 27. Oktober 2025

• Group Inertial Poser: Multi-Person Pose and Global Translation from Sparse Inertial Sensors and Ultra-Wideband Ranging

  Ying Xue, Jiaxi Jiang, Rayan Armani, Dominik Hollidt, Yi-Chi Liao, Christian Holz · 27. Oktober 2025

• Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videos

  Qixiu Li, Yu Deng, Yaobo Liang, Lin Luo, Lei Zhou, Chengtang Yao, Lingqi Zeng, Zhiyuan Feng, Huizhi Liang, Sicheng Xu, Yizhong Zhang, Xi Chen, Hao Chen, Lily Sun, Dong Chen, Jiaolong Yang, Baining Guo · 27. Oktober 2025