Human Pose and Action Recognition

Últimos artículos

• Pose-to-Biomechanics: Bridging 3D Human Pose Estimation and Biomechanical Attribute Prediction

  Ayda Eghbalian, Kevin Desai · 10 de julio de 2026

  Recent progress in 3D human pose estimation has made markerless recovery of skeletal motion increasingly accurate and scalable. However, most pose estimators remain optimized for geometric keypoint accuracy, while many real-world applications in rehabilitation, sports science, ergonomics, and clinic…

• Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control

  Jianjie Fang, Yongyan Xu, Ziyou Wang, Chen Gao, Yuchao Huang, Zhaolu Wang, Rongze Tang, Mingyuan Jia, Baining Zhao, Weichen Zhang, Xin Zhang, Haisheng Su, Yu Shang, Wei Wu, Xinlei Chen, Yong Li · 7 de julio de 2026

  World models are rapidly becoming a core infrastructure for embodied intelligence and interactive agents: they provide controllable simulators in which agents can perceive, act, forecast, and acquire scalable experience. Yet current video generation world models are still organized around isolated c…

• Domain Knowledge-Informed Self-Supervised Representations for Workout Form Assessment

  Paritosh Parmar, Amol Gharat, Helge Rhodin · 7 de julio de 2026

  Maintaining proper form while exercising is important for preventing injuries and maximizing muscle mass gains. Detecting errors in workout form naturally requires estimating human's body pose. However, off-the-shelf pose estimators struggle to perform well on the videos recorded in gym scenarios du…

• FuseMamba-VD: Dual Branch VideoMamba with Gated Class Token Fusion for Violence Detection

  Damith Chamalke Senadeera, Muhammad Awais, Shibo Li, Dimitrios Kollias, Gregory Slabaugh · 7 de julio de 2026

  The rapid proliferation of surveillance cameras has increased the demand for automated violence detection. While CNNs and Transformers have shown success in extracting spatio-temporal features, they struggle with long-term dependencies and computational efficiency. We propose FuseMamba-VD: Dual Bran…

• AViS-Mamba: Adaptive Visual Steering of Audio State-Space Dynamics for Violence Detection

  Damith Chamalke Senadeera, Dimitrios Kollias, Gregory Slabaugh · 7 de julio de 2026

  Automatic violence detection from video is challenging because violent interactions may be distant, occluded, or only partially visible. Audio can provide complementary evidence for violent events that are difficult to recognize from visual information alone. However, audio itself may be absent, dub…

• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models

  Alexander Popov, Alperen Degirmenci, David Wehr, Shashank Hegde, Ryan Oldja, Alexey Kamenev, Bertrand Douillard, David Nist\'er, Urs Muller, Ruchi Bhargava, Stan Birchfield, Nikolai Smolyanskiy · 7 de julio de 2026

  We propose the use of latent space generative world models to address the covariate shift problem in autonomous driving. A world model is a neural network capable of predicting an agent's next state given past states and actions. By leveraging a world model during training, the driving policy effect…

• WorldOdysseyBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models

  Ting-Bing Xu, Jiacheng Sui, Zhe Gao, Kewei Shi, Wenjin Yang, Zhicheng Liu, Zhaoxu Sun, Mingchao Sun, Hongyu Pan, Fan Jiang, Mu Xu, Qi Fan, Yang Gao, Yong Li, Baoquan Chen · 3 de julio de 2026

  Despite rapid progress in interactive world models (IWMs), existing benchmarks evaluate action following only at trajectory level and ignore memory and interaction physics. We introduce WorldOdysseyBench, an open-world benchmark for long-horizon stability across four dimensions, each with tailored i…

• Why Can't I Open My Drawer? Mitigating Object-Driven Shortcuts in Zero-Shot Compositional Action Recognition

  Geo Ahn, Inwoong Lee, Taeoh Kim, Minho Shim, Dongyoon Wee, Jinwoo Choi · 3 de julio de 2026

  Zero-Shot Compositional Action Recognition (ZS-CAR) requires recognizing novel verb-object combinations composed of previously observed primitives. In this work, we tackle a key failure mode: models predict verbs via object-driven shortcuts (i.e., relying on the labeled object class) rather than tem…

• KGS-GCN: Kinematics-Driven Gaussian Splatting and Probabilistic Topology for Skeleton-Based Action Recognition

  Yuhan Chen, Yicui Shi, Guofa Li, Liping Zhang, Jie Li, Jiaxin Gao, Wenbo Chu · 2 de julio de 2026

  Skeleton-based action recognition is widely applied in sensor-based systems, including human-computer interaction and intelligent surveillance. However, typical sensors produce sparse and discrete joint coordinates, often leading to the loss of fine-grained spatiotemporal information during dynamic …

• Learning When to Listen: Gated Affect Fusion for Human Motion Prediction

  Jingni Huang · 2 de julio de 2026

  Human motion forecasting in unconstrained real-world videos remains challenging due to the ambiguity of future behaviors and the presence of noisy multimodal observations. While facial affect potentially provides complementary behavioral cues, its practical utility and mechanistic boundaries within …

• EgoGapBench: Benchmarking Egocentric Action Selection in Multi-Agent Scenes

  Jihyeok Jung (KAIST AI), Jeewu Lee (Sogang University), Sanghyeop Kim (Sogang University), Chanhee Han (Ministry of Science and ICT), Seong Joon Oh (KAIST AI) · 2 de julio de 2026

  Existing egocentric benchmarks have primarily constructed the egocentric setting from first-person-view data, which makes it difficult to evaluate egocentric perspective itself in isolation. However, understanding first-person-view input and taking an egocentric perspective are separable abilities, …

• Partial Skeleton Visibility for Action Recognition: A Constrained Field-of-View Approach

  Yingjie Dai, Tianyang Xu, Yanglin Deng, Xiao-Jun Wu, Josef Kittler · 2 de julio de 2026

  Skeleton-based action recognition has achieved remarkable success by exploiting joint coordinates and their topological connections, yet prevailing methods overwhelmingly assume complete and clean skeleton inputs. In real-world deployments, such as egocentric vision, crowded surveillance, wearable d…

• SkillSpotter: Pose-Aware Multi-View Skilled Action Detection and Grading in Ego-Exo Videos

  Bj\"orn Braun, Christian Holz · 1 de julio de 2026

  To enable personalized, real-time coaching using Augmented Reality glasses or fixed camera setups in domains such as sports, cooking, or music, a system must understand not just what a person does, but how well they execute an activity. In an ego-exo video setting, this requires simultaneously detec…

• WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models

  Ting-Bing Xu, Jiacheng Sui, Zhe Gao, Kewei Shi, Wenjin Yang, Zhicheng Liu, Zhaoxu Sun, Mingchao Sun, Hongyu Pan, Fan Jiang, Mu Xu, Qi Fan, Yong Li, Baoquan Chen · 1 de julio de 2026

  Despite rapid progress in interactive world models (IWMs), existing benchmarks evaluate action following only at trajectory level and ignore memory and interaction physics. We introduce WorldRoamBench, an open-world benchmark for long-horizon stability across four dimensions, each with tailored inno…

• HAT-4D: Lifting Monocular Video for 4D Multi-Object Interactions via Human-Agent Collaboration

  Jiaxin Li, Yuxiang Wu, Zhenkai Zhang, Xinrui Shi, Haoyuan Wang, Yichen Zhao, Su Linxiang, Chenyang Yu, Mingyu Zhang, Yifan Ding, Boran Wen, Li Zhang, Ruiyang Liu, Yong-Lu Li · 29 de junio de 2026

  Extracting dynamic 4D object interactions from massive, in-the-wild monocular videos offers a highly efficient data collection pathway for scaling Embodied AI and training VLAs. However, existing monocular 4D reconstruction methods primarily focus on isolated objects, often failing under the severe …

• Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training

  Francis Xiatian Zhang, Hao Yao, Shengxuan Chen, Hong Zhu, Hongxiao Jia, Sisi Zheng, Hubert P. H. Shum · 29 de junio de 2026

  Vision-based assessment can provide convenient and cost-effective evaluation in Traditional Chinese Medicine (TCM) rehabilitation training, where action quality assessment (AQA) from computer vision offers a promising solution. Existing automatic AQA frameworks for physical therapy typically rely on…

• ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency

  Shih-Po Lee, Reza Ghoddoosian, Faizan Siddiqui, Enna Sachdeva, Behzad Dariush · 25 de junio de 2026

  An efficient and accurate system for detecting errors in procedural tasks is crucial for supporting human needs in daily life, as it can provide instant notifications and guide people to correct mistakes. In this work, we study real-time online error detection in procedural videos from a simple but …

• Robust Zero-Shot Generalization for Open-Vocabulary Action Recognition via Task Arithmetic

  Francesca Morandi, Omayma Moussadek, Federico Venturini, Mauro Suardi, Alessandro Banzatti, Francesco Cannarile, Angelo Porrello, Simone Calderara · 23 de junio de 2026

  Open Vocabulary Action Recognition (OVAR) enables the recognition of novel actions by leveraging vision-language representations, overcoming the limitations of traditional closed-set approaches. However, achieving robust performance in real-world scenarios typically requires domain-specific fine-tun…

• P-JEPA: Procedural Video Representation Learning via Joint Embedding Predictive Architecture

  Felix Tristram, Stefano Gasperini, Benjamin Killeen, Marcel Walch, Christian Benz, Nassir Navab, Ghazal Ghazaei · 23 de junio de 2026

  The increasing maturity of embodied AI platforms has driven a growing interest in procedural video representation learning to support intelligent assistance systems for complex, multi-step tasks. Leveraging large-scale latent predictive training, video foundation models capture video dynamics, enabl…

• MemoryVAM: Integrating Memory into Video Action Model for Robot Manipulation

  Yuxin Jiang, Chang Yu, Yunuo Chen, Xiang Feng, Yin Yang, Nishank Gite, Chenfanfu Jiang · 23 de junio de 2026

  Video-world-model policies learn action-relevant representations by predicting future observations. However, they condition on only a short observation window, which renders long-horizon manipulation non-Markovian when the correct action depends on earlier events that are no longer visible. We prese…

• EgoExo-Con: Exploring View-Invariant Video Temporal Understanding

  Minjoon Jung, Junbin Xiao, Junghyun Kim, Byoung-Tak Zhang, Angela Yao · 23 de junio de 2026

  Do Video-LLMs have consistent temporal understanding when videos capture the same event from different viewpoints? To study this question, we introduce EgoExo-Con(sistency), a benchmark of synchronized egocentric and exocentric video pairs with human-refined queries that ensure all concepts are visi…

• UNIEGO: Proxies as Mediators for Unified Egocentric Video Representation Learning

  Wenhao Chi, Arkaprava Sinha, Dominick Reilly, Hieu Le, Srijan Das · 19 de junio de 2026

  Egocentric video understanding is inherently limited by the narrow perspective of wearable cameras: a single viewpoint, a single modality, a single model cannot capture the full richness of human action. We argue that a truly expressive egocentric representation must subsume complementary knowledge …

• FlexLAM: Resolving the Bottleneck Trade-off in Latent Action Learning

  Takanori Yoshimoto, Yang Hu, Naruya Kondo, Tatsuya Matsushima · 19 de junio de 2026

  Latent actions provide a compact interface between action-free video and downstream decision-making, yet existing Latent Action Models (LAMs) force every transition through a fixed-capacity bottleneck. We identify a bottleneck trade-off: overly tight codes can discard transition cues needed for acti…

• The Reward Was in Your Data All Along: Correcting Flow Matching with Discriminator-Guided RL

  Nicolas Beltran-Velez, Felix Friedrich, Zhang Xiaofeng, Reyhane Askari-Hemmat, Xiaochuang Han, Adriana Romero-Soriano, Michal Drozdzal · 18 de junio de 2026

  Score- and flow-matching models often rely on preference-based reinforcement learning for two purposes: aligning with subjective preferences and, surprisingly, recovering properties such as visual realism and coherent object structure that matching-based training is intended to learn from the data i…

• VGGHeads: 3D Multi Head Alignment with a Large-Scale Synthetic Dataset

  Orest Kupyn, Eugene Khvedchenia, Christian Rupprecht · 18 de junio de 2026

  Human head detection, keypoint estimation, and 3D head model fitting are essential tasks with many applications. However, traditional real-world datasets often suffer from bias, privacy, and ethical concerns, and they have been recorded in laboratory environments, which makes it difficult for traine…