Surgical Simulation and Training

Latest papers

• CasCVS-Net: A Staged Multi-Task Cascade for Critical View of Safety Assessment

  Bock-Zien Toh, Yuanchuan Ren, Tay Aw Yu, Ng Khee Ong, Zhehua Mao, Sophia Bano · 24 September 2026

  Automated assessment of the Critical View of Safety (CVS) in laparoscopic cholecystectomy requires both recognition of the three CVS criteria and anatomical grounding in small, rare, and often occluded hepatocystic structures. Learning-based methods differ in the anatomical information they use, fro…

• A Hierarchy-Aware Video-Language Model Evaluation and Hyperbolic Baseline for Surgery

  Ana Manzano Rodr\'iguez, Pascal Mettes, Marlies P. Schijven, Cees G. M. Snoek · 24 September 2026

  Surgical procedures follow a phase-to-step hierarchy, yet the video-language models used to recognize them are evaluated with flat per-level metrics that ignore cross-level coherence and error structure. In this paper we make two contributions to address this problem, (i) we introduce SurgHiBench, t…

• Video-based Surgical Skill Assessment Using Dynamics-and-Uncertainty-Aware Tree-based Gaussian Process Classifier

  Arefeh Rezaei, Mohammad Javad Ahmadi, Amir Molaei, Hamid D. Taghirad · 22 September 2026

  The proposed pipeline integrates a representation-flow convolutional neural network with a dynamics- and uncertainty-aware tree-based Gaussian Process classifier. In this framework, latent motion dynamics are exploited both as discriminative representations and as a source of input uncertainty, enha…

• LD-RSVIS: A Large-Scale and Diverse Benchmark for Referring Surgical Video Instrument Segmentation

  Zan Wang, Yunhe Feng, Dong Nie, Oluwatosin Oluwadare, Kewei Sha, Yan Huang, Heng Fan · 22 September 2026

  Referring surgical video instrument segmentation (RSVIS) aims at segmenting the instrument in a surgical video, given a textual description. Despite recent progress, current models are trained and assessed on relatively small-scale benchmarks, hindering the development of more general RSVIS. In addi…

• SIRA: Reasoning-Aware Surgical Instrument Segmentation via Query-Anchored Alignment

  Zhibo Zhang, Qijie Wang, Zengqiang Yan · 21 September 2026

  Surgical instrument segmentation (SIS) plays a critical role in robotic assistance and surgical workflow analysis. However, most existing SIS methods formulate segmentation as a category-driven localization problem, limiting their ability to capture procedural context and task-dependent semantics in…

• LapaTrack-3D: 6 DoF pre-operative shape tracking for laparoscopic surgery

  Jingwei Song, Javid Hussain Jakir, Ray Zhang, Wenwei Zhang, Hao Zhou, Xiaomeng Xian, Maani Ghaffari · 18 September 2026

  This work proposes a real-time 6 Degree-of-Freedom (6 DoF) tracking algorithm for monocular laparoscopic surgery. It provides alignment between intra-operative video and pre-operative data (e.g., CT). The 6 DoF tracking offers a solution for accurately locating the internal anatomy of the target org…

• RAUL: Reference-Assisted Ureteroscopy Localization for Skill Assessment

  Fangjie Li, Mai Bui, Charan Mohan, Michael Miga, Matthieu Chabanas, Nicholas Kavoussi, Jie Ying Wu · 18 September 2026

  Objective: Incomplete navigation of anatomy during ureteroscopic kidney stone surgeries can contribute to repeat interventions. While skilled surgeons have lower reintervention rates, there are no objective metrics to quantify scope-navigation performance to evaluate when a trainee becomes skilled. …

• TEDi: Temporal Memory-Enhanced and Denoising Transformer for Surgical Instrument Segmentation

  Jiahong Yuan, Weiming Mi, Tao Zhang, Haoyin Zhou · 16 September 2026

  Query-based segmentation methods have shown promising potential for surgical instrument segmentation and recognition, which is essential for scene understanding and downstream tasks in computer assisted surgery. However, most existing approaches predominantly rely on per-frame predictions and overlo…

• OphBiWSSD: Scaling Temporal Action Localization in Ophthalmic Surgeries with Bidirectional Weight-tied State Space Duality

  Yang Liu, Qionghong Ma, Joongwon Chae, Lihui Luo, Yibing Shen, Yulin Zhuo, Yingting Zhu, Jiashu Chang, Xiaoyun Zhong, Dongmei Yu, Peter E. Lobie, Peiwu Qin, Chengming Yang · 14 September 2026

  High-frequency surgical maneuvers in ophthalmology necessitate high-fidelity temporal modeling, yet characterizing long-range procedural dependencies remains computationally prohibitive for attention-based architectures. Existing models often require aggressive temporal downsampling, which compromis…

• A Voice-Interactive Multi-Agent System for Smart Operating Rooms: Architecture Design and Key Technologies

  Tianxiang Zhou · 11 September 2026

  This paper presents SurgicalRoomAgent, a voice-interactive multi-agent system for smart operating rooms based on large language models (LLMs). The system achieves natural language understanding, device control, intraoperative recording, and surgical report generation through a layered architecture c…

• ReGround-Surg: Reliability-Guided Anchor Grounding for Referring Surgical Video Segmentation

  Jiaxin Wen, Ming Yin, Lu Liu, Zeyu Fu · 26 August 2026

  Referring surgical video segmentation requires segmenting a target instrument or tissue region across video frames according to a natural language expression. Recent Segment Anything Model 2 (SAM2) based two-stage methods (e.g., ReSurgSAM2) first ground the referred target in an initial or selected …

• Hierarchical Prototype-Memory Adaptation of SAM for Surgical Instrument Segmentation

  Xinning Yao, Jingjing Wang, Jinghua Yue, Xiaoyan Luo, Fugen Zhou, Bo Liu · 26 August 2026

  Surgical instrument segmentation (SIS) is fundamental for computer-assisted surgery, where reliable instrument masks enable precise scene understanding and clinical assistance. Recently, adapting foundation models like the Segment Anything Model (SAM) to the surgical domain via prompt-learning has s…

• Constructing Predictive Surgical Path for AI-based Capsulorhexis Skill Transfer

  Mohammad Javad Ahmadi, Hamid D. Taghirad · 25 August 2026

  Automated training of surgeons is one of the most crucial factors that significantly minimize surgical training risks and expenses. With recent advances in artificial intelligence (AI) knowledge and available data from various surgeries, AI's involvement in surgical training is becoming very promisi…

• Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning

  Weiliang Huang, Huanrong Liu, Bob Zhang, Qi Dou, Zhen Chen, Yun Gu, Guy Rosman, Qingbiao Li · 21 August 2026

  Reliable surgical planning requires models to anticipate not only how instruments will move, but also how the operative visual state will evolve together with such motion. Existing approaches typically treat future scene generation and instrument trajectory prediction as two separate tasks. Scene-on…

• Artificial Intelligence for Workflow Analysis in Colorectal Surgery: A Multicentric, Cross-Procedural Development and Generalization Study

  Pietro Mascagni, Julia Alekseenko, Pooja P Jain, Marta Goglia, Andrea Balla, Ludovica Baldari, Gianfranco Silecchia, Claudio Fiorillo, Vincenzo Tondolo, Salvador Morales-Conde, Luigi Boni, Sergio Alfieri, Nicolas Padoy · 21 August 2026

  Minimally invasive colorectal surgeries (MIS-CRS) are characterised by significant variability and inconsistent outcomes. ColoWorkflow, a tool for the video-based assessment (VBA) of MIS-CRS workflow, was recently validated. However, manual VBA is time-consuming, limiting implementation. This study …

• Beyond Instrument Motion: Recognizing Tissue Tension Toward Surgical Skill Assessment

  Marko Haralovi, Zhiqi Miao, Alexander Machiel Bont, Jiapan Guo, Frans van Workum, Estefania Talavera · 19 August 2026

  Surgical performance assessment in minimally invasive surgery largely relies on manual expert review, making it time-consuming, subjective, and difficult to scale. While existing surgical video understanding methods address tasks such as instrument segmentation, surgical phase recognition, and actio…

• Looking Beyond the Scale: Do Surgical Skill Models Learn Transferable Representations Across Assessment Rubrics?

  Hanna Hoffmann, Felix von Bechtolsheim, Stefanie Speidel, Rebecca Hisey · 19 August 2026

  Vision-based surgical skill assessment has shown strong in-domain results, yet a fundamental question remains unasked: do these models learn transferable representations of surgical proficiency, or do they merely encode dataset-specific visual patterns? This paper systematically analyzes what limits…

• Incision trajectory tracing for electrosurgical navigation by CNN-based knife contacting frames extraction method

  Yu Chun Wang, Kaixu Chen, Naoto Ienaga, Yoshihiro Kuroda · 17 August 2026

  Background and Objective: Image-guided surgical navigation has been actively studied because of its advantage of identifying subsurface targets and critical structures, whereas it requires incision trajectories to update the preoperative three-dimensional model dynamically during the surgery. The no…

• MedClaw: Heuristic Agent Harness for Long-Horizon Surgical Video Reasoning

  Yingying Fan, Penghui Du, Leyan Zhu, Runze He, Zimeng Wu, Yuxuan Zhang, Liang Chen, Jiahao Xie, Jiangtang Wang, Shuai Shao, Anchao Yang, Yutong Bai, Yan Wang · 17 August 2026

  Understanding tens-of-minutes surgical videos requires long-horizon temporal reasoning, answering what happens before, after, or across stages of a procedure by grounding the question in visual evidence spread across time. Existing approaches handle this poorly: a one-shot vision-language model (VLM…

• Bootstrapping Vision-Language Model for Hysteroscopic Surgical Scene Segmentation

  Jun Huang, Meiyi Chen, Zijie Yue, Yuhang Xiao, Fang Li, Hanli Wang, Xiaowen Tong, Yi Guo, Miaojing Shi · 11 August 2026

  Hysteroscopic surgical scene segmentation plays a pivotal role in understanding the hysteroscopic intraoperative environment as well as computer-assisted intervention. However, this task presents unique challenges due to the high morphological similarity among different lesions and the presence of a…

• Toward Mask Annotation-Free Surgical Instrument Segmentation from Endoscopic Images Using Text-Prompted Segment Anything Model 3 (SAM3)

  Nakul Poudel, Richard Simon, Cristian A. Linte · 11 August 2026

  Surgical instrument segmentation is a fundamental task for computer-assisted interventions, yet most existing methods rely on pixel-level annotations or manual spatial prompts, which limit scalability and automation. The recently introduced Segment Anything Model 3 (SAM3) offers a pathway to annotat…

• SurgLAT: Surgical Latent Attention Tracking for Depth-Aware Robotic Laparoscope Control

  Rulin Zhou, Qiujie Song, Yujie Ma, An Wang, Wanhao Liu, Guoheng Ma, Yidu Wang, Guankun Wang, Xingrong Diao, Jiankun Wang, Chaowei Zhu, Xianming Liu, Hongliang Ren · 11 August 2026

  Autonomous laparoscopic camera control requires continuous understanding of the surgeon's operative intent in dynamic surgical scenes, where the target operative region is not a stable physical object but a latent and temporally evolving attention state. In this work, we present Surgical Latent Atte…

• Surg-UniWorld: A Unified Surgical World Model with Multimodal Control Experts

  Rulin Zhou, Wanhao Liu, Guoheng Ma, Liangjin Shao, Qiujie Song, Yidu Wang, Guankun Wang, Tong Chen, Long Bai, Luping Zhou, Hongliang Ren · 10 August 2026

  Controllable surgical world models can provide a generative foundation for surgical artificial intelligence and simulation by synthesizing realistic instrument--tissue interactions. However, existing methods lack a unified multimodal control paradigm, while direct fusion of heterogeneous visual cond…

• SPIRIT: Spatio-temporal Pairwise Relational Modeling of Instrument-Tissue Interactions for Surgical Action Triplet Recognition

  Saurav Sharma, Lorenzo Arboit, Nabani Banik, Sarah Meuli, Julia Alekseenko, Jan Liechti, Franziska Heitzinger, Michela Orsi, Didier Mutter, Daniel Gero, Philipp C. Nett, Beat P. Muller, Joel L. Lavanchy, Nicolas Padoy · 4 August 2026

  Fine-grained understanding of surgical activity is essential for context-aware assistance in the operating room, including safety monitoring, adverse event identification, and skill assessment. Surgical action triplets, defined as tuples of the form <instrument, verb, target>, provide a structured d…

• Extended KAFR: A kinematic-adaptive paradigm for the efficient analysis of surgical video

  Huu Phong Nguyen, Shekhar Madhav Khairnar, Ganesh Sankaranarayanan · 4 August 2026

  Artificial Intelligence is increasingly applied to surgical video analysis for phase segmentation, skill assessment, and workflow optimization. A key challenge is the length of surgical recordings, often one to several hours, creating substantial computational burden. We previously developed Kinemat…