Augmented Reality Applications

Neueste Paper

• Egosurg: Arbitrary view synthesis for egocentric replay of operating room workflows from ambient cameras

  Han Zhang, Lalithkumar Seenivasan, Jose L. Porras, Roger D. Soberanis-Mukul, Hao Ding, Hongchao Shu, Benjamin D. Killeen, Ankita Ghosh, Lonny Yarmus, Jeffrey K. Jopling, Masaru Ishii, Angela C. Argento, Mathias Unberath · 28. August 2026

  Observing surgical practice has historically relied on fixed vantage points or recollections, leaving the egocentric perspectives that shape clinical decisions undocumented. Ambient fixed cameras capture the operating room (OR) at room scale but cannot recover what any individual team member actuall…

• Marker-free deformable registration and fusion for augmented reality-guided positive margin localization during tumor resection surgery

  Yue Yang, Annie Benson, Matthieu Chabanas, Jason Slagle, Thomas Myles, Matthew B. Weinger, Jon S. Heiselman, Michael I. Miga, Michael Topf, Jie Ying Wu · 16. Juli 2026

  Positive margins in head and neck oncologic surgery require mapping specimen-side pathology findings to the patient resection bed. This is challenging because pathologists identify the positive margin on slices of the resected, deformed specimen, while surgeons must relocate the corresponding site o…

• VistaRef: Boosting Visual Spatial Orientation Awareness for Pointing-to-Object Detection

  Ling Li, Zhizhen Cai, Xinkun Wu, Ziyu Zhu, Jiaqing Lyu, Bowen Liu, Zhidong Deng · 24. Juni 2026

  Grounding deictic gestures in natural images is fundamental to AR and human-robot collaboration, providing a basis for seamless spatial interaction. While Transformer-based visual models have achieved significant progress in general object detection, their global attention mechanisms often neglect m…

• Hylos: Operability Contracts for Model-Native Spatial Intelligence

  Christopher Da Silva · 26. Mai 2026

  Foundation models can increasingly describe, reconstruct, and generate 3D objects, assemblies, scenes, and environments, but visually plausible spatial output is not yet operable 3D. A generated object or environment becomes useful to an agent only when the system can identify its entities, frames, …

• An Object-Centered Data Acquisition Method for 3D Gaussian Splatting using Mobile Phones

  Yuezhe Zhang, Luqian Bai, Mengting Yu, Lei Wei, Shuai Wan, Yifan Zhang · 22. April 2026

  Data acquisition through mobile phones remains a challenge for 3D Gaussian Splatting (3DGS). In this work we target the object-centered scenario and enable reliable mobile acquisition by providing on-device capture guidance and recording onboard sensor signals for offline reconstruction. After the c…

• RHINO-AR: An Augmented Reality Exhibit for Teaching Mobile Robotics Concepts in Museums

  Nils Dengler, Tim Graf, Leif Van Holland, Patrick Stotko, Reinhard Klein, Maren Bennewitz · 21. April 2026

  We present RHINO-AR, an interactive Augmented Reality (AR) museum exhibit that reintroduces the historical mobile robot RHINO into its original exhibition environment at the Deutsches Museum Bonn. The system builds on our previous work RHINO-VR, which reconstructed the robot and the environment in v…

• MRGEN: A Conceptual Framework for LLM-Powered Mixed Reality Authoring Tools for Education

  Mohammed Oussama Seddini (LIUM), Mohamed Ez-Zaouia (UR, IRISA, DRUID), Ngoc Luyen Le (Heudiasyc), Iza Marfisi (LIUM, IUT Laval) · 20. April 2026

  Mixed Reality (MR) offers immersive and multimodal opportunities for education but remains difficult for teachers to author without technical expertise. We propose MRGEN, a conceptual framework for LLM-powered authoring tools to support teachers in creating MR learning activities that work on mobile…

• GeoVision-Enabled Digital Twin for Hybrid Autonomous-Teleoperated Medical Responses

  Parham Kebria, Soheil Sabri, Laura J Brattain · 16. April 2026

  Remote medical response systems are increasingly being deployed to support emergency care in disaster-affected and infrastructure-limited environments. Enabled by GeoVision capabilities, this paper presents a Digital Twin architecture for hybrid autonomous-teleoperated medical response systems. The …

• SurgNavAR: An Augmented Reality Surgical Navigation Framework for Optical See-Through Head Mounted Displays

  Abdullah Thabit, Mohamed Benmahdjoub, Rafiuddin Jinabade, Hizirwan S. Salim, Marie-Lise C. van Veelen, Mark G. van Vledder, Eppo B. Wolvius, Theo van Walsum · 1. April 2026

  Augmented reality (AR) devices with head mounted displays (HMDs) facilitate the direct superimposition of 3D preoperative imaging data onto the patient during surgery. To use an HMD-AR device as a stand-alone surgical navigation system, the device should be able to locate the patient and surgical in…

• All-in-One Augmented Reality Guided Head and Neck Tumor Resection

  Yue Yang, Matthieu Chabanas, Carrie Reale, Annie Benson, Jason Slagle, Matthew Weinger, Michael Topf, Jie Ying Wu · 1. April 2026

  Positive margins are common in head and neck squamous cell carcinoma, yet intraoperative re-resection is often imprecise because margin locations are typically communicated verbally from pathology. We present an all-in-one augmented reality (AR) system that relocalizes positive margins from a resect…

• Towards an End-to-End System for 3D Tracking of Physical Objects in Virtual Immersive Environments

  Stanisław Knapiński, Maciej Grzeszczuk, Barbara Karpowicz, Pavlo Zinevych, Wieslaw Kopec · 31. März 2026

  This work aims to establish an end-to-end system for tracking of physical 3D objects for virtual reality (VR) applications. We focus on training applications requiring real-time tracking of the position of small physical objects and their reflection in VR space. Out goal is to perform object trackin…

• Shadowless Projection Mapping for Tabletop Workspaces with Synthetic Aperture Projector

  Takahiro Okamoto, Masaki Takeuchi, Masataka Sawayama, Daisuke Iwai · 13. März 2026

  Projection mapping (PM) enables augmented reality (AR) experiences without requiring users to wear head-mounted displays and supports multi-user interaction. It is regarded as a promising technology for a variety of applications in which users interact with content superimposed onto augmented object…

• Extend Your Horizon: A Device-Agnostic Surgical Tool Tracking Framework with Multi-View Optimization for Augmented Reality

  Jiaming Zhang, Mingxu Liu, Hongchao Shu, Ruixing Liang, Yihao Liu, Ojas Taskar, Amir Kheradmand, Mehran Armand, Alejandro Martin-Gomez · 10. März 2026

  Surgical navigation provides real-time guidance by estimating the pose of patient anatomy and surgical instruments to visualize relevant intraoperative information. In conventional systems, instruments are typically tracked using fiducial markers and stationary optical tracking systems (OTS). Augmen…

• Extended Reality (XR): The Next Frontier in Education

  Shadeeb Hossain · 2. März 2026

  Extended Reality (XR), encompassing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), is revolutionizing education by creating immersive, interactive learning environments. This article explores the potential of XR to enhance student engagement, experiential learning, and skill d…

• Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR

  Hanyuan Zhang, Lucas He, Runlong He, Weixi Yi, Abdolrahim Kadkhodamohammadi, Danail Stoyanov, Brian R. Davidson, Evangelos B. Mazomenos, Matthew J. Clarkson · 20. Februar 2026

  Augmented reality (AR) guidance in laparoscopic liver surgery requires accurate registration of preoperative 3D models to intraoperative 2D video, but remains challenging due to partial visibility, specularities, and tissue deformation. Existing methods often rely on contour-based rigid initializati…

• ARport: An Augmented Reality System for Markerless Image-Guided Port Placement in Robotic Surgery

  Zheng Han, Zixin Yang, Yonghao Long, Lin Zhang, Peter Kazanzides, Qi Dou · 17. Februar 2026

  Purpose: Precise port placement is a critical step in robot-assisted surgery, where port configuration influences both visual access to the operative field and instrument maneuverability. To bridge the gap between preoperative planning and intraoperative execution, we present ARport, an augmented re…

• CA-YOLO: Cross Attention Empowered YOLO for Biomimetic Localization

  Zhen Zhang, Qing Zhao, Xiuhe Li, Cheng Wang, Guoqiang Zhu, Yu Zhang, Yining Huo, Hongyi Yu, Yi Zhang · 10. Februar 2026

  In modern complex environments, achieving accurate and efficient target localization is essential in numerous fields. However, existing systems often face limitations in both accuracy and the ability to recognize small targets. In this study, we propose a bionic stabilized localization system based …

• SpeechLess: Micro-utterance with Personalized Spatial Memory-aware Assistant in Everyday Augmented Reality

  Yoonsang Kim, Devshree Jadeja, Divyansh Pradhan, Yalong Yang, Arie Kaufman · 3. Februar 2026

  Speaking aloud to a wearable AR assistant in public can be socially awkward, and re-articulating the same requests every day creates unnecessary effort. We present SpeechLess, a wearable AR assistant that introduces a speech-based intent granularity control paradigm grounded in personalized spatial …

• Memento: Towards Proactive Visualization of Everyday Memories with Personal Wearable AR Assistant

  Yoonsang Kim, Yalong Yang, Arie E. Kaufman · 27. Januar 2026

  We introduce Memento, a conversational AR assistant that permanently captures and memorizes user's verbal queries alongside their spatiotemporal and activity contexts. By storing these "memories," Memento discovers connections between users' recurring interests and the contexts that trigger them. Up…

• Augmented Assembly: Object Recognition and Hand Tracking for Adaptive Assembly Instructions in Augmented Reality

  Alexander Htet Kyaw, Haotian Ma, Sasa Zivkovic, Jenny Sabin · 21. Januar 2026

  Recent advances in augmented reality (AR) have enabled interactive systems that assist users in physical assembly tasks. In this paper, we present an AR-assisted assembly workflow that leverages object recognition and hand tracking to (1) identify custom components, (2) display step-by-step instruct…

• Atelier \`a la conf\'erence IHM 2025 : RA Permanente

  Maxime Cauz, Thibaut Septon, Elise Hallaert, Theo Leclercq, Bruno Dumas, Charles Bailly, Clement Tyminski, Matias Peraza, Sophie Lepreux, Emmanuel Dubois · 16. Januar 2026

  As we move towards more ubiquitous computing, the concept of pervasive augmented reality (PAR) could lead to a major evolution in the relationship between humans, computing and the world. The experience of a continuously augmented world can have both benefits and undesirable consequences for users' …

• Generative Augmented Reality: Paradigms, Technologies, and Future Applications

  Chen Liang, Jiawen Zheng, Yufeng Zeng, Yi Tan, Hengye Lyu, Yuhui Zheng, Zisu Li, Yueting Weng, Jiaxin Shi, Hanwang Zhang · 24. November 2025

  This paper introduces Generative Augmented Reality (GAR) as a next-generation paradigm that reframes augmentation as a process of world re-synthesis rather than world composition by a conventional AR engine. GAR replaces the conventional AR engine's multi-stage modules with a unified generative back…

• AI Assisted AR Assembly: Object Recognition and Computer Vision for Augmented Reality Assisted Assembly

  Alexander Htet Kyaw, Haotian Ma, Sasa Zivkovic, Jenny Sabin · 10. November 2025

  We present an AI-assisted Augmented Reality assembly workflow that uses deep learning-based object recognition to identify different assembly components and display step-by-step instructions. For each assembly step, the system displays a bounding box around the corresponding components in the physic…

• Markerless Augmented Reality Registration for Surgical Guidance: A Multi-Anatomy Clinical Accuracy Study

  Yue Yang, Fabian Necker, Christoph Leuze, Michelle Chen, Andrey Finegersh, Jake Lee, Vasu Divi, Bruce Daniel, Brian Hargreaves, Jie Ying Wu, Fred M Baik · 5. November 2025

  Purpose: In this paper, we develop and clinically evaluate a depth-only, markerless augmented reality (AR) registration pipeline on a head-mounted display, and assess accuracy across small or low-curvature anatomies in real-life operative settings. Methods: On HoloLens 2, we align Articulated HAnd T…