Skip to main content

A mixed reality framework for microsurgery simulation with visual-tactile perception.

Xiang, N., Liang, H-N., Yu, L., Yang, X. and Zhang, J. J., 2023. A mixed reality framework for microsurgery simulation with visual-tactile perception. The Visual Computer. (In Press)

Full text available as:

[img] PDF
toTVC_CGI23_.pdf - Accepted Version
Restricted to Repository staff only until 30 June 2024.
Available under License Creative Commons Attribution Non-commercial.


DOI: 10.1007/s00371-023-02964-1


Microsurgery is a general term for surgery combining surgical microscope and specialized precision instruments during operation. Training in microsurgery requires considerable time and training resources. With the rapid development of computer technologies, virtual surgery simulation has gained extensive attention over the past decades. In this work, we take advantage of mixed reality (MR) that creates an interactive environment where physical and digital objects coexist, and present an MR framework for the microsurgery simulation. It enables users to practice anastomosis skills with real microsurgical instruments rather than additional haptic feedback devices that are typically used in virtual reality-based systems, and to view a realistic rendering intra-operative scene at the same time, thus creating an immersive training experience with such a visual-tactile interactive environment. A vision-based tracking system is proposed to simultaneously track microsurgical instruments and artificial blood vessels, and a learning-based anatomical modeling approach is introduced to facilitate the development of simulations in different microsurgical specialities by rapidly creating virtual assets. Moreover, we build a prototype system for the simulation specializing in microvascular hepatic artery reconstruction to demonstrate the feasibility and applicability of our framework.

Item Type:Article
Uncontrolled Keywords:Virtual surgery; Mixed reality; 3D tracking; 3D modeling
Group:Faculty of Media & Communication
ID Code:38859
Deposited By: Symplectic RT2
Deposited On:09 Aug 2023 11:25
Last Modified:09 Aug 2023 11:25


Downloads per month over past year

More statistics for this item...
Repository Staff Only -