Artificial intelligence integration in surgery through hand and instrument tracking: a systematic literature review

Authors

• Kivanc Yangi, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.Follow
• Thomas J. On, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.
• Yuan Xu, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.
• Arianna S. Gholami, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.
• Jinpyo Hong, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.
• Alexander G. Reed, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.
• Pravarakhya Puppalla, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.
• Jiuxu Chen, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.
• Jonathan A. Tangsrivimol, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.
• Baoxin Li, School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, United States.
• Marco Santello, School of Biological and Health Systems Engineering, Arizona State University, Tempe, AZ, United States.
• Michael T. Lawton, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.
• Mark C. Preul, The Loyal and Edith Davis Neurosurgical Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.

Document Type

Article

Abstract

OBJECTIVE: This systematic literature review of the integration of artificial intelligence (AI) applications in surgical practice through hand and instrument tracking provides an overview of recent advancements and analyzes current literature on the intersection of surgery with AI. Distinct AI algorithms and specific applications in surgical practice are also examined. METHODS: An advanced search using medical subject heading terms was conducted in Medline (via PubMed), SCOPUS, and Embase databases for articles published in English. A strict selection process was performed, adhering to PRISMA guidelines. RESULTS: A total of 225 articles were retrieved. After screening, 77 met inclusion criteria and were included in the review. Use of AI algorithms in surgical practice was uncommon during 2013-2017 but has gained significant popularity since 2018. Deep learning algorithms (n = 62) are increasingly preferred over traditional machine learning algorithms (n = 15). These technologies are used in surgical fields such as general surgery (n = 19), neurosurgery (n = 10), and ophthalmology (n = 9). The most common functional sensors and systems used were prerecorded videos (n = 29), cameras (n = 21), and image datasets (n = 7). The most common applications included laparoscopic (n = 13), robotic-assisted (n = 13), basic (n = 12), and endoscopic (n = 8) surgical skills training, as well as surgical simulation training (n = 8). CONCLUSION: AI technologies can be tailored to address distinct needs in surgical education and patient care. The use of AI in hand and instrument tracking improves surgical outcomes by optimizing surgical skills training. It is essential to acknowledge the current technical and social limitations of AI and work toward filling those gaps in future studies.

Publication Date

1-1-2025

Publication Title

Frontiers in surgery

ISSN

2296-875X

Volume

12

First Page

1528362

PubMed ID

40078701

Digital Object Identifier (DOI)

10.3389/fsurg.2025.1528362

Recommended Citation

Yangi, Kivanc; On, Thomas J.; Xu, Yuan; Gholami, Arianna S.; Hong, Jinpyo; Reed, Alexander G.; Puppalla, Pravarakhya; Chen, Jiuxu; Tangsrivimol, Jonathan A.; Li, Baoxin; Santello, Marco; Lawton, Michael T.; and Preul, Mark C., "Artificial intelligence integration in surgery through hand and instrument tracking: a systematic literature review" (2025). Neurosurgery. 2347.
https://scholar.barrowneuro.org/neurosurgery/2347