Predictive performance of the American College of Surgeons universal risk calculator in neurosurgical patients

Document Type

Article

Abstract

OBJECTIVE The American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) universal Surgical Risk Calculator is an online decision-support tool that uses patient characteristics to estimate the risk of adverse postoperative events. Further validation of this risk calculator in the neurosurgical population is needed; therefore, the object of this study was to assess the predictive performance of the ACS NSQIP Surgical Risk Calculator in neurosurgical patients treated at a tertiary care center. METHODS A single-center retrospective review of 1006 neurosurgical patients treated in the period from September 2011 through December 2014 was performed. Individual patient characteristics were entered into the NSQIP calculator. Predicted complications were compared with actual occurrences identified through chart review and administrative quality coding data. Statistical models were used to assess the predictive performance of risk scores. Traditionally, an ideal risk prediction model demonstrates good calibration and strong discrimination when comparing predicted and observed events. RESULTS The ACS NSQIP risk calculator demonstrated good calibration between predicted and observed risks of death (p = 0.102), surgical site infection (SSI; p = 0.099), and venous thromboembolism (VTE; p = 0.164) Alternatively, the risk calculator demonstrated a statistically significant lack of calibration between predicted and observed risk of pneumonia (p = 0.044), urinary tract infection (UTI; p < 0.001), return to the operating room (p < 0.001), and discharge to a rehabilitation or nursing facility (p < 0.001). The discriminative performance of the risk calculator was assessed using the c-statistic. Death (c-statistic 0.93), UTI (0.846), and pneumonia (0.862) demonstrated strong discriminative performance. Discharge to a rehabilitation facility or nursing home (c-statistic 0.794) and VTE (0.767) showed adequate discrimination. Return to the operating room (c-statistic 0.452) and SSI (0.556) demonstrated poor discriminative performance. The risk prediction model was both well calibrated and discriminative only for 30-day mortality. CONCLUSIONS This study illustrates the importance of validating universal risk calculators in specialty-specific surgical populations. The ACS NSQIP Surgical Risk Calculator could be used as a decision-support tool for neurosurgical informed consent with respect to predicted mortality but was poorly predictive of other potential adverse events and clinical outcomes.

Medical Subject Headings

Female; Humans; Male; Neurosurgical Procedures (adverse effects); Postoperative Complications (etiology); Retrospective Studies; Risk Assessment; United States

Publication Date

3-1-2018

Publication Title

Journal of neurosurgery

E-ISSN

1933-0693

Volume

128

Issue

3

First Page

942

Last Page

947

PubMed ID

28452615

Digital Object Identifier (DOI)

10.3171/2016.11.JNS161377

Link to Full Text

Share

COinS