Assessment of Clinical Criteria for Sepsis for the Third International Consensus Definitions for Sep
The Article: Seymour, CW, et al. Assessment of Clinical Criteria for Sepsis for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016; 315(8): 762-774.
The Idea: The Third International Consensus Definitions for Sepsis and Septic Shock recently redefined the term sepsis as “life-threatening organ dysfunction due to a dysregulated host response to infection.” The purpose of this study was to evaluate the validity of existing and novel clinical criteria with the goal of generating recommendations for clinical criteria that could be used to identify sepsis in patients with suspected or confirmed infection.
The Study: Retrospective cohort study among adult encounters with suspected infection. The primary cohort was all hospital encounters between 2010 and 2012 at 12 community and academic hospitals in the UPMC healthcare system in southwestern Pennsylvania. This cohort included all medical and surgical encounters in the ED, hospital ward, and ICU. A random split sample (50/50) was created from this cohort to form a derivation cohort (for developing new clinical criteria for sepsis) and a validation cohort (to assess new and existing criteria). External data sets were also included from: 1) all inpatient encounters at 20 Kaiser Permanente Northern California hospitals from 2009 to 2013, 2) all encounters in 130 hospitals in the US VA system from 2008 to 2010, 3) all non-trauma, non-arrest EMS records from 5 ALS agencies from 2009-2010 that transport to 14 hospitals in King County, Washington, and 4) all patients from 2011-2012 at one German hospital enrolled in the ALERTS prospective study (hospital-acquired infection study). SIRS criteria, SOFA score and modified LODS score were calculated for the period of time from 48h before to 24h after onset of infection and daily. For EHR records, the first episode of suspected infection was defined as the combination of antibiotics and body fluid cultures obtained within a certain time frame (if antibiotics were given first, culture obtained within 24 hours, or if cultures were obtained first, antibiotic ordered within 72 hours). The onset of infection was the time when the first of these 2 events occurred. For non-EHR records, administrative claims identified infection present on admission.
Using the derivation cohort, new, simple criteria were developed. The model was created based on the assumption that hospital mortality would be more common in encounters with infected patients with sepsis. Variables were dichotomized based on their optimal cutoffs using ROC curves for in-hospital mortality. Multiple logistic regression was used with forward selection of variables to include in the new model with a stepwise approach to keep variables that would improve the model’s ability to predict the outcome of interest, while limiting the number of variables included. To improve accuracy, a point score of 1 was assigned to each variable in the model regardless of the regression coefficient.
Criterion validity was assessed by using predictive validity of the candidate criteria with the outcomes. The primary outcome was in-hospital mortality. The secondary outcome was in-hospital mortality plus ICU length of stay greater than or equal to 3 days. To measure predictive validity, a baseline risk model was created for in-hospital mortality based on pre-infection criteria via logistic regression models. The baseline model was based on age, sex, race/ethnicity, and Charlson comorbidity score. The encounters were divided into deciles of baseline risk and within each decile, the rate of the primary and secondary outcomes was determined, comparing encounters with infection with 2 or more of SIRS/SOFA/LODS/qSOFA points vs. encounters with less than 2 criteria of the same score. The ROC curves were used for each outcome and then added to the baseline risk model. Analyses were performed in ICU encounters and non-ICU encounters separately at onset of infection. A post hoc analysis looked at adding serum lactate to the qSOFA model at three different thresholds: 2.0, 3.0, and 4.0.
The Results: 177 hospitals in 5 US and non-US data sets between 2008 and 2013, for a total of 4,885,558 encounters were studied. The primary cohort consisted of 1,309,025 records at UPMC; of these, 148,907 encounters had suspected infection, with most presenting outside of the ICU (89%). The first infection was commonly suspected within 48h of admission (86%), most often presented in the ED (44%), compared with in the ward (33%) or ICU (11%). Mortality was low (4%). The median time from the start of the encounter until suspected infection (culture or antibiotics ordered) was 4.2 hours. In KPNC hospitals, the first suspected infections occurred outside the ICU 98% of the time, and similar mortality was observed (5%) with a similar number identified within 48h of admission (81%). Serum lactate was measured in 57% suspected infection encounters at KPNC vs. <10% in other cohorts. VA hospitals encounters with suspected infection had similar mortality rates (6%), but were more likely to be identified in the ICU (19%).
Of the ICU encounters with suspected infection in the UPMC validation cohort (7932 encounters (11%)), most had 2 or more points on the existing criteria: 88% LODS, 91% SOFA, and 84% SIRS near the time of suspected infection, and a mortality of 18% for all scores at this threshold. SOFA and LODS had better statistical agreement with each other (alpha = 0.87, 95%CI 0.87-0.88), but lower with SIRS (alpha = 0.43, 95% CI 0.41-0.46 for SOFA; alpha = 0.41, 95%CI 0.38-0.43 for LODS). The predictive validity for hospital mortality with SOFA (AUROC 0.74, 95% CI 0.73-0.76) and LODS (AUROC 0.75, 95% CI 0.73-0.76) were not statistically different, but both were statistically greater than the predictive value for SIRS (AUROC 0.64, 95% CI 0.62-0.66). There were 66,522 (89%) encounters with suspected infection outside the ICU. 20,130 (30%) met no SIRS criteria, 27,650 (41%) had no SOFA points, and 29,789 (45%) had no LODS points. Agreement patterns between existing criteria were similar compared to the comparison of criteria for encounters with suspected infection in the ICU.
The qSOFA model is: GCS of 13 or less, SBP of 100 or less, and RR of 22/min or more, each worth 1 point for score range of 0-3. 73-90% encounters with infection had less than 2 qSOFA points and mortality ranged from 1-24% over the score range. 24% of encounters with infection had 2 or 3 qSOFA points and accounted for 70% of deaths and 70% of deaths or ICU stay of 3 days or longer. Predictive validity of qSOFA was good for in-hospital mortality (AUROC = 0.81, 95% CI 0.80-0.82), was not statistically different from LODS (p=0.77), and was statistically greater than SOFA or change in SOFA score (p<0.001) when the suspected infection occurred outside the ICU. 70% of decedents had 2 or more qSOFA points and 78% of survivors had less than 2 qSOFA points. The qSOFA model was tested in 4 external data sets (706,399 patient encounters at 165 hospitals with 6,508 encounters out of hospital, 619,137 non-ICU, and 80,595 ICU encounters). qSOFA maintained consistent predictive validity for both community infection (AUROC = 0.71) and hospital-acquired infection (AUROC = 0.75). These results were similar in the VA dataset (AUROC 0.78), where no GCS data was available.
Post-hoc addition of lactate levels of 2.0, 3.0, or 4.0 to qSOFA for analysis with KPNC data for a revised 4 point score statistically changed the predictive validity of qSOFA (AUROC 0.80, 95% CI 0.79-0.81 with lactate) vs. 0.79, 95% CI 0.78-0.80 p<.001 without lactate, and was consistent for higher thresholds of lactate and when using a continuous distribution.
The Takeaway: For encounters with suspected infection in the ICU, SOFA and LODS had greater predictive validity for in-hospital mortality when compared with SIRS criteria. Outside the ICU, qSOFA had greater predictive validity than SOFA score and maintained validity with evaluation using a variety of measurement conditions, in academic vs. community hospitals, international hospital, and for community and hospital-acquired infections. Task force recommendations are to use the SOFA score of 2 points or more in encounters with suspected infection as criteria for sepsis and use the qSOFA score in a non-ICU setting to consider the possibility of sepsis.