This paper is in the following e-collection/theme issue:

Reviews in Medical Informatics (83) Decision Support for Health Professionals (1145) Electronic Health Records (971) Quality Improvement (226) Secondary Use of Clinical Data for Research and Surveillance (325) Clinical Information and Decision Making (1339) Registered Report (528)

Published on in Vol 10, No 5 (2022): May

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/35061, first published .
Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Pediatric, Neonatal, and Maternal Inpatients: Scoping Review

Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Pediatric, Neonatal, and Maternal Inpatients: Scoping Review

Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Pediatric, Neonatal, and Maternal Inpatients: Scoping Review

Authors of this article:

Khalia Ackermann1 Author Orcid Image ;   Jannah Baker1 Author Orcid Image ;   Marino Festa2 Author Orcid Image ;   Brendan McMullan3, 4 Author Orcid Image ;   Johanna Westbrook1 Author Orcid Image ;   Ling Li1 Author Orcid Image
Article Authors Cited by Tweetations (9) Metrics

    Review

    1Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia

    2Kids Critical Care Research, Department of Paediatric Intensive Care, Children's Hospital at Westmead, Sydney, Australia

    3Department of Immunology and Infectious Diseases, Sydney Children's Hospital, Randwick, Sydney, Australia

    4Faculty of Medicine & Health, University of New South Wales, Sydney, Australia

    Corresponding Author:

    Khalia Ackermann, MPH

    Centre for Health Systems and Safety Research

    Australian Institute of Health Innovation

    Level 6, 75 Talavera road

    Macquarie University, 2109

    Australia

    Phone: 61 2 9850 2432

    Email: khalia.ackermann@mq.edu.au


    Background: Sepsis is a severe condition associated with extensive morbidity and mortality worldwide. Pediatric, neonatal, and maternal patients represent a considerable proportion of the sepsis burden. Identifying sepsis cases as early as possible is a key pillar of sepsis management and has prompted the development of sepsis identification rules and algorithms that are embedded in computerized clinical decision support (CCDS) systems.

    Objective: This scoping review aimed to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of pediatric, neonatal, and maternal inpatients at risk of sepsis.

    Methods: MEDLINE, Embase, CINAHL, Cochrane, Latin American and Caribbean Health Sciences Literature (LILACS), Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and ProQuest Dissertations and Theses Global (PQDT) were searched by using a search strategy that incorporated terms for sepsis, clinical decision support, and early detection. Title, abstract, and full-text screening was performed by 2 independent reviewers, who consulted a third reviewer as needed. One reviewer performed data charting with a sample of data. This was checked by a second reviewer and via discussions with the review team, as necessary.

    Results: A total of 33 studies were included in this review—13 (39%) pediatric studies, 18 (55%) neonatal studies, and 2 (6%) maternal studies. All studies were published after 2011, and 27 (82%) were published from 2017 onward. The most common outcome investigated in pediatric studies was the accuracy of sepsis identification (9/13, 69%). Pediatric CCDS systems used different combinations of 18 diverse clinical criteria to detect sepsis across the 13 identified studies. In neonatal studies, 78% (14/18) of the studies investigated the Kaiser Permanente early-onset sepsis risk calculator. All studies investigated sepsis treatment and management outcomes, with 83% (15/18) reporting on antibiotics-related outcomes. Usability and cost-related outcomes were each reported in only 2 (6%) of the 31 pediatric or neonatal studies. Both studies on maternal populations were short abstracts.

    Conclusions: This review found limited research investigating CCDS systems to support the early detection of sepsis among pediatric, neonatal, and maternal patients, despite the high burden of sepsis in these vulnerable populations. We have highlighted the need for a consensus definition for pediatric and neonatal sepsis and the study of usability and cost-related outcomes as critical areas for future research.

    International Registered Report Identifier (IRRID): RR2-10.2196/24899

    JMIR Med Inform 2022;10(5):e35061

    doi:10.2196/35061

    Keywords

    sepsis; early detection of disease; computerized clinical decision support; patient safety; electronic health records; sepsis care pathway 


    Sepsis Identification

    Sepsis, redefined in adults in 2016 as “life-threatening organ dysfunction caused by a dysregulated host response to infection” [1], was associated with an estimated 11 million deaths worldwide in 2017 [2]. Neonatal, pediatric, and obstetric populations are particularly vulnerable to developing sepsis [2-4].

    Children aged <5 years accounted for approximately 40% of the estimated 50 million people diagnosed with sepsis in 2017 [2]. Furthermore, a recent report indicated that children aged <1 year have a considerably higher sepsis incidence rate compared with other age groups in Australia [5]. An estimated 28 neonatal sepsis cases occur per 1000 live births, with an associated mortality rate of 17.6% [4]. Survivors of pediatric sepsis have a substantial reduction in health-related quality of life compared with nonsepsis cases, with increased risk of hospital readmissions, cognitive impairment, and physical disability [6-9]. Similarly, surviving neonatal sepsis is associated with both short- and long-term neurodevelopmental delay and disability [10,11].

    The most recent consensus definition of pediatric sepsis was presented in 2005, applicable to children from full-term birth to 18 years of age, and defined pediatric sepsis as modified “systemic inflammatory response syndrome (SIRS) in the presence of or as a result of suspected or proven infection” [12]. The definition of pediatric septic shock, a severe and often fatal progression of sepsis, was refined by the 2020 Surviving Sepsis Campaign guidelines to “severe infection leading to cardiovascular dysfunction (including hypotension, need for treatment with vasoactive medication, or impaired perfusion)” [13]. There is currently no formal definition of sepsis distinct to the neonatal population [14,15]; however, a recent systematic review of randomized controlled trials found neonatal sepsis to be most commonly defined by blood culture alone, followed closely by blood culture combined with clinical signs [16].

    In the maternal population, a consensus definition for maternal sepsis was presented in 2017, defined as “organ dysfunction resulting from infection during pregnancy, child-birth, post-abortion, or post-partum period” [3]. The World Health Organization Global Maternal Sepsis Study [17] found the ratio of maternal infections in hospitalized women to be 70.4 (95% CI 67.7-73.1) women per 1000 live births. Furthermore, in 2014, a World Health Organization analysis indicated that 10.7% of maternal deaths between 2003 and 2009 were associated with sepsis [18]. Maternal sepsis also affects the health of the child and has been associated with serious complications, such as neonatal sepsis, spontaneous abortions, preterm births, and over 4.5 times the risk of death in the child [3,19,20].

    Prompt initiation of treatment is critical for successful sepsis management [21-23]. The earlier sepsis is detected, the faster therapies can be initiated [24]. Therefore, early detection is key to improving patient outcomes. However, pediatric, neonatal, and maternal sepsis can be challenging to identify. Age-dependent physiological norms contribute to vague or nonspecific symptoms and extreme variation between patient presentations, making it difficult for clinicians to distinguish between benign conditions and more severe disease [3,15,25-28]. Recently, clinical tools, often as part of associated care bundles and clinical programs, have been developed to facilitate improved sepsis recognition, organ dysfunction assessment, and prediction of poor outcomes for pediatric (eg, pediatric sequential organ failure assessment [29], pediatric logistic organ dysfunction-2 score [30], and pediatric sepsis score [31]), neonatal (eg, neonatal sequential organ failure assessment [32]), and maternal sepsis (eg, modified obstetric early warning score [33] and sepsis in obstetrics score [34]). However, these tools typically rely on timely and regular vital sign monitoring by clinical staff to ensure that deteriorating patients are promptly detected [35,36].

    CCDS Systems

    The widespread implementation of clinical information systems has allowed for sepsis recognition tools to be integrated into computerized clinician decision support (CCDS) systems [37,38] to assist clinical staff with decision-making [39]. In particular, CCDS systems can be used to improve the early detection of sepsis by monitoring patient data and automatically alerting when a patient shows signs consistent with sepsis [36]. Over the last 20 years, 2 types of CCDS systems have been developed: knowledge-based CCDS using preprogrammed rules [39] and adaptive systems using machine learning and artificial intelligence techniques [40]. This review is focused only on knowledge-based CCDS systems.

    Research Questions and Aims

    Despite the critical importance of sepsis detection, there is a paucity of research on pediatric, neonatal, and maternal sepsis recognition tools [14,15,17,37]. In this scoping review, we mapped the available research investigating the use of knowledge-based CCDS systems for the early detection of sepsis in pediatric, neonatal, and maternal inpatients to provide an overview of the field and identify knowledge gaps for future research. Specifically, we aimed to (1) scope the study contexts, designs, and research methods used; (2) summarize the study outcomes investigated; and (3) map the range of CCDS system designs and implementation features, such as the clinical criteria for sepsis.


    Overview

    A protocol detailing the methodology of this scoping review has been previously published [41]. This review follows the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) statement [42]. A completed PRISMA-ScR checklist can be found in Multimedia Appendix 1.

    Study Selection

    To identify relevant studies, we used a broad 3-step strategy [41], during which an experienced librarian was consulted. The final search strategy combined terms for sepsis, clinical decision support, and early detection, excluding terms for artificial intelligence, and was used to search MEDLINE, Embase, CINAHL, Cochrane, Latin American and Caribbean Health Sciences Literature (LILACS), Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and ProQuest Dissertations and Theses Global (PQDT). The search strategy used for MEDLINE is presented in Multimedia Appendix 2. The search was conducted in September 2020.

    The search results were exported to an EndNote X9 (Clarivate) library. After deduplication, 2 reviewers (KA and JB) independently performed title, abstract, and full-text screening using the eligibility criteria reported in our protocol [41]. The reference lists of relevant systematic reviews and salient papers were manually searched by one reviewer (KA) with a second reviewer (JB) double-checking their inclusion to identify any further studies. Any disagreements were resolved through discussion or consultation with a third reviewer (LL). A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram visually representing this process is presented in Figure 1.

    ‎
    Figure 1. Flowchart of the search results and screening process. LILACS: Latin American and Caribbean Health Sciences Literature; PQDT: ProQuest Dissertations and Theses.

    A total of 2 reviewers (KA and JB) independently piloted title and abstract screening with a random selection of 25 articles and full-text screening with a random selection of 10 articles. The results were discussed with a third reviewer (LL) to ensure consensus before undertaking the full screen. The 2 reviewers (KA and JB) had 100% agreement in the title and abstract pilot screen, 97.6% agreement in title and abstract screening, 60% agreement in the pilot full-text screen, and 77.4% agreement in full-text screening. Both peer-reviewed journal articles and gray literature studies, such as conference abstracts and theses, were included in this review. The gray literature that was later published as a peer-reviewed article was removed. Studies reporting the same methods and study cohorts but measuring different outcomes were included.

    We chose to publish the results of this review in 2 manuscripts separated by patients’ age, given the distinct sepsis presentations and pathophysiology of pediatric, neonatal, and maternal patients compared with adults [3,28,43]. The results of the review investigating adult CCDS systems have been published previously [44].

    Data Charting

    The form used for data charting was designed using Microsoft Access based on the data charting form previously used for adult studies [44]. The original version was refined based on sample data extracted from 2 pediatric, 2 neonatal, and 1 maternal study. The remaining studies were charted by a single reviewer (KA), with a sample of studies checked by a second reviewer (JB), and ongoing consultation with a third reviewer (LL). We accepted any definition of the charted items, as detailed in the studies.

    The final form abstracted data based on all 3 aims and included all components, as listed in our protocol [41], with some minor adjustments, as presented in Multimedia Appendix 3 [45-51]. The outcomes listed comprised (1) outcomes reported in the aims, methods, and results and (2) outcomes from the study sections that met our inclusion criteria [41]. The following were excluded: (1) outcomes mentioned in the methods or introduction but not in the results; (2) analysis of demographic or clinical features not specifically identifying the performance of the alert, unless they were the only outcome or the main outcome reported; (3) outcomes not discussed in the aims or methods and not included in the main results tables; and (4) balancing and process outcome measures. We distinguished live CCDS as systems that were implemented and actively alerting and silent CCDS as systems that were implemented and running with alerts muted.

    Analyzing and Reporting the Results

    The abstracted data were analyzed through a narrative review, with accompanying statistical summaries organized by population group and aims. Tables were created using frequency counts and percentages to summarize the data and produce graphical figures where appropriate. The results are presented separately for the journal articles and conference abstracts.

    The charted data demonstrated substantial diversity; hence, individual categories were grouped to allow for meaningful analysis. We have included a breakdown of what is included in each group in Multimedia Appendix 4.

    Ethics Approval

    This scoping review used data collected from published studies (including publicly available gray literature). No individual patient was involved, and only aggregate-level data were presented; hence, ethical approval or consent to participate was not required.


    Study Characteristics

    A database search returned 22,190 results. After deduplication, 12,139 studies were included for title and abstract screening. The full texts of 368 articles were screened, and 146 studies were identified for inclusion in the review. Manual searching identified a further 11 records. Of the 157 included studies, 33 (21%) [52-84] investigated pediatric, neonatal, and maternal populations. In comparison, 124 (79%) studies examined adult or unspecified age (assumed adult) inpatient populations (Figure 1). Thus, pediatric, neonatal, and maternal studies only represented 8.3% (13/157), 11.5% (18/157), and 1.3% (2/157) of the total studies, respectively. This process is visually presented in a PRISMA flowchart, as shown in Figure 1. A table detailing the main characteristics of the 33 included studies is presented in Multimedia Appendix 5 [52-84].

    Pediatric Studies

    Of the 13 studies investigating pediatric CCDS systems, 7 (54%) were journal articles and 6 (46%) were conference abstracts (Table 1). All studies were published in 2012 or later, with most journal articles (6/7, 86%) published after 2016 (Figure 2). Of the 13 studies, 11 (85%) were conducted in the United States, whereas the remaining 2 (15%) studies did not specify in which country they were conducted [64,73] (Multimedia Appendix 6). Of the 13 studies, 12 (92%) were conducted in children’s hospitals, whereas the remaining study [58] was conducted at a general hospital. All studies used quantitative methods, with the principal study design split between single cohort and before-after studies (Table 1).

    The most common outcomes investigated were patient outcomes and sepsis treatment and management outcomes (Figure 3). Only 1 (8%) conference abstract [58] investigated an outcome related to the CCDS system usability, and none of the studies investigated pediatric CCDS-related cost outcomes (Figure 3). The most commonly investigated patient outcome was sepsis identification (9/13, 69%; Table 1). Pediatric CCDS systems were compared with the gold standard to measure the extent to which they identified sepsis. The gold standard definition used to determine true sepsis cases differed between studies, with 13 different definitions used to define sepsis across 9 studies (Table 1). Similarly, the method used to identify gold standard cases varied across studies: 38% (5/13) performed a chart review, 8% (1/13) prospectively screened patients, 8% (1/13) applied a manual screening tool, 8% (1/13) performed both a chart review and screened patients, and 8% (1/13) did not specify.

    The main characteristics of the investigated pediatric CCDS systems are presented in Table 2. Most commonly, pediatric CCDS systems were live (10/13, 77%), homegrown (11/13, 85%), alerted via the electronic health record (6/13, 46%), and responded to by nurses (6/13, 46%) and other clinicians (5/13, 38%; Table 2).

    The criteria used by the CCDS systems to identify sepsis cases are summarized in Table 3. In general, a diverse range of criteria was used to identify suspected sepsis cases, with 18 clinical criteria used across 9 pediatric CCDS systems in 8 studies included in this review. The remaining 5 pediatric studies [73,74,80,82,83], all conference abstracts, did not specify the CCDS system criteria used for sepsis case identification and were not included in Table 3. A total of 2 particular systems appear to be the subject of more than one study: the first in the studies by Dewan et al [61] and Vidrine et al [81] and the second in the studies by Stinson et al [77] and Viteri et al [82]. One journal article [64] is counted twice in Table 3, as it contains 2 separate electronic CCDS systems with different criteria: one with automated continuous screening and the other with clinician-initiated screening.

    Table 1. Context and outcome characteristics for pediatric studies.
    Study characteristicsNumber of studies by publicationTotala

    		Journal articlesConference abstracts
    Subtotal, n7613
    Principal study type, n (%)

    		Single cohort3 (43)4 (67)7 (54)

    		Before-after4 (57)2 (33)6 (46)
    Setting, n (%)

    		Hospital wideb0 (0)2 (33)2 (15)

    		Emergency department4 (57)1 (17)5 (38)

    		Intensive care unit2 (29)0 (0)2 (15)

    		Inpatient units1 (14)3 (50)4 (31)
    Number of participants, n (%)

    		≤1001 (14)2 (33)3 (23)

    		101-10,0001 (14)2 (33)3 (23)

    		10,001-100,0002 (29)1 (17)3 (23)

    		>100,0002 (29)0 (0)2 (15)

    		Unspecified1 (14)1 (17)2 (15)
    Funding, n (%)

    		Yes (noncommercial)2 (29)0 (0)2 (15)

    		No2 (29)0 (0)2 (15)

    		Unspecified3 (43)6 (100)9 (69)
    Outcomes, n (%)

    		Patient outcomes


    		Sepsis identification5 (71)4 (67)9 (69)



    		Gold standard definitionc




    		Goldstein et al [12]2 (29)0 (0)2 (15)




    		American Academy of Pediatrics Sepsis Collaborative tool [85]1 (14)0 (0)1 (8)




    		Clinician discretion3 (43)2 (33)5 (38)




    		Improving Pediatric Sepsis Outcomes definition [86]1 (14)0 (0)1 (8)




    		International Classification of Diseases codes1 (14)0 (0)1 (8)




    		Not specified1 (14)2 (33)3 (23)


    		Other4 (57)1 (17)5 (38)

    		Sepsis treatment or management, n (%)


    		Timeliness of alert or intervention3 (43)1 (17)4 (31)


    		Other6 (86)1 (17)7 (54)

    		Usability, n (%)


    		Satisfaction0 (0)1 (17)1 (8)

    aThe percentages were calculated from the number of pediatric studies (n=13). As some studies reported multiple outcomes for each category, there were more than 13 outcomes in some categories, and therefore, the percentages add to more than 100%.

    bIf the study setting was not explicitly stated, it was assumed to be hospital wide.

    cSome studies have used multiple definitions of sepsis as part of their gold standard.

    ‎
    Figure 2. Studies investigating neonatal and pediatric computerized clinician decision support systems by year, population, and publication type.
    ‎
    Figure 3. Outcome categories reported by studies by publication type and population.
    Table 2. Computerized clinical decision support characteristics in pediatric studies.
    CCDSa characteristicsNumber of studies by publicationTotalb

    		Journal articlesConference abstracts
    Subtotal, n7613
    CCDS type, n (%)

    		Homegrownc6 (86)5 (83)11 (85)

    		Commercial, n (%)0 (0)1 (17)1 (8)


    		Epic monitor0 (0)1 (17)1 (8)

    		Unspecified1 (14)0 (0)1 (8)
    Silent or lived, n (%)

    		Live5 (71)5 (83)10 (77)

    		Silent1 (14)1 (17)2 (15)

    		Both (pre or post)1 (14)0 (0)1 (8)
    Related interventions, n (%)

    		None2 (29)5 (83)7 (54)

    		Response team4 (57)1 (17)5 (38)

    		Education and information resources3 (43)1 (17)4 (31)

    		Order sets3 (43)1 (17)4 (31)

    		Sepsis protocol1 (14)1 (17)2 (15)

    		Other2 (29)1 (17)3 (23)
    Responding personnel, n (%)

    		Nurses6 (86)0 (0)6 (46)

    		Other clinicians3 (43)2 (33)5 (38)

    		Response team0 (0)2 (33)2 (15)

    		Not specified0 (0)3 (50)3 (23)
    Alert delivery, n (%)

    		Electronic health record6 (86)0 (0)6 (46)

    		Emergency department tracking board1 (14)0 (0)1 (8)

    		Not specified0 (0)6 (100)6 (46)

    aCCDS: computerized clinical decision support.

    bThe percentages were calculated from the number of pediatric studies (n=13). As some studies reported multiple characteristics for each category, there were more than 13 characteristics in some categories; therefore, the percentages add to more than 100%.

    cHomegrown CCDS systems are defined as CCDS systems that have been designed by the institution implementing them, rather than commercially available systems [41].

    dA live CCDS system is a system that is implemented and being used by clinicians in real time during the study. Silent systems are systems that have been implemented but do not alert clinicians during the study and thus do not influence treatment.

    Table 3. Clinical criteria used by pediatric computerized clinical decision support (CCDS) systems for sepsis identification.

    		Study

    		Balamuth et al, 2017 [55]Cruz et al, 2012 [59]Dewan et al, 2020 [61]Eisenberg et al, 2021 [64] (clinician-initiated)Eisenberg et al, 2021 [64] (automated)Lloyd et al, 2018 [71]Stinson et al, 2019 [77]Vidrine et al, 2020 [81]Coffman et al, 2018a [58]Total, n (%b)
    Temperature✓✓✓✓✓✓✓✓✓9 (69)
    Capillary refill or perfusion✓✓✓✓
    		✓✓✓
    		7 (54)
    Mental status✓✓✓✓
    		✓✓✓
    		7 (54)
    Heart rate✓✓
    		✓✓✓✓

    		6 (46)
    Hypotension✓
    		✓✓
    		✓✓✓
    		6 (46)
    High-risk patient✓✓
    		✓
    		✓✓

    		5 (38)
    Pulse assessment

    		✓✓
    		✓✓✓
    		5 (38)
    Skin assessment

    		✓✓
    		✓✓✓
    		5 (38)
    Respiratory rate


    		✓✓✓✓

    		4 (31)
    Infection concern, change in clinical or sepsis risk✓

    		✓
    		✓


    		3 (23)
    Blood culture order

    		✓



    		✓
    		2 (15)
    Leukocyte count



    		✓



    		1 (8)
    Cardiac organ dysfunction



    		✓



    		1 (8)
    Noncardiac organ dysfunction



    		✓



    		1 (8)
    Change in Pediatric Early Warning Score







    		✓1 (8)
    Family concern







    		✓1 (8)
    Vital sign change







    		✓1 (8)
    Patient risk change







    		✓1 (8)

    aThis study is a conference abstract, and the other 8 studies are journal articles.

    bThe percentages were calculated from the number of pediatric studies (n=13).

    Neonatal Studies

    Of the 18 articles investigating neonatal CCDS systems, 14 (78%) were journal articles and 4 (22%) were conference abstracts. All studies were published in 2015 or later, with most published in 2018 (n=9; Figure 2). Overall, 61% (11/18) of the studies were conducted in the United States, 11% (2/18) were conducted in the Netherlands, 11% (2/18) did not specify location, and 1 (6%) study each was set in Australia, Israel, and the United Kingdom (Multimedia Appendix 6). All neonatal studies used quantitative methods to investigate the CCDS systems. A total of 89% (16/18) of studies were single site, with the remaining 11% (2/18) of studies involving 4 [66] and 2 sites [69]. The gestational age range of neonates included in these studies was quite diverse, with 35 weeks and older being the most common inclusion threshold (Table 4).

    The most common outcome used to investigate neonatal CCDS systems was sepsis treatment and management outcomes, followed by patient outcomes (Figure 3; Table 4). CCDS-related usability and cost outcomes were only investigated by 1 and 2 studies, respectively [53,56,66] (Figure 3; Table 4). Of the sepsis treatment and management outcomes, antibiotics-related outcomes were reported most frequently (15/18, 83%; Table 4). Table 5 reports the main characteristics of the neonatal CCDS systems. Notably, most studies investigated early-onset sepsis (15/18, 83%) using the neonatal early-onset sepsis risk calculator developed by the Kaiser Permanente team [87-89] (14/18, 78%).

    Table 4. Context and outcome characteristics in neonatal studies.
    Study characteristicsNumber of studies by publicationTotala

    		Journal articlesConference abstracts
    Subtotal, n14418
    Principal study type, n (%)

    		Single cohort3 (21)3 (75)6 (33)

    		Before-after9 (64)1 (25)10 (56)

    		Interrupted time series2 (14)0 (0)2 (11)
    Setting, n (%)

    		Hospital wideb4 (29)2 (50)6 (33)

    		Nursery7 (50)2 (50)9 (50)

    		ICUc3 (21)0 (0)3 (17)
    Number of participants, n (%)

    		≤1000 (0)1 (25)1 (6)

    		101-10005 (36)1 (25)6 (33)

    		1001-10,0006 (43)0 (0)6 (33)

    		>10,0012 (14)0 (0)2 (11)

    		Unspecified1 (7)2 (50)3 (17)
    Age of included neonates, n (%)

    		<33 weeks gestation1 (7)0 (0)1 (6)

    		≥34 weeks gestation3 (21)1 (25)4 (22)

    		≥35 weeks gestation4 (29)1 (25)5 (28)

    		≥36 weeks gestation2 (14)0 (0)2 (11)

    		>37 weeks gestation1 (7)0 (0)1 (6)

    		First month of life1 (7)0 (0)1 (6)

    		Unspecified2 (14)2 (50)4 (22)
    Funding, n (%)

    		Yes (noncommercial)1 (7)0 (0)1 (6)

    		No7 (50)0 (0)7 (39)

    		Unspecified6 (43)4 (100)10 (56)
    Outcomes, n (%)

    		Patient outcomes


    		ICU admission4 (29)0 (0)4 (22)


    		Length of stay3 (21)1 (25)4 (22)


    		Other4 (29)1 (25)5 (28)

    		Sepsis treatment or management


    		Antibiotics12 (86)3 (75)15 (83)


    		Laboratory evaluation8 (57)3 (75)11 (61)


    		Timeliness of alert or intervention2 (14)0 (0)2 (11)


    		Sepsis guideline compliance2 (14)0 (0)2 (11)


    		Other4 (29)1 (25)5 (28)

    		Usability


    		Effectiveness1 (7)0 (0)1 (6)

    		Cost2 (14)0 (0)2 (11)

    aThe percentages were calculated from the number of neonatal studies (n=18). As some studies have reported multiple outcomes for each category, there were more than 18 outcomes in some categories; therefore, the percentages add to more than 100%.

    bIf the study setting was not explicitly stated, it was assumed to be hospital wide.

    cICU: intensive care unit.

    Table 5. Computerized clinical decision support characteristics in neonatal studies.
    CCDSa characteristicsNumber of studies by publicationTotalb

    		Journal articlesConference abstracts
    Subtotal, n14418
    Type of sepsis, n (%)

    		Early-onset sepsis12 (86)3 (75)15 (83)

    		Late-onset sepsis1 (7)0 (0)1 (6)

    		Sepsis1 (7)1 (25)2 (11)
    General CCDS criteria, n (%)

    		Kaiser Permanente early-onset sepsis risk [89]12 (86)2 (50)14 (78)

    		Epic Monitor [65]1 (7)1 (25)2 (11)

    		RALIS [69]1 (7)0 (0)1 (6)

    		Not specified0 (0)1 (25)1 (6)
    Silent or livec, n (%)

    		Live12 (86)4 (100)16 (89)

    		Silent1 (7)0 (0)1 (6)

    		Both (pre or post)1 (7)0 (0)1 (6)
    Related interventions, n (%)

    		Education and information resources8 (57)1 (25)9 (50)

    		None4 (29)3 (75)7 (39)

    		Sepsis protocol4 (29)0 (0)4 (22)

    		Order sets2 (14)0 (0)2 (11)

    		Other5 (36)1 (25)6 (33)
    Responding personnel, n (%)

    		Nurses4 (29)1 (25)5 (28)

    		Other clinicians10 (71)0 (0)10 (56)

    		Paramedics1 (7)1 (25)2 (11)

    		Not specified2 (14)2 (50)4 (22)
    Alert delivery, n (%)

    		Calculated by personnel10 (71)3 (75)13 (72)

    		Other2 (14)0 (0)2 (11)

    		Not specified2 (14)1 (25)3 (17)

    aCCDS: computerized clinical decision support.

    bThe percentages were calculated from the number of neonatal studies (n=18). As some studies have reported multiple characteristics for each category, there were more than 18 characteristics, therefore, the percentages add to more than 100%.

    cA live CCDS system is a system that is implemented and being used by clinicians in real time during the study. Silent systems are systems that have been implemented but do not alert clinicians during the study and thus do not influence treatment.

    Maternal Studies

    Only 2 studies—those by Davis et al [60] and Blumenthal et al [57]—have investigated CCDS systems for sepsis in pregnant or immediately postpartum populations. Both studies were abstracts and used quantitative methods. Blumenthal et al [57] used a before-after study design, whereas Davis et al [60] did not provide sufficient information for the study design to be determined. Davis et al [60] conducted a single-site, hospital-wide study in the United States, and Blumenthal et al [57] conducted a study at 3 sites but did not specify in which country. None of the studies reported on the number of participants. To identify maternal sepsis, Davis et al [60] used the obstetric-adjusted systemic inflammatory response syndrome (SIRS) criteria (comprising SIRS with the addition of fetal heart rate) plus organ dysfunction, whereas Blumenthal et al [57] used a maternal early warning score (comprising temperature plus heart rate, altered mental state, respiratory rate, and mean arterial pressure). Both studies investigated sepsis treatment and management outcomes, with Blumenthal et al [57] additionally investigating patient outcomes.


    Principal Findings

    This review comprehensively scoped the current literature on CCDS systems for early detection of sepsis in pediatric, neonatal, and maternal hospital populations. Overall, our findings highlight the scarcity of studies in these unique populations when compared with the general adult population, representing only 21% (33/157) of studies. Furthermore, only 64% (21/33) of studies were peer-reviewed journal articles. Given the high burden of sepsis in pediatric, neonatal, and maternal patients, this comparatively small number of studies is concerning [2-4,18] and underlines the critical need for future high-quality research into CCDS systems for these vulnerable populations. However, the rapid expansion of this field in recent years is encouraging, with all 33 studies published in the last 10 years and the majority (26/33, 79%) published in the last 5 years.

    Pediatric Sepsis

    Our findings emphasize the variability in pediatric studies that have evaluated the use of sepsis CCDS systems. In particular, we found great variability across the clinical criteria used for pediatric sepsis identification, with 18 different clinical criteria used in numerous combinations across 8 studies (Table 3). Furthermore, a range of gold standard definitions was applied, of which the most common was clinician discretion rather than published tools [12,85,86], highlighting the lack of a consensus definition and tool for pediatric sepsis identification. Hospital settings varied widely between studies, and numerous related interventions were implemented alongside the pediatric CCDS, with few similarities. This variability makes it difficult to compare studies and draw generalized conclusions from the literature. All studies were single cohort or before-after studies, highlighting the need for more robust study designs to provide stronger evidence regarding the use of CCDS systems.

    The heterogeneity in the clinical criteria used, both for the CCDS system and the gold standard definitions, can be attributed to a lack of current consensus regarding pediatric identification, risk stratification, and diagnosis. Although the definition of adult sepsis was updated in 2016 [1], followed by the publication of the quick sepsis-related organ failure assessment tool [90], the most recent pediatric sepsis consensus definition was in 2005 [12] and has exhibited numerous limitations [31,91,92]. An extensive study by Weiss et al [93] found an interrater agreement of only 0.57 between the 2005 consensus and physician diagnosis of pediatric sepsis, further emphasizing the inadequacies of the current consensus criteria in practice. Researchers have since attempted to adapt the quick sepsis-related organ failure assessment to the pediatric population or pediatric logistic organ dysfunction-2, a pediatric deterioration tool, to sepsis [29,30,94,95]. Preliminary results from these studies show promise, demonstrating moderate to high prognostic accuracy for poor patient outcomes, such as mortality and pediatric intensive care unit admission [29,30,94,95]. Critical to this challenge is the unique pathophysiology of pediatric sepsis, in which simply age-adjusting adult sepsis criteria is controversial and inadequate [91,96]. For example, hypotension is commonly used as a key indicator of septic shock in adults; however, it is less useful in children, as hypotension is typically not present until much later in the disease course [25,26,91]. In addition, symptoms considered key to adult sepsis identification, such as tachycardia and tachypnea, are common in febrile children regardless of disease severity and can often be present due to crying and distress [25,26,95]. Therefore, there have been numerous calls by both academics and clinicians for an updated pediatric consensus in recent years [13,43,91,95]. In 2019, the Society of Critical Care Medicine convened the Pediatric Sepsis Definition Taskforce to update the consensus criteria for pediatric sepsis identification [97]. Although they have recently published a systematic review investigating the individual factors, clinical criteria, or illness severity scores that are used to identify children with sepsis who are at higher risk of developing organ dysfunction or death, the task force has not yet released an updated definition [97]. The absence of an up-to-date consensus for defining or detecting pediatric sepsis has likely contributed to the high diversity of CCDS clinical criteria used in pediatric populations and the range of definitions used for gold standard pediatric sepsis detection. Our findings demonstrate the need for more robust evidence to investigate the appropriate clinical criteria for pediatric sepsis and reinforce the urgent need for an updated consensus on the definition of pediatric sepsis.

    Notably, an updated pediatric consensus must consider the extensive chronological and developmental age-dependent variability found in the pediatric population. For example, the pathophysiology of sepsis is expected to differ significantly among an adolescent, a child aged 5 years, and an infant aged 2 months. This will likely affect how different pediatric age groups present with sepsis, and accounting for these changes may not be as simple as adjusting the normal threshold of different vital signs according to age. This diversity needs to be studied and reflected in future consensus definitions and clinical criteria of the CCDS system.

    Neonatal Sepsis

    Our findings report considerable variation across neonatal studies, despite most studies evaluating the same CCDS system: the Kaiser Permanente early-onset sepsis risk calculator (KPC) [89]. In particular, the gestational age of the neonates included in the study varied considerably (Table 4). Most studies investigated moderate to late preterm and term infants, with cutoffs for gestational age ranging from ≥34 to >37 weeks [98] or infants within their first month of life. A single study [69] investigated very preterm infants at <33 weeks gestational age [98], indicating a key research gap, as preterm infants are at a considerably higher risk of sepsis and infection than full-term newborns [14,28,32,99]. A recent study [99] demonstrated that more than one-third (38%) of extremely preterm infants, defined as infants ≤28 weeks’ gestation, had late-onset sepsis. The included studies investigated a diverse range of outcomes, related interventions, and responding personnel. Large multisite studies would improve the generalizability of the literature and thus should be considered despite the substantial difficulty in undertaking them.

    Of the 18 neonatal studies included in this review, 14 (78%) investigated KPC [89]. This calculator combines the baseline early-onset sepsis incidence with maternal and infant characteristics and a clinical evaluation [89]. It aims to identify neonates at risk of early-onset sepsis, defined as sepsis within the first 72 hours after birth [28,87,88]. Under conventional sepsis management guidelines, many neonates are given potentially unnecessary antibiotic therapy as a precaution against sepsis, resulting in unintended negative effects [14,87]. A systematic review and meta-analysis performed by Achten et al [100] demonstrated that the use of KPC was associated with a reduction in antibiotic use. However, a more recent meta-analysis [101] showed that the KPC missed many cases of early-onset sepsis compared with the UK National Institute for Health and Care Excellence guidelines. This results in delayed or missed treatment for these neonates and suggests that further evaluation of the calculator is required [101]. In addition, the KPC is only designed for predicting sepsis risk in infants born at ≥34 weeks’ gestation within a very narrow early-onset sepsis time frame [87-89]. Our review identified only 17% (3/18) of neonatal studies that did not examine early-onset sepsis, with 6% (1/18) investigating late-onset sepsis and 11% (2/18) investigating general neonatal sepsis. Late-onset neonatal sepsis, often defined as sepsis occurring ≥3 days after birth, is a leading cause of mortality in vulnerable preterm infants [28,32,99,102]. This calls attention to a clear knowledge gap for future research into CCDS systems for neonatal sepsis occurring outside the initial 72 hours of life.

    To date, no consensus definition has been developed for neonatal sepsis [15,16,28,103]. As the neonatal population is uniquely different from adults and older children, current adult and pediatric clinical criteria cannot be simply adapted [15,32,103]. A recently published systematic review [16] highlighted the variance in the currently used definitions of neonatal sepsis in randomized controlled trials. Surprisingly, the most commonly used definition was microbiological culture by itself or in combination with clinical signs and symptoms, despite the proven low sensitivity of this method and the high incidence of culture-negative sepsis among the neonatal population [14,16,102]. Similarly, some studies included in this review required a positive culture test to diagnose neonatal sepsis. A consensus on the definition of neonatal sepsis is needed to better identify suspected neonatal sepsis in clinical practice, for research studies, and to improve antibiotic stewardship in newborns [14,15,28,103]. Furthermore, any consensus criterion must acknowledge the age-related variability inherent to the neonatal population, as sepsis pathophysiology differs considerably between a preterm neonate and an infant in their first month of life [103].

    Maternal Sepsis

    Despite the devastating consequences of sepsis in pregnant and immediately postpartum women [3,17,18], our comprehensive literature search identified only 2 studies that evaluated the use of CCDS systems for maternal sepsis. Pregnancy involves extensive physiological, hormonal, and psychological changes, which may mask the common symptoms of sepsis, resulting in delayed diagnosis and treatment [3,19,104]. A systematic review by Bauer et al [104] demonstrated that healthy pregnant women during the second and third trimesters often demonstrate considerable overlap with the SIRS criteria. This alteration of the usual physiological state must be represented in CCDS systems to ensure that sepsis in pregnant and immediately postpartum women is detected early, without the risk of unnecessary treatment in healthy patients. The lack of high-quality peer-reviewed studies in this population underlines a concerning knowledge gap in the literature, for which further research is urgently needed.

    Usability and Cost of CCDS Systems

    The usability of any health intervention technology is critical for its successful implementation [105-108]. Therefore, investigating the usability of CCDS systems is essential for developing efficient and functional systems. In particular, alarm fatigue is a well-established usability concern for CCDS systems [109]. Alarm fatigue occurs when clinicians become desensitized to frequent inappropriate alarms and begin ignoring or overriding alerts, reducing the effectiveness of alert systems and potentially impacting patient outcomes [109,110]. To prevent alarm fatigue, CCDS systems must be carefully calibrated to avoid unnecessary frequent alerting [109,110]. None of the studies reported in this review investigated alarm fatigue in response to the implemented CCDS system, despite its importance for successful CCDS use.

    Understanding the cost or cost-effectiveness of an intervention supports policy and clinical decision-making when determining resource allocation under limited health care budgets [111]. This is especially true for sepsis, which represents a large financial burden on the health system through both acute hospital care and long-term treatment and rehabilitation [112,113]. Of the 33 studies included in this review, only 4 (12%) investigated outcomes related to cost or usability, 1 (3%) in pediatric and 3 (9%) in neonatal populations, demonstrating a clear evidence gap for future research.

    Strengths and Limitations

    This review comprehensively searched the available literature, both peer reviewed and gray, on the use of CCDS systems for inpatients with neonatal, pediatric, and maternal sepsis. Owing to time and resource constraints, the searches were limited to studies available in the English language and thus may have missed publications in other languages. Furthermore, the data extraction was performed by only 1 reviewer (KA). To limit any consequential data entry errors, the extraction form was extensively piloted, and any issues were cross-checked and fully discussed with the review team.

    Conclusions

    Our findings have illustrated a comparative scarcity of studies investigating CCDS systems in pediatric, neonatal, and maternal inpatients, despite their high sepsis burden. Further research is needed to evaluate CCDS systems for the early detection of sepsis in these vulnerable populations. We identified extensive variation in the clinical criteria and gold standard definitions used by pediatric CCDS systems, and our findings reinforce calls for updated pediatric and neonatal sepsis consensus definitions. The review also shows a clear absence of studies investigating CCDS systems for sepsis identification in maternal inpatients, high-risk preterm populations, and neonates outside the first 72 hours of life. Finally, our review demonstrated a lack of studies investigating the usability and cost of CCDS systems, both of which are key to their effectiveness and sustainability. In conclusion, our review has identified substantial and important knowledge gaps in the literature evaluating CCDS systems for the early detection of sepsis in pediatric, neonatal, and maternal populations, which would benefit greatly from future research.

    Acknowledgments

    The authors would like to thank Mr Jeremy Cullis, an experienced clinical librarian, for his expert advice and assistance in designing our final MEDLINE search strategy and in translating it for other databases. The authors have received no financial support or funding for this study.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist.

    PDF File (Adobe PDF File), 516 KB
    ‎
    Multimedia Appendix 2

    MEDLINE search strategy.

    PDF File (Adobe PDF File), 75 KB
    ‎
    Multimedia Appendix 3

    Adjustments made to data charting form.

    PDF File (Adobe PDF File), 283 KB
    ‎
    Multimedia Appendix 4

    Definitions of categories combining multiple subgroups.

    PDF File (Adobe PDF File), 175 KB
    ‎
    Multimedia Appendix 5

    Main study characteristics table.

    PDF File (Adobe PDF File), 238 KB
    ‎
    Multimedia Appendix 6

    Study setting by country.

    PDF File (Adobe PDF File), 122 KB
    1. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016 Feb 23;315(8):801-810 [FREE Full text] [CrossRef] [Medline]
    2. Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the Global Burden of Disease Study. Lancet 2020 Jan 18;395(10219):200-211 [FREE Full text] [CrossRef] [Medline]
    3. Bonet M, Nogueira Pileggi V, Rijken MJ, Coomarasamy A, Lissauer D, Souza JP, et al. Towards a consensus definition of maternal sepsis: results of a systematic review and expert consultation. Reprod Health 2017 May 30;14(1):67 [FREE Full text] [CrossRef] [Medline]
    4. Fleischmann C, Reichert F, Cassini A, Horner R, Harder T, Markwart R, et al. Global incidence and mortality of neonatal sepsis: a systematic review and meta-analysis. Arch Dis Child 2021 Jan 22;106(8):745-752 [FREE Full text] [CrossRef] [Medline]
    5. Li L, Sunderland N, Rathnayake K, Westbrook JI. Epidemiology of Sepsis in Australian Public Hospitals: A Mixed Methods, National Longitudinal Study (2013-2018). ACSQHC. 2020.   URL: https:/​/www.​safetyandquality.gov.au/​sites/​default/​files/​2020-05/​epidemiology_of_sepsis_-_february_2020_002.​pdf [accessed 2021-11-03]
    6. Killien EY, Farris RW, Watson RS, Dervan LA, Zimmerman JJ. Health-related quality of life among survivors of pediatric sepsis. Pediatr Crit Care Med 2019 Jun;20(6):501-509 [FREE Full text] [CrossRef] [Medline]
    7. Weiss SL, Fitzgerald JC, Pappachan J, Wheeler D, Jaramillo-Bustamante JC, Salloo A, Sepsis Prevalence‚ Outcomes‚Therapies (SPROUT) Study InvestigatorsPediatric Acute Lung InjurySepsis Investigators (PALISI) Network. Global epidemiology of pediatric severe sepsis: the sepsis prevalence, outcomes, and therapies study. Am J Respir Crit Care Med 2015 May 15;191(10):1147-1157 [FREE Full text] [CrossRef] [Medline]
    8. Farris RW, Weiss NS, Zimmerman JJ. Functional outcomes in pediatric severe sepsis: further analysis of the researching severe sepsis and organ dysfunction in children: a global perspective trial. Pediatr Crit Care Med 2013 Nov;14(9):835-842 [FREE Full text] [CrossRef] [Medline]
    9. Prout AJ, Talisa VB, Carcillo JA, Angus DC, Chang CH, Yende S. Epidemiology of readmissions after sepsis hospitalization in children. Hosp Pediatr 2019 Apr;9(4):249-255 [FREE Full text] [CrossRef] [Medline]
    10. Savioli K, Rouse C, Susi A, Gorman G, Hisle-Gorman E. Suspected or known neonatal sepsis and neurodevelopmental delay by 5 years. J Perinatol 2018 Nov;38(11):1573-1580. [CrossRef] [Medline]
    11. Cai S, Thompson DK, Anderson PJ, Yang JY. Short- and long-term neurodevelopmental outcomes of very preterm infants with neonatal sepsis: a systematic review and meta-analysis. Children (Basel) 2019 Dec 01;6(12):131 [FREE Full text] [CrossRef] [Medline]
    12. Goldstein B, Giroir B, Randolph A, International Consensus Conference on Pediatric Sepsis. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med 2005 Jan;6(1):2-8. [CrossRef] [Medline]
    13. Weiss SL, Peters MJ, Alhazzani W, Agus MS, Flori HR, Inwald DP, et al. Surviving sepsis campaign international guidelines for the management of septic shock and sepsis-associated organ dysfunction in children. Pediatr Crit Care Med 2020 Feb;21(2):e52-106. [CrossRef] [Medline]
    14. Wynn JL, Polin RA. Progress in the management of neonatal sepsis: the importance of a consensus definition. Pediatr Res 2018 Jan;83(1-1):13-15. [CrossRef] [Medline]
    15. McGovern M, Giannoni E, Kuester H, Turner MA, van den Hoogen A, Bliss JM, Infection‚ Inflammation‚ ImmunologyImmunisation (I4) section of the ESPR. Challenges in developing a consensus definition of neonatal sepsis. Pediatr Res 2020 Jul;88(1):14-26. [CrossRef] [Medline]
    16. Hayes R, Hartnett J, Semova G, Murray C, Murphy K, Carroll L, Infection‚ Inflammation‚ ImmunologyImmunisation (I4) section of the European Society for Paediatric Research (ESPR). Neonatal sepsis definitions from randomised clinical trials. Pediatr Res 2021 Nov 06 (forthcoming). [CrossRef] [Medline]
    17. WHO Global Maternal Sepsis Study (GLOSS) Research Group. Frequency and management of maternal infection in health facilities in 52 countries (GLOSS): a 1-week inception cohort study. Lancet Glob Health 2020 May;8(5):e661-e671 [FREE Full text] [CrossRef] [Medline]
    18. Say L, Chou D, Gemmill A, Tunçalp Ö, Moller A, Daniels J, et al. Global causes of maternal death: a WHO systematic analysis. Lancet Glob Health 2014 Jun;2(6):e323-e333 [FREE Full text] [CrossRef] [Medline]
    19. Escobar MF, Echavarría MP, Zambrano MA, Ramos I, Kusanovic JP. Maternal sepsis. Am J Obstet Gynecol MFM 2020 Aug;2(3):100149. [CrossRef] [Medline]
    20. Scott S, Kendall L, Gomez P, Howie SR, Zaman SM, Ceesay S, et al. Effect of maternal death on child survival in rural West Africa: 25 years of prospective surveillance data in The Gambia. PLoS One 2017;12(2):e0172286 [FREE Full text] [CrossRef] [Medline]
    21. Seymour CW, Gesten F, Prescott HC, Friedrich ME, Iwashyna TJ, Phillips GS, et al. Time to treatment and mortality during mandated emergency care for sepsis. N Engl J Med 2017 Jun 08;376(23):2235-2244 [FREE Full text] [CrossRef] [Medline]
    22. Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016. Crit Care Med 2017 Mar;45(3):486-552. [CrossRef] [Medline]
    23. Weiss SL, Fitzgerald JC, Balamuth F, Alpern ER, Lavelle J, Chilutti M, et al. Delayed antimicrobial therapy increases mortality and organ dysfunction duration in pediatric sepsis. Crit Care Med 2014 Nov;42(11):2409-2417 [FREE Full text] [CrossRef] [Medline]
    24. Cecconi M, Evans L, Levy M, Rhodes A. Sepsis and septic shock. Lancet 2018 Jul 07;392(10141):75-87. [CrossRef] [Medline]
    25. Schlapbach LJ, Weiss SL, Wolf J. Reducing collateral damage from mandates for time to antibiotics in pediatric sepsis-primum non nocere. JAMA Pediatr 2019 May 01;173(5):409-410. [CrossRef] [Medline]
    26. Cruz AT, Lane RD, Balamuth F, Aronson PL, Ashby DW, Neuman MI, et al. Updates on pediatric sepsis. J Am Coll Emerg Physicians Open 2020 Oct;1(5):981-993 [FREE Full text] [CrossRef] [Medline]
    27. Kim F, Polin RA, Hooven TA. Neonatal sepsis. BMJ 2020 Oct 01;371:m3672. [CrossRef] [Medline]
    28. Shane AL, Sánchez PJ, Stoll BJ. Neonatal sepsis. Lancet 2017 Oct 14;390(10104):1770-1780. [CrossRef] [Medline]
    29. Matics TJ, Sanchez-Pinto LN. Adaptation and validation of a pediatric sequential organ failure assessment score and evaluation of the sepsis-3 definitions in critically ill children. JAMA Pediatr 2017 Oct 02;171(10):e172352 [FREE Full text] [CrossRef] [Medline]
    30. Leclerc F, Duhamel A, Deken V, Grandbastien B, Leteurtre S, Groupe Francophone de Réanimation et Urgences Pédiatriques (GFRUP). Can the pediatric logistic organ dysfunction-2 score on day 1 be used in clinical criteria for sepsis in children? Pediatr Crit Care Med 2017 Aug;18(8):758-763. [CrossRef] [Medline]
    31. Schlapbach LJ, MacLaren G, Festa M, Alexander J, Erickson S, Beca J, et al. Prediction of pediatric sepsis mortality within 1 h of intensive care admission. Intensive Care Med 2017 Aug;43(8):1085-1096. [CrossRef] [Medline]
    32. Wynn JL, Polin RA. A neonatal sequential organ failure assessment score predicts mortality to late-onset sepsis in preterm very low birth weight infants. Pediatr Res 2020 Jul;88(1):85-90 [FREE Full text] [CrossRef] [Medline]
    33. Edwards SE, Grobman WA, Lappen JR, Winter C, Fox R, Lenguerrand E, et al. Modified obstetric early warning scoring systems (MOEWS): validating the diagnostic performance for severe sepsis in women with chorioamnionitis. Am J Obstet Gynecol 2015 Apr;212(4):536.e1-536.e8. [CrossRef] [Medline]
    34. Aarvold AB, Ryan HM, Magee LA, von Dadelszen P, Fjell C, Walley KR. Multiple organ dysfunction score is superior to the obstetric-specific sepsis in obstetrics score in predicting mortality in septic obstetric patients. Crit Care Med 2017 Jan;45(1):e49-e57 [FREE Full text] [CrossRef] [Medline]
    35. Bhattacharjee P, Edelson DP, Churpek MM. Identifying patients with sepsis on the hospital wards. Chest 2017 Apr;151(4):898-907 [FREE Full text] [CrossRef] [Medline]
    36. Makam AN, Nguyen OK, Auerbach AD. Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: a systematic review. J Hosp Med 2015 Jun;10(6):396-402 [FREE Full text] [Medline]
    37. Wulff A, Montag S, Marschollek M, Jack T. Clinical decision-support systems for detection of systemic inflammatory response syndrome, sepsis, and septic shock in critically ill patients: a systematic review. Methods Inf Med 2019 Dec;58(S 02):e43-e57 [FREE Full text] [CrossRef] [Medline]
    38. Joshi M, Ashrafian H, Arora S, Khan S, Cooke G, Darzi A. Digital alerting and outcomes in patients with sepsis: systematic review and meta-analysis. J Med Internet Res 2019 Dec 20;21(12):e15166 [FREE Full text] [CrossRef] [Medline]
    39. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020;3:17 [FREE Full text] [CrossRef] [Medline]
    40. Petersen C, Smith J, Freimuth RR, Goodman KW, Jackson GP, Kannry J, et al. Recommendations for the safe, effective use of adaptive CDS in the US healthcare system: an AMIA position paper. J Am Med Inform Assoc 2021 Mar 18;28(4):677-684 [FREE Full text] [CrossRef] [Medline]
    41. Li L, Ackermann K, Baker J, Westbrook J. Use and evaluation of computerized clinical decision support systems for early detection of sepsis in hospitals: protocol for a scoping review. JMIR Res Protoc 2020 Nov 20;9(11):e24899 [FREE Full text] [CrossRef] [Medline]
    42. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018 Oct 02;169(7):467-473 [FREE Full text] [CrossRef] [Medline]
    43. Schlapbach LJ. Paediatric sepsis. Curr Opin Infect Dis 2019 Oct;32(5):497-504. [CrossRef] [Medline]
    44. Ackermann K, Baker J, Green M, Fullick M, Varinli H, Westbrook J, et al. Computerized clinical decision support systems for the early detection of sepsis among adult inpatients: scoping review. J Med Internet Res 2022 Feb 23;24(2):e31083 [FREE Full text] [CrossRef] [Medline]
    45. Viswanathan M, Berkman N, Dryden D, Hartling L. Assessing risk of bias and confounding in observational studies of interventions or exposures: further development of the RTI item bank. In: AHRQ Methods for Effective Health Care. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013.
    46. Ranganathan P, Aggarwal R. Study designs: part 1 - An overview and classification. Perspect Clin Res 2018;9(4):184-186 [FREE Full text] [Medline]
    47. Aggarwal R, Ranganathan P. Study designs: part 2 - descriptive studies. Perspect Clin Res 2019;10(1):34-36 [FREE Full text] [CrossRef] [Medline]
    48. Ranganathan P, Aggarwal R. Study designs: part 3 - Analytical observational studies. Perspect Clin Res 2019;10(2):91-94 [FREE Full text] [CrossRef] [Medline]
    49. Aggarwal R, Ranganathan P. Study designs: part 4 - Interventional studies. Perspect Clin Res 2019;10(3):137-139 [FREE Full text] [CrossRef] [Medline]
    50. Aggarwal R, Ranganathan P. Study designs: part 5 - interventional studies (II). Perspect Clin Res 2019;10(4):183-186 [FREE Full text] [CrossRef] [Medline]
    51. Ergonomics of human-system interaction - Part 11: usability: definitions and concepts. ISO standard no. 9241-11:2018(EN). The International Organization for Standardization. 2018.   URL: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en [accessed 2021-02-26]
    52. Achten NB, Dorigo-Zetsma JW, van der Linden PD, van Brakel M, Plötz FB. Sepsis calculator implementation reduces empiric antibiotics for suspected early-onset sepsis. Eur J Pediatr 2018 May;177(5):741-746. [CrossRef] [Medline]
    53. Achten NB, Visser DH, Tromp E, Groot W, van Goudoever JB, Plötz FB. Early onset sepsis calculator implementation is associated with reduced healthcare utilization and financial costs in late preterm and term newborns. Eur J Pediatr 2020 May;179(5):727-734 [FREE Full text] [CrossRef] [Medline]
    54. Arora V, Strunk D, Furqan SH, Schweig L, Lefaiver C, George J, et al. Optimizing antibiotic use for early onset sepsis: a tertiary NICU experience. J Neonatal Perinatal Med 2019;12(3):301-312. [CrossRef] [Medline]
    55. Balamuth F, Alpern ER, Abbadessa MK, Hayes K, Schast A, Lavelle J, et al. Improving recognition of pediatric severe sepsis in the emergency department: contributions of a vital sign-based electronic alert and bedside clinician identification. Ann Emerg Med 2017 Dec;70(6):759-68.e2 [FREE Full text] [CrossRef] [Medline]
    56. Beavers JB, Bai S, Perry J, Simpson J, Peeples S. Implementation and evaluation of the early-onset sepsis risk calculator in a high-risk university nursery. Clin Pediatr (Phila) 2018 Aug;57(9):1080-1085. [CrossRef] [Medline]
    57. Blumenthal E, Hooshvar N, Tancioco V, Newman R, Senderoff D, McNulty J. 238: Maternal Early Warning Trigger tool improves clinical care in large community hospital system. In: Proceedings of the SMFM 40th Annual Meeting--The Pregnancy Meeting. 2020 Presented at: SMFM 40th Annual Meeting--The Pregnancy Meeting; Feb 3-8, 2020; Grapevine, TX, United States   URL: https://doi.org/10.1016/j.ajog.2019.11.254 [CrossRef]
    58. Coffman Z, Smith J, Hubbard C, Chen E, McDaniel L. Using the electronic medical record to create a pediatric sepsis alert. In: Proceedings of the National Conference on Education. 2018 Presented at: National Conference on Education; Oct 22-25, 2016; San Francisco, CA, United States   URL: https:/​/publications.​aap.org/​pediatrics/​article/​141/​1_MeetingAbstract/​444/​1975/​Using-the-Electronic-Medical-Record-to-Create-a
    59. Cruz AT, Williams EA, Graf JM, Perry AM, Harbin DE, Wuestner ER, et al. Test characteristics of an automated age- and temperature-adjusted tachycardia alert in pediatric septic shock. Pediatr Emerg Care 2012 Sep;28(9):889-894. [CrossRef] [Medline]
    60. Davis T, Beigi R, Petticord V, Manetta L, Simhan H. Accurate identification of obstetric sepsis using e-record tools UPMC Magee-women’s hospital. Pennsylvania Patient Safety Advisory 2018;15(supp. 1):53 [FREE Full text]
    61. Dewan M, Vidrine R, Zackoff M, Paff Z, Seger B, Pfeiffer S, et al. Design, implementation, and validation of a pediatric ICU sepsis prediction tool as clinical decision support. Appl Clin Inform 2020 Mar;11(2):218-225 [FREE Full text] [CrossRef] [Medline]
    62. Dhudasia MB, Mukhopadhyay S, Puopolo KM. Implementation of the sepsis risk calculator at an academic birth hospital. Hosp Pediatr 2018 May;8(5):243-250. [CrossRef] [Medline]
    63. Eason J, Ward H, Danko O, Richardson K, Vaitkute R, McKeon-Carter R. Early-onset sepsis: can we screen fewer babies safely? Arch Dis Child 2021 Jan;106(1):86-88. [CrossRef] [Medline]
    64. Eisenberg M, Freiman E, Capraro A, Madden K, Monuteaux MC, Hudgins J, et al. Comparison of manual and automated sepsis screening tools in a pediatric emergency department. Pediatrics 2021 Feb;147(2):e2020022590. [CrossRef] [Medline]
    65. Emmanuel J, Torres A. The impact of automated electronic surveillance of electronic medical records on pediatric inpatient care. Cureus 2018 Oct 01;10(10):e3395 [FREE Full text] [CrossRef] [Medline]
    66. Fowler NT, Garcia M, Hankins C. Impact of integrating a neonatal early-onset sepsis risk calculator into the electronic health record. Pediatr Qual Saf 2019;4(6):e235 [FREE Full text] [CrossRef] [Medline]
    67. Gievers LL, Sedler J, Phillipi CA, Dukhovny D, Geddes J, Graven P, et al. Implementation of the sepsis risk score for chorioamnionitis-exposed newborns. J Perinatol 2018 Nov;38(11):1581-1587. [CrossRef] [Medline]
    68. Goyack L, Ma D, Miller J, Ye G, Mattox T, Torres A. 880: Evaluation of an automated electronic surveillance software program of an electronic medical record. In: Proceedings of the 45th Critical Care Congress of the Society of Critical Care Medicine, SCCM 2015. 2015 Presented at: 45th Critical Care Congress of the Society of Critical Care Medicine, SCCM 2015; Feb 20-24, 2016; Orlando, FL, United States   URL: https://doi.org/10.1097/01.ccm.0000474708.59510.b5 [CrossRef]
    69. Gur I, Riskin A, Markel G, Bader D, Nave Y, Barzilay B, et al. Pilot study of a new mathematical algorithm for early detection of late-onset sepsis in very low-birth-weight infants. Am J Perinatol 2015 Mar;32(4):321-330. [CrossRef] [Medline]
    70. Klingaman C, King L, Neff-Bulger M. Improved newborn care: evidence-based protocol for the evaluation and management of early-onset sepsis. Am J Med Qual 2018;33(1):106. [CrossRef] [Medline]
    71. Lloyd JK, Ahrens EA, Clark D, Dachenhaus T, Nuss KE. Automating a manual sepsis screening tool in a pediatric emergency department. Appl Clin Inform 2018 Oct;9(4):803-808 [FREE Full text] [CrossRef] [Medline]
    72. Mahdally S, Kim S. Implementing a model to predict risk of neonatal sepsis in late-preterm and term infants: a quality improvement initiative. In: Proceedings of the American Academy of Pediatrics National Conference and Exhibition 2017. 2018 Presented at: American Academy of Pediatrics National Conference and Exhibition 2017; Sep 16-19, 2017; Chicago, IL, United States   URL: https:/​/publications.​aap.org/​pediatrics/​article/​142/​1_MeetingAbstract/​193/​2465/​Implementing-a-Model-to-Predict-Risk-of-Neonatal
    73. Mangubat PM, Shah S. 105 comparing accuracy of 2 phases of a pediatric electronic severe sepsis screening algorithm. In: Proceedings of the Critical Care Congress 2015. 2014 Presented at: Critical Care Congress 2015; Jan 17-21, 2015; Phoenix, AZ, United States   URL: https://doi.org/10.1097/01.ccm.0000457602.28660.aa [CrossRef]
    74. Salomon J, Serrao K, Guardiola J, Bonura A. 1335: Early detection of pediatric sepsis using an electronic medical record-based screening tool. In: Proceedings of the 46th Critical Care Congress of the Society of Critical Care Medicine, SCCM 2016. 2016 Presented at: 46th Critical Care Congress of the Society of Critical Care Medicine, SCCM 2016; Jan 21-25, 2017; Honolulu, HI, United States   URL: https://doi.org/10.1097/01.ccm.0000510009.95039.f3 [CrossRef]
    75. Sharma V, Adkisson C, Gupta K. Managing infants exposed to maternal chorioamnionitis by the use of early-onset sepsis calculator. Glob Pediatr Health 2019;6:2333794X19833711 [FREE Full text] [CrossRef] [Medline]
    76. Skey D, Walters J, Surujdyal J. Limiting newborn antibiotic exposure through refined sepsis screening. In: Proceedings of the American Academy of Pediatrics National Conference and Exhibition 2017. 2018 Presented at: American Academy of Pediatrics National Conference and Exhibition 2017; Sep 16-19, 2017; Chicago, IL, United States   URL: https:/​/publications.​aap.org/​pediatrics/​article/​142/​1_MeetingAbstract/​574/​2994/​Limiting-Newborn-Antibiotic-Exposure-through
    77. Stinson HR, Viteri S, Koetter P, Stevens E, Remillard K, Parlow R, et al. Early experience with a novel strategy for assessment of sepsis risk: the shock huddle. Pediatr Qual Saf 2019;4(4):e197 [FREE Full text] [CrossRef] [Medline]
    78. Stipelman CH, Smith ER, Diaz-Ochu M, Spackman J, Stoddard G, Kawamoto K, et al. Early-onset sepsis risk calculator integration into an electronic health record in the nursery. Pediatrics 2019 Aug;144(2):e20183464. [CrossRef] [Medline]
    79. Strunk T, Buchiboyina A, Sharp M, Nathan E, Doherty D, Patole S. Implementation of the neonatal sepsis calculator in an Australian tertiary perinatal centre. Neonatology 2018;113(4):379-382. [CrossRef] [Medline]
    80. Torres A, Goyack L, Negron J, Miller J, Ye G, Lawless S. 821: Automated electronic surveillance of electronic medical records for shock in pediatric inpatients. In: Proceedings of the 45th Critical Care Congress of the Society of Critical Care Medicine, SCCM 2015. 2015 Presented at: 45th Critical Care Congress of the Society of Critical Care Medicine, SCCM 2015; Feb 20-24, 2016; Orlando, FL, United States p. 206-207   URL: https://doi.org/10.1097/01.ccm.0000474649.45155.26 [CrossRef]
    81. Vidrine R, Zackoff M, Paff Z, Seger B, Satterlee M, Buenaventura E, et al. Improving timely recognition and treatment of sepsis in the pediatric ICU. Jt Comm J Qual Patient Saf 2020 May;46(5):299-307. [CrossRef] [Medline]
    82. Viteri S, Koetter P, Stinson H, Stevens E, Frizzola M. 1541: Comparison of a pediatric septic shock electronic screening tool and pediatric early warning score. In: Proceedings of the 47th Society of Critical Care Medicine Critical Care Congress, SCCM 2018. 2018 Presented at: 47th Society of Critical Care Medicine Critical Care Congress, SCCM 2018; Feb 25-28, 2018; San Antonio, TX, United States   URL: https://doi.org/10.1097/01.ccm.0000529542.20350.1a [CrossRef]
    83. West A, Hallman M, Giles K, Guynn A, May W, Shah S. 1540: Accuracy of detecting clinically relevant severe sepsis in children using a real-time EMR algorithm. In: Proceedings of the 47th Society of Critical Care Medicine Critical Care Congress, SCCM 2018. 2018 Presented at: 47th Society of Critical Care Medicine Critical Care Congress, SCCM 2018; Feb 25-28, 2018; San Antonio, TX, United States   URL: https://doi.org/10.1097/01.ccm.0000529541.26188.44 [CrossRef]
    84. Zayek M, Bhat J, Bonner K, Blake M, Peevy K, Jha OP, et al. Implementation of a modified neonatal early-onset sepsis calculator in well-baby nursery: a quality improvement study. Pediatr Qual Saf 2020;5(4):e330 [FREE Full text] [CrossRef] [Medline]
    85. Lane RD, Funai T, Reeder R, Larsen GY. High reliability pediatric septic shock quality improvement initiative and decreasing mortality. Pediatrics 2016 Oct;138(4):e20154153. [CrossRef] [Medline]
    86. Larsen GY, Brilli R, Macias CG, Niedner M, Auletta JJ, Balamuth F, Improving Pediatric Sepsis Outcomes Collaborative Investigators. Development of a quality improvement learning collaborative to improve pediatric sepsis outcomes. Pediatrics 2021 Jan;147(1):e20201434 [FREE Full text] [CrossRef] [Medline]
    87. Escobar GJ, Puopolo KM, Wi S, Turk BJ, Kuzniewicz MW, Walsh EM, et al. Stratification of risk of early-onset sepsis in newborns ≥ 34 weeks' gestation. Pediatrics 2014 Jan;133(1):30-36 [FREE Full text] [CrossRef] [Medline]
    88. Puopolo KM, Draper D, Wi S, Newman TB, Zupancic J, Lieberman E, et al. Estimating the probability of neonatal early-onset infection on the basis of maternal risk factors. Pediatrics 2011 Nov;128(5):e1155-e1163 [FREE Full text] [CrossRef] [Medline]
    89. Neonatal early-onset sepsis calculator. Kaiser Permanente Division of Research.   URL: https://neonatalsepsiscalculator.kaiserpermanente.org/ [accessed 2022-04-13]
    90. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 2016 Feb 23;315(8):762-774 [FREE Full text] [CrossRef] [Medline]
    91. Schlapbach LJ, Kissoon N. Defining pediatric sepsis. JAMA Pediatr 2018 Apr 01;172(4):312-314. [CrossRef] [Medline]
    92. Scott HF, Deakyne SJ, Woods JM, Bajaj L. The prevalence and diagnostic utility of systemic inflammatory response syndrome vital signs in a pediatric emergency department. Acad Emerg Med 2015 Apr;22(4):381-389 [FREE Full text] [CrossRef] [Medline]
    93. Weiss SL, Fitzgerald JC, Maffei FA, Kane JM, Rodriguez-Nunez A, Hsing DD, SPROUT Study InvestigatorsPediatric Acute Lung InjurySepsis Investigators Network. Discordant identification of pediatric severe sepsis by research and clinical definitions in the SPROUT international point prevalence study. Crit Care 2015 Sep 16;19:325 [FREE Full text] [CrossRef] [Medline]
    94. van Nassau SC, van Beek RH, Driessen GJ, Hazelzet JA, van Wering HM, Boeddha NP. Translating sepsis-3 criteria in children: prognostic accuracy of age-adjusted quick sofa score in children visiting the emergency department with suspected bacterial infection. Front Pediatr 2018;6:266 [FREE Full text] [CrossRef] [Medline]
    95. Schlapbach LJ, Straney L, Bellomo R, MacLaren G, Pilcher D. Prognostic accuracy of age-adapted SOFA, SIRS, PELOD-2, and qSOFA for in-hospital mortality among children with suspected infection admitted to the intensive care unit. Intensive Care Med 2018 Feb;44(2):179-188 [FREE Full text] [CrossRef] [Medline]
    96. Weiss SL, Deutschman CS. Are septic children really just "septic little adults"? Intensive Care Med 2018 Mar;44(3):392-394 [FREE Full text] [CrossRef] [Medline]
    97. Menon K, Schlapbach LJ, Akech S, Argent A, Biban P, Carrol ED, Pediatric Sepsis Definition Taskforce of the Society of Critical Care Medicine. Criteria for pediatric sepsis-a systematic review and meta-analysis by the pediatric sepsis definition taskforce. Crit Care Med 2022 Jan 01;50(1):21-36 [FREE Full text] [CrossRef] [Medline]
    98. Preterm birth. World Health Organization.   URL: https://www.who.int/news-room/fact-sheets/detail/preterm-birth [accessed 2022-04-13]
    99. Greenberg RG, Kandefer S, Do BT, Smith PB, Stoll BJ, Bell EF, Eunice Kennedy Shriver National Institute of Child HealthHuman Development Neonatal Research Network. Late-onset sepsis in extremely premature infants: 2000-2011. Pediatr Infect Dis J 2017 Aug;36(8):774-779 [FREE Full text] [CrossRef] [Medline]
    100. Achten NB, Klingenberg C, Benitz WE, Stocker M, Schlapbach LJ, Giannoni E, et al. Association of use of the neonatal early-onset sepsis calculator with reduction in antibiotic therapy and safety: a systematic review and meta-analysis. JAMA Pediatr 2019 Nov 01;173(11):1032-1040 [FREE Full text] [CrossRef] [Medline]
    101. Pettinger KJ, Mayers K, McKechnie L, Phillips B. Sensitivity of the Kaiser Permanente early-onset sepsis calculator: a systematic review and meta-analysis. EClinicalMedicine 2020 Feb;19:100227 [FREE Full text] [CrossRef] [Medline]
    102. Bekhof J, Reitsma JB, Kok JH, Van Straaten IH. Clinical signs to identify late-onset sepsis in preterm infants. Eur J Pediatr 2013 Apr;172(4):501-508. [CrossRef] [Medline]
    103. Molloy EJ, Wynn JL, Bliss J, Koenig JM, Keij FM, McGovern M, on behalf of the Infection‚ Inflammation‚ ImmunologyImmunisation (I4) section of the ESPR. Neonatal sepsis: need for consensus definition, collaboration and core outcomes. Pediatr Res 2020 Jul;88(1):2-4. [CrossRef] [Medline]
    104. Bauer ME, Bauer ST, Rajala B, MacEachern MP, Polley LS, Childers D, et al. Maternal physiologic parameters in relationship to systemic inflammatory response syndrome criteria: a systematic review and meta-analysis. Obstet Gynecol 2014 Sep;124(3):535-541. [CrossRef] [Medline]
    105. Miller K, Capan M, Weldon D, Noaiseh Y, Kowalski R, Kraft R, et al. The design of decisions: matching clinical decision support recommendations to Nielsen's design heuristics. Int J Med Inform 2018 Sep;117:19-25 [FREE Full text] [CrossRef] [Medline]
    106. Miller K, Mosby D, Capan M, Kowalski R, Ratwani R, Noaiseh Y, et al. Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support. J Am Med Inform Assoc 2018 May 01;25(5):585-592 [FREE Full text] [CrossRef] [Medline]
    107. Sagar K, Saha A. A systematic review of software usability studies. Int J Inf Technol 2017:1-24 [FREE Full text] [CrossRef]
    108. Ellsworth MA, Dziadzko M, O'Horo JC, Farrell AM, Zhang J, Herasevich V. An appraisal of published usability evaluations of electronic health records via systematic review. J Am Med Inform Assoc 2017 Jan;24(1):218-226 [FREE Full text] [CrossRef] [Medline]
    109. Jankovic I, Chen JH. Clinical decision support and implications for the clinician burnout crisis. Yearb Med Inform 2020 Aug;29(1):145-154 [FREE Full text] [CrossRef] [Medline]
    110. The Lancet Respiratory Medicine. Crying wolf: the growing fatigue around sepsis alerts. Lancet Respir Med 2018 Mar;6(3):161. [CrossRef] [Medline]
    111. Ernst FR, Levy H, Qualy RL. Simplified pharmacoeconomics of critical care and severe sepsis. J Intensive Care Med 2007;22(5):283-293. [CrossRef] [Medline]
    112. Arefian H, Heublein S, Scherag A, Brunkhorst FM, Younis MZ, Moerer O, et al. Hospital-related cost of sepsis: a systematic review. J Infect 2017 Feb;74(2):107-117. [CrossRef] [Medline]
    113. Hajj J, Blaine N, Salavaci J, Jacoby D. The "Centrality of sepsis": a review on incidence, mortality, and cost of care. Healthcare (Basel) 2018 Jul 30;6(3):90 [FREE Full text] [CrossRef] [Medline]

    ‎
    CCDS: computerized clinician decision support
    KPC: Kaiser Permanente early-onset sepsis risk calculator
    LILACS: Latin American and Caribbean Health Sciences Literature
    PQDT: ProQuest Dissertations and Theses Global
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews

    Edited by C Lovis; submitted 02.12.21; peer-reviewed by M Dewan, M Sehgal, J Woods; comments to author 20.01.22; revised version received 27.02.22; accepted 19.03.22; published 06.05.22

    Copyright

    ©Khalia Ackermann, Jannah Baker, Marino Festa, Brendan McMullan, Johanna Westbrook, Ling Li. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 06.05.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included.