This is an openaccess article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
Hemodynamic assessment of critically ill patients is a challenging endeavor, and advanced monitoring techniques are often required to guide treatment choices. Given the technical complexity and occasional unavailability of these techniques, estimation of cardiac function based on clinical examination is valuable for critical care physicians to diagnose circulatory shock. Yet, the lack of knowledge on how to best conduct and teach the clinical examination to estimate cardiac function has reduced its accuracy to almost that of “flipping a coin.”
The aim of this study was to investigate the decisionmaking process underlying estimates of cardiac function of patients acutely admitted to the intensive care unit (ICU) based on current standardized clinical examination using Bayesian methods.
Patient data were collected as part of the Simple Intensive Care StudiesI (SICSI) prospective cohort study. All adult patients consecutively admitted to the ICU with an expected stay longer than 24 hours were included, for whom clinical examination was conducted and cardiac function was estimated. Using these data, first, the probabilistic dependencies between the examiners’ estimates and the set of clinically measured variables upon which these rely were analyzed using a Bayesian network. Second, the accuracy of cardiac function estimates was assessed by comparison to the cardiac index values measured by critical care ultrasonography.
A total of 1075 patients were included, of which 783 patients had validated cardiac index measurements. A Bayesian network analysis identified two clinical variables upon which cardiac function estimate is conditionally dependent, namely, noradrenaline administration and presence of delayed capillary refill time or mottling. When the patient received noradrenaline, the probability of cardiac function being estimated as reasonable or good P(E_{R,G}) was lower, irrespective of whether the patient was mechanically ventilated (P[E_{R,G}ventilation, noradrenaline]=0.63, P[E_{R,G}ventilation, no noradrenaline]=0.91, P[E_{R,G}no ventilation, noradrenaline]=0.67, P[E_{R,G}no ventilation, no noradrenaline]=0.93). The same trend was found for capillary refill time or mottling. Sensitivity of estimating a low cardiac index was 26% and 39% and specificity was 83% and 74% for students and physicians, respectively. Positive and negative likelihood ratios were 1.53 (95% CI 1.191.97) and 0.87 (95% CI 0.800.95), respectively, overall.
The conditional dependencies between clinical variables and the cardiac function estimates resulted in a network consistent with known physiological relations. Conditional probability queries allow for multiple clinical scenarios to be recreated, which provide insight into the possible thought process underlying the examiners’ cardiac function estimates. This information can help develop interactive digital training tools for students and physicians and contribute toward the goal of further improving the diagnostic accuracy of clinical examination in ICU patients.
ClinicalTrials.gov NCT02912624; https://clinicaltrials.gov/ct2/show/NCT02912624
In hemodynamically unstable patients admitted to the intensive care unit (ICU) for circulatory shock, the diagnosis and treatment decisions initially rely on accurate assessment of clinical examination [
Hemodynamic assessment of critically ill patients is challenging; depending on the type of shock, patients present with highly variable states of circulating blood volume, cardiac contractility, sympathetic nervous activity, vascular tone, and microcirculatory dysfunction. In addition, assessment is even more difficult if comorbidities are present [
The first step in developing improved clinical examination structures for hemodynamic estimates is to study the current clinical practice. To understand how students and physicians diagnosed low cardiac index, Bayesian networks can be used to gain insight into the thought process behind the educated guess on hemodynamic status.
Bayesian networks have been frequently used to model domain knowledge in the context of decision support in other fields of medicine, given their ability to be interpreted as causal networks when no confounders are present [
The aim of this study was to use Bayesian networks to investigate the decisionmaking process underlying estimates of cardiac function of patients acutely admitted to the ICU, based on current standardized clinical examination using Bayesian methods. Additionally, we aimed to determine the diagnostic accuracy of the current standardized clinical examination for estimating cardiac function in patients acutely admitted to the ICU.
This study was a predefined substudy of the prospective observational cohort Simple Intensive Care StudiesI (SICSI) (ClinicalTrial.gov trial registration: NCT02912624) [
The primary aim was to determine the conditional probabilities relating the variables measured during clinical examination to the cardiac function estimate made by the examiners.
The secondary aim of this study was to assess the diagnostic accuracy of cardiac function estimates made by the examiners and compare them to the cardiac index measured by critical care ultrasonography (CCUS).
Bayesian networks are probabilistic models that represent the conditional (in)dependence relations between a set of variables in the form of a directed acyclic graph. In the graph, each variable is represented as a node and the directed edges (arcs) connecting the nodes represent the conditional dependency relations among the variables. Given the conditional (in)dependencies implied by the directed acyclic graph, the joint probability distribution of all variables can be factorized into a product of simpler local probability distributions.
From the initial set of variables registered during clinical examination, 14 clinical variables available from bedside monitors and patient record files, perfusors, physical examination, and the cardiac function estimate were included for modeling (
The network structure was learned using the MaxMin HillClimbing algorithm with the BayesianDirichlet equivalent scoring metric, as implemented in the R package “bnlearn” [
A set of restrictions can be applied to enforce certain connections between arcs in the network, so that prior knowledge is implemented
After the restrictions are defined, to obtain a confidence measure for the presence and directionality of the individual network edges, the bootstrap technique was applied. R=2000 bootstrap samples were generated from the original data, and the MaxMin HillClimbing algorithm was used to search for the best network for each bootstrap data set. This gives R=2000 best networks, and the confidence on the presence of an edge ranges from 0 (learned from 0 bootstrap samples) to 1 (learned from all bootstrap samples) [
To determine the distributions of the variables and calculate the associated probabilities of the network, the adjacency matrix of the average bootstrapped directed acyclic graph was reproduced using the Bayesian network function, and belief propagation was carried out using the
Patients underwent a protocolized and standardized clinical examination and subsequent CCUS, as described in the SICSI protocol [
Due to the observational nature of the study, a formal sample size calculation was not possible. Statistical analyses were performed in STATA 15.0 (StataCorp, College Station, Texas) and R version 3.5.1 (R Core Team, Vienna, Austria). Data are presented as mean with SD when normally distributed, or as median with interquartile range in case of skewed data. Dichotomous and categorical data are presented in proportions. Sensitivity and specificity for both the network’s and the examiners’ estimated guess were calculated by crosstabulation of the respective predictions and the validated cardiac index measurements. Additionally, positive predictive values (PPV) and negative predictive values (NPV) and positive likelihood ratios (LR+) and negative likelihood ratios (LR) were calculated with 95% CIs for the examiners’ estimates. For these, the overall accuracy was further expressed as a proportion of correctly classified cardiac index measurements (true negative and true positive measures) among all measures.
A total of 1075 patients fulfilled our inclusion criteria, of which 1073 patients had available cardiac function estimates and were therefore included in the Bayesian network analysis. Of the included patients, 783 (73%) had validated cardiac index measurements and were included in the diagnostic accuracy tests. Further, 569 patients (73%) were included by students and 214 patients (27%) were included by physicians.
Characteristics of included patients according to availability of cardiac index measurements are shown in
The structure learned for the network identified two clinical variables, namely, noradrenaline administration and the presence of delayed capillary refill time or mottling (dCRTM), upon which the estimates of cardiac function are directly conditionally dependent (
As denoted in
The probability queries conducted with the conditional probabilities for
Finally, an area under the receiver operating characteristic curve of 0.58 was obtained for the 10fold crossvalidated predictions of cardiac function made by the consensus network, with a specificity of 36% and a sensitivity of 79% [
Patient characteristics.
Variable  No cardiac index measurement (n=292)  Cardiac index measurement (n=783)  Total (N=1075)  
Age (years), mean (SD)  62 (14)  62 (15)  62 (15)  .75  
Male gender, n (%)  188 (64)  486 (62)  674 (63)  .49  
Body mass index (kg/m^{2}), mean (SD)  27.5 (5.4)  26.7 (5.6)  26.9 (5.5)  .04  
Arterial pressure (mm Hg), mean (SD)  78 (14)  79 (14)  78 (14)  .30  
Heart rate (bpm^{a}), mean (SD)  87 (22)  88 (21)  88 (21)  .35  
Irregular heart rhythm, n (%)  28 (10)  88 (11)  116 (11)  .44  
Central venous pressure (mm Hg), median (IQR)  9 (5, 12)  9 (5, 13)  9 (5, 13)  .74  
Patients administered noradrenaline, n (%)  142 (49)  386 (49)  528 (49)  .85  
Urine output (mL/kg/h), median (IQR)  0.6 (0.3, 1.2)  0.7 (0.4, 1.2)  0.6 (0.4, 1.2)  .22  
Respiratory rate (bpm), mean (SD)  18 (5)  18 (6)  18 (6)  .50  
Mechanical ventilation, n (%)  179 (61)  452 (58)  631 (59)  .29  
Positive endexpiratory pressure (cm H_{2}O), median (IQR)  7 (5, 8)  7 (5, 8)  7 (5, 8)  .41  
Central temperature (°C), mean (SD)  37.0 (0.9)  36.9 (0.9)  36.9 (0.9)  .84  
Difference between central temperature and temperature of the dorsum of the foot (°C), mean (SD)  7.7 (3.2)  7.8 (3.2)  7.8 (3.2)  .66  
Subjective “cold” temperature, n (%)  109 (37.6)  289 (37.1)  398 (37.2)  .88  







Knee (s), median (IQR)  3.0 (2.0, 4.5)  3.0 (2.0, 4.5)  3.0 (2.0, 4.5)  .48 

Sternum (s), median (IQR)  2.8 (2.0, 3.0)  3.0 (2.0, 3.0)  3.0 (2.0, 3.0)  .84 

Finger (s), median (IQR)  3.0 (2.0, 4.0)  2.5 (2.0, 4.0)  2.5 (2.0, 4.0)  .37 

.64  

None  157 (58.8)  397 (56.8)  554 (57.3) 


Mild  24 (9.0)  79 (11.3)  103 (10.7) 


Moderate  75 (28.1)  201 (28.8)  276 (28.6) 


Severe  11 (4.1)  22 (3.1)  33 (3.4) 

Hemoglobin (mmol/L), mean (SD)  6.8 (1.5)  6.8 (1.4)  6.8 (1.4)  .90  
Lactate (mmol/L)  1.4 (0.9, 2.4)  1.4 (0.9, 2.2)  1.4 (0.9, 2.2)  .79  
ICU^{b} length of stay (days)  3.5 (1.9, 6.9)  3.1 (1.9, 6.5)  3.2 (1.9, 6.6)  .29  
SAPS^{c} II (points)  47 (37, 58)  44 (34, 56)  45 (35, 57)  .037  
APACHE^{d} IV score (points)  77 (56, 92)  73 (55, 91)  74 (56, 92)  .14  
90day mortality, n (%)  81 (27.7)  217 (27.7)  298 (27.7)  .99  

.004  

Poor  8 (2.8)  18 (2.3)  26 (2.4) 


Moderate  46 (15.9)  165 (21.1)  211 (19.7) 


Reasonable  164 (56.6)  349 (44.6)  513 (47.8) 


Good  72 (24.8)  251 (32.1)  323 (30.1) 

^{a}bpm: beats per minute.
^{b}ICU: intensive care unit.
^{c}SAPS: Simplified Acute Physiology Score.
^{d}APACHE: Acute Physiology and Chronic Health Evaluation.
Strength and direction coefficients of the consensus directed acyclic graph.
From  To  Strength  Direction 
Age  Irregular rhythm  0.983  1.00 
Mechanically ventilated  High respiratory rate  0.994  0.504 
Mechanically ventilated  dCRTM^{a}  0.875  0.884 
Irregular rhythm  Tachycardia  0.848  0.954 
Tachycardia  High respiratory rate  0.999  0.931 
Tachycardia  Low SBP^{b}  0.821  0.883 
Tachycardia  Elevated lactate  0.832  0.821 
Low SBP  Low MAP^{c}  1  1 
Low DBP^{d}  Low MAP  1  1 
Elevated lactate level  Oliguria  0.728  0.803 
Elevated lactate level  Noradrenaline administration  1  1 
Noradrenaline administration  Mechanically ventilated  1  0.957 
Noradrenaline administration  Estimate  0.999  1 
dCRTM  Estimate  0.876  1 
^{a}dCRTM: delayed capillary refill time or mottling.
^{b}SBP: systolic blood pressure.
^{c}MAP: mean arterial pressure.
^{d}DBP: diastolic blood pressure.
Consensus directed acyclic graph. Red lines represent direct conditional dependencies to estimate. Black lines represent direct conditional dependencies to other variables. Width of the line represents strength coefficient. The dotted line represents the weakest strength coefficient. DBP: diastolic blood pressure; SBP: systolic blood pressure; MAP: mean arterial pressure; CRT: capillary refill time.
Tree diagram showing the conditional probabilities queries for estimate associated with multiple scenarios during clinical examination. At each step, only the variables above the split are known and as more information becomes available, the conditional probabilities change. P=Poor; M=Moderate; R=Reasonable; G=Good; CRT: capillary refill time.
Diagnostic accuracy tests for estimating of a low cardiac index showed a sensitivity of 26% and 39%, a specificity of 83% and 74%, PPV of 45% and 48%, NPV of 67% and 66%, LR+ of 1.52 and 1.52, and LR of 0.89 and 0.82 for students and physicians, respectively. The overall accuracy of cardiac index estimates was 63% and 61% for students and physicians, respectively. For all patients combined, sensitivity was 30%, specificity was 80%, PPV was 46%, NPV was 67%, LR+ was 1.53, LR was 0.87, and the overall accuracy of diagnostic tests was 62% (
Accuracy, sensitivity, specificity, predictive values, and likelihood ratios for students’ and physicians’ estimates.
Variable  Students (n=569)  Physicians (n=214)  Overall (N=783) 
Sensitivity, % (95% CI)  26 (2033)  39 (2850)  30 (2536) 
Specificity, % (95% CI)  83 (7886)  74 (6682)  80 (7784) 
Positive predictive value, % (95% CI)  45 (3853)  48 (3958)  46 (4053) 
Negative predictive value, % (95% CI)  67 (6569)  66 (6171)  67 (6569) 
Positive likelihood ratio, 95% CI  1.52 (1.102.09)  1.52 (1.022.25)  1.53 (1.191.97) 
Negative likelihood ratio, 95% CI  0.89 (0.810.98)  0.82 (0.671.00)  0.87 (0.800.95) 
Overall accuracy, % (95% CI)  63 (5967)  61 (5467)  62 (5966) 
Clinical examination is used daily by physicians as an easy, cheap, and noninvasive way of gathering information to guide interventions and further diagnostic testing. Clinical signs such as oliguria; altered consciousness; and cold, clammy skin are known possible indicators of organ hypoperfusion and are used to diagnose shock in critically ill patients [
Validation of the network structure was a crucial yet challenging step toward our goal of trying to obtain a plausible representation of the examiners’ knowledge network and thought process at bedside. We believe to have tackled this challenge in the best way possible by validating it in three different ways: using the bootstrapping process to generate a consensus network; conducting expert validation of the plausibility of the arcs; and using the network as a predictor, as previously suggested [
As any exploratory study, however, we faced several limitations. The first was practical, as not all included patients had cardiac index measurements, since CCUS is not applicable for every ICU patient and views obtained by CCUS can be obstructed due to lines, wounds, or excess adiposity [
Previous studies on the diagnostic accuracy of clinical examination have found the performance of experienced physicians and students to be comparable [
This study confirms that the accuracy of cardiac function estimates remains low for both students and physicians, and it identifies noradrenaline administration and delayed CRT or mottling as seemingly the major factors influencing these estimates. Although it will remain challenging to try to replicate the thought process of the examiner, not only methodologically, but also because different individuals have different levels of knowledge and different examination routines, Bayesian networks seem like a promising tool to help break down and better understand the educated guessing process. The insight gained in studies such as this one, can help teach students think about how they think and, on a clinical level, provide muchneeded guidance for prioritization of variables during clinical examination. In fact, our team is currently compiling the knowledge acquired in the SICSI substudies to build an interactive game for medical students, residents, and specialists. This electronic learning tool will ask the player to estimate cardiac function using the same scale and data from variables such as bedside monitor hemodynamic variables, ventilator and pump settings, and urine output.
Variables included in the Bayesian network and the respective Cramér V similarity measure.
Acute Physiology and Chronic Health Evaluation
critical care ultrasonography
diastolic blood pressure
delayed capillary refill time or mottling
intensive care unit
negative likelihood ratios
positive likelihood ratios
mean arterial pressure
negative predictive values
positive predictive values
Simplified Acute Physiology Score
systolic blood pressure
Simple Intensive Care StudiesI
TK and JCF performed the data analysis and drafted the manuscript, and all other authors reviewed and provided feedback with each draft. All authors approved of the final manuscript.
None declared.