Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, December 24 through Wednesday, December 26 inclusive. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 21.11.16 in Vol 4, No 4 (2016): Oct-Dec

This paper is in the following e-collection/theme issue:

    Review

    A Review of Visual Representations of Physiologic Data

    1Center for Biomedical Informatics, Department of Pediatrics, University of Tennessee Health Science Center, Memphis, TN, United States

    2University of Ontario Institute of Technology, Oshawa, ON, Canada

    *all authors contributed equally

    Corresponding Author:

    Rishikesan Kamaleswaran, PhD

    Center for Biomedical Informatics

    Department of Pediatrics

    University of Tennessee Health Science Center

    50 N Dunlap St

    Room 496R

    Memphis, TN, 38111

    United States

    Phone: 1 4163002871

    Fax:1 4163002871

    Email:


    ABSTRACT

    Background: Physiological data is derived from electrodes attached directly to patients. Modern patient monitors are capable of sampling data at frequencies in the range of several million bits every hour. Hence the potential for cognitive threat arising from information overload and diminished situational awareness becomes increasingly relevant. A systematic review was conducted to identify novel visual representations of physiologic data that address cognitive, analytic, and monitoring requirements in critical care environments.

    Objective: The aims of this review were to identify knowledge pertaining to (1) support for conveying event information via tri-event parameters; (2) identification of the use of visual variables across all physiologic representations; (3) aspects of effective design principles and methodology; (4) frequency of expert consultations; (5) support for user engagement and identifying heuristics for future developments.

    Methods: A review was completed of papers published as of August 2016. Titles were first collected and analyzed using an inclusion criteria. Abstracts resulting from the first pass were then analyzed to produce a final set of full papers. Each full paper was passed through a data extraction form eliciting data for comparative analysis.

    Results: In total, 39 full papers met all criteria and were selected for full review. Results revealed great diversity in visual representations of physiological data. Visual representations spanned 4 groups including tabular, graph-based, object-based, and metaphoric displays. The metaphoric display was the most popular (n=19), followed by waveform displays typical to the single-sensor-single-indicator paradigm (n=18), and finally object displays (n=9) that utilized spatiotemporal elements to highlight changes in physiologic status. Results obtained from experiments and evaluations suggest specifics related to the optimal use of visual variables, such as color, shape, size, and texture have not been fully understood. Relationships between outcomes and the users’ involvement in the design process also require further investigation. A very limited subset of visual representations (n=3) support interactive functionality for basic analysis, while only one display allows the user to perform analysis including more than one patient.

    Conclusions: Results from the review suggest positive outcomes when visual representations extend beyond the typical waveform displays; however, there remain numerous challenges. In particular, the challenge of extensibility limits their applicability to certain subsets or locations, challenge of interoperability limits its expressiveness beyond physiologic data, and finally the challenge of instantaneity limits the extent of interactive user engagement.

    JMIR Med Inform 2016;4(4):e31

    doi:10.2196/medinform.5186

    KEYWORDS



    Introduction

    Two less formal reviews and one systematic review were published in the last decade, reporting positive impact of visual representations in the critical care setting. Sanderson et al provide a forward-looking analysis of representation of physiological data [1] in anesthesiology [2]. Drews and Westenskow review several graphical displays that facilitate rapid translation of physiological event knowledge for anesthesiologists [3]. An initial systematic review was published in 2007 by Görges and Staggers that reviews general physiologic data displays; however, with emphasis on surgical and anesthesiology specialities [4]. While those reviews provide important knowledge about the state of the art in physiologic data, they present only a partial aggregation of results, and limited knowledge that could be used to enhance the design of physiological visualizations. Furthermore, key elements such as the nature of visual variables utilized in the encoding, support for interactive exploration, and common design considerations were not discussed. All reviews focused on displays that support short-term patient monitoring tasks. Visualizations supporting longitudinal monitoring and interactive visual analysis of physiological data were not sufficiently addressed.

    The aim of this specific review is of 3 parts: (1) identify the design decisions used in the development of novel physiologic visual representations; (2) review the utilization of temporal parameters namely: trajectory, frequency, and duration in visual designs using physiologic parameters; and (3) review the nature of interactive functions afforded for rich exploration tasks. With that in mind, this paper presents an analysis of a broad spectrum of physiological visual representations used at the bed-side, in the surgical ward, and for clinical research.


    Methods

    The review was conducted in 2 phases: the first phase identified the key terms to be included in the search strategy, while the second phase broadened the search strategy and used structured analysis method. In the first phase we used Google Scholar, and 25 papers were found to be relevant. The search was limited to the last 15 years and used a combination of keywords that were known to the author, such as “(physiologic* or clinical or hemodynamic) and (visual* or graphic*) and (interface or display),” where asterisk was used to search for terms that started with the specific key words. In the second phase, we used 6 prominent sources including: IEEE Explore, ACM Digital Library, MEDLINE, EMBASE, ISI Web of Science, and Google Scholar. A broad search strategy was used to capture as many representations as were possible. Index terms were used to filter articles and included “data display*,” “diagnosis, computer-assisted,” “monitoring, physiologic/methods*,” “*computer graphics,” “user-computer interface,” “data display,” “interview* or discussion* or questionnaire* or “focus group*” or qualitative or ethnograph* or fieldwork or “field work” or “key informant,” “task performance and analysis,” “graphic* adj2 display*”.

    For screening articles in the second phase, we used rigorous inclusion criteria (Textbox 1) that initially classified visualizations across 4 groups. The groups were (1) tabular displays, (2) waveform displays, (3) object displays, and (4) ecological displays. Inclusion criteria relating to outcome measures are divided into 3 sets of measures (Textbox 2). They include temporal and duration, human and qualitative factors, and quantitative measures. The physiological parameters tested are listed in Textbox 3. We placed a restriction in years from January 1, 1983 to August 1, 2016 and limited our results to human studies in critical care, anesthesiology, and surgery. We included snowballing of references and manual searches on Google Scholar and PubMed. This resulted in a total of 1262 titles generated for review. Relevant titles were identified using rigorous inclusion criteria (Textbox 1). In total, 171 titles were then designated for abstract review. Following that, 78 abstracts were selected for full review, and 39 papers were selected for inclusion in the analysis. Bias was mitigated by having 2 researchers screen independently, and differences were resolved through discussions until consensus was reached.


    Textbox 1. Inclusion criteria.
    View this box

    Textbox 2. Reported metrics.
    View this box

    Textbox 3. Physiological parameters tested.
    View this box

    Following the creation of the inclusion criteria, an online data extraction form was developed using Google forms and used to evaluate all papers. The data extraction form consisted of 6 sections that were identified as potential areas of interest. For each full paper reviewed, 74 questions were screened. Questions to be included in the data extraction forms were selected from themes identified in the pilot study. In particular, questions were generated to elicit detail about the study, design, and results from any human experiment or evaluations. Where appropriate the questions were marked as either not reported if data was missing, or not applicable if the question was a follow-up of a prior conditional question. The data was then thematically synthesized based on aggregations of results by descriptive codes. The thematic synthesis is presented using a series of matrices presented in the next section.


    Results

    Phase 1 and 2

    All papers included in the analysis were passed through the data extraction form and resulted in an initial comprehensive matrix of results. Of 74 questions that were initially probed, results that yielded over 75% not reported, or not applicable across all papers analyzed were removed from our analysis. Then 39 variables were selected for inclusion in the initial matrix. Phase 1 results are summarized in the comprehensive matrix of design properties (Table 1 and Figure 1) and phase 2 results are summarized in the Comprehensive Matrix of Study Results (Table 2 and Table 3). The comprehensive matrix of design properties presents 10 variables which are divided between 2 tables. Variables appearing in (Table 1) are “Target Users”, “Year”, “Clinical Context”, “Number of Variables”, and “Display Type”.

    “Target Users” relates to the clinical specialty, and “Year” is the approximate date the prototype was tested. Due to the difference between the dates of publication and evaluation, this value was approximated based on the date of submission of the article. “Clinical Context” conveys the copresence of contextual clinical information, and “Number of Variables” refers to the total number of physiological or clinical variables that were visible in a single screen. “Display Type” lists the types of graphics utilized by the paper belonging to one of: tabular, object, or metaphoric displays. “Color(s) Used” identifies the hue where available. “Pre-attentive Processing” lists particular visual variables that were used in the visual representation such as: shape, size, and dimension. “Gestalts” refers to the designer’s use of grouping laws identified by Gestalt’s laws of perception: the use of proximity, similarity, closure, symmetry, and continuity as a means of discerning visual objects presented in the display [5]. Finally “Interactive Controls” refers to the ability for the display to support direct manipulation of one or more properties and “Iterative Design” identifies displays that were built using user-centered design approaches that include users into key decision making processes prior to the development of the display.

    A second matrix, titled the Comprehensive Matrix of Study Results presents additional 11 variables that were identified in papers which presented study results. Table 2 lists “Setting,” “Study Type,” “Results Reported,” “Realism,” “Cognitive Workload,” “Historic Trends,” “Visual Encoding for Temporal Trajectory,” “Visual Encoding for Duration,” “Visual Encoding for Frequency”. Table 3 lists “Counter-balanced for Display or Scenario,” “Were Scenarios Clinically Relevant,” and “Function Supporting Case-controlled Analysis.” “Setting” describes the location where the study was physically conducted; for instance, the lab, clinic, or public areas. “Study Type” identifies research method used to validate the display. “Results Reported” summarizes key findings from the study, and “Realism” presents the latency of the display as well as the ability of the display to mimic real-world dynamism. “Cognitive Workload” reports on findings indicating reduced or increased workload and “Historic Trends” identifies displays that present historical trends that are greater than 5 minutes. The variables beginning with visual encodings for temporal trajectory, duration, and frequency identify particular techniques used by the displays to represent trends, duration of events, and frequency of events. The counter-balanced variable identifies methodologies that used strategies to minimize learning effects during the experiment. Finally, the clinical scenario variable lists the displays that were evaluated using real-world clinical scenarios. These tables, along with descriptions of the results are presented in the next section.

    Comprehensive Matrix of Design Properties

    The goal of the comprehensive matrix of design properties is to present design decisions that were followed to develop prototypes across all 39 papers analyzed. Visual representations were found across mainly anesthesiology (n=17), critical care (n=20), and in some multi-discipline (n=2) environments. Only one display was developed as a tool for intensive nurses [42]. Multi-discipline environments consist of 2 or more specialities, such as integrated in-patient and out-patient systems. Visual displays started to become actively contributed from the early 1990s, then increasing every 10 years, 1984 (n=1), 1985-1994 (n=8), 1995-2004 (n=13), and 2005-2016 (n=17). Integrated clinical data was also found across some displays (n=16), while a greater number of displays were devoted to the display of physiological waveforms (n=24). Number of variables presented in a single screen was wide-ranging; most displays contained greater than 20 variables per screen (n=15), followed by 11-20 variables (n=12), while 7 displays contained between 0-4 variables.

    Reviewed visual representations included a mix of display formats, such as tabular (TB), object-based (OB), waveform (WF), and metaphoric (MT). Standalone MT representations were most commonly seen (n=12), followed by standalone WF (n=6). With respect to combinatory displays, TB appeared with WF (n=6) most frequently, WF with MT (n=6), and followed by WF with an OB (n=4). When identifying the display type most frequently paired in a combinatory display, WF (n=12) appeared most often, followed by TB (n=7) and OB (n=6) displays. Overall, across all identified papers including those where multiple representatives were presented, metaphoric displays were the most popular (n=22), followed by waveform displays (n=20), and object displays (n=10).

    Visual representations utilized at least 2 of the primary colors, red, blue or green (n=21), while yellow (n=11) and turquoise (n=4) were also popular options. Three papers utilized discrete color encoding, 2 papers [25,43] mentioned the source of their color coding. A number of papers did not specify the type of color that was used (n=10). Pre-attentive processing of items were commonly exploited through manipulating visual variables such as color (n=24) and size (n=12), followed by dimension (n=7), and shape (n=5).

    Visual representations also exploited some aspect of Gestalt’s law of groupings, such as continuity (n=18) with waveform displays, closure (n=17) when identifying boundaries, symmetry (n=14) with visual metaphors and object-based displays, and proximity (n=7) to aid in higher level detection of abnormal events. The most popular interaction method that was supported was selection (n=13). Selection allows the user to select visual objects directly to reveal greater details. This was followed by interactive filtering (n=7) to select partial ranges such as short durations of time. Finally, in many cases designs were proposed without following user-centered design approaches (n=28). In total, 10 papers reported using user-centered design processes, while 4 papers described a structured approach used in developing the proposed visual design [5,31,43,44].

    Figure 1. Comprehensive Matrix of Design Properties.
    View this figure
    Table 1. Comprehensive matrix of design properties.
    View this table
    Table 2. Comprehensive matrix of study results.
    View this table
    Table 3. Comprehensive matrix of study results.
    View this table

    Comprehensive Matrix of Study Design

    The Comprehensive Matrix of Study Design (Figure 2) presents the results that were reported by authors for any evaluation or experiment. While the search strategy yielded 39 full papers that were identified for analysis, only 29 of these papers contained primary study results from a case study, evaluation, or human experiment, and employed 1 of naïve, novice, or expert participants in the evaluation method. Naïve participants were generally undergraduate students with little or no prior clinical knowledge. Novice participants ranged from undergraduate computer science or nursing students to newly graduated clinical staff. Expert participants had at least 10 years of experience.

    The number of participants exposed to test conditions highly varied; however, the majority of studies employed at least 15 participants. Six studies used a sample size greater than 20 to test for detection, diagnostic, and treatment accuracy, with the minimum and maximum being 4 and 32 participants, respectively. Most displays integrated these systems in a single display using live or static representations (n=15), while displays that were presented as case studies (in situ) were connected to central monitoring systems. Some displays supported views of clinical information that integrated data from other clinical and laboratory systems (n=15) [45]. Most prototypes that were evaluated used more than one data stream, with the exception of the studies that contained low-frequency updates (n=9). Most evaluations or experiments utilized more than one condition to test each display; however, a few did not have any scenarios or patient conditions (n=9). A large number of studies also did not utilize data from more than one patient-source (n=26).

    Most of the studies were conducted in laboratory environments (n=24), followed by evaluations or experiments in the intensive care unit (n=12). Some studies were evaluated over multiple specialities (n=2). A majority of studies used some form of experimentation to validate their designs (n=21), although the specific method of experimentation was not always explicitly mentioned. Evaluations involved clinicians and mixed qualitative and quantitative methods were used to report results (n=8). Applications were primarily qualitative in nature, often depicting results through anecdotes (n=7). The remaining studies were design papers that investigated novel visual representations without involving prototypes. Of the papers that reported results (n=31), most reported positive findings (n=27), but in some cases negative results were also reported (n=4). A between-group experimental study yielded site-dependent results that were skewed towards the site that produced the visual representation. For evaluations or experiments the source of data to support realism was spread across live simulations (n=19), live patient-origin data (n=9), or static patient-generated data (n=11). Most studies did not test for cognitive overload using ad hoc methods or traditional workload score metrics such as the NASA Task Load Index (NASA-TLX) (n=30). Where cognitive workload was reported (n=7), most were reported to have reduced cognitive overload (n=5), while others reported no change or mixed results (n=3).

    Long-term historical values, specifically ranges exceeding 5 minutes of monitoring were not included in majority of the displays (n=28). Tri-event parameters, namely, trajectory, frequency, and duration were seldom supported by visual representations, where these parameters were identified, trajectory was most frequently found (n=27). Temporal trajectory was encoded using curves (n=25) such as in a line graph, or glyphs (n=9). In terms of duration, the second tri-event parameter was seen across 10 displays, of which, glyphs (n=6), text (n=4) or curves (n=2) representations were utilized. Frequency, the last tri-event parameter was also seen in some visual representations encoded by glyph (n=5) or text (n=4) where supported. Where displays were validated through experimentation, both the display and scenarios were more often counterbalanced (n=12), while some experiments counterbalanced only the scenario (n=6) and others only the display (n=4). Scenarios were utilized across many studies utilizing experimentation or evaluation methodologies (n=22) and most were clinically relevant problems (n=21). Finally, only one of the evaluated visual representations supported the ability to perform analysis across multiple patients.


    Discussion

    Principal Findings

    A total of 19 novel visual representations were identified from analysis of the literature. Novel displays were seen across 4 main groups, including tabular, waveform (graph-based), object, and metaphors. The latter 2 are aggregated together as ecologic displays.

    Tabular Displays

    The early- 1990’s saw growing interest in converting large volumes of paper patient charts to “virtual” records [24,46-49]. Initial representations adopted by these virtual patient records were largely tabular and text-dominant, and sometimes contributed negatively to information overload [48]. Figure 2 [50] illustrates an example of a traditional virtual patient chart that mimics a traditional paper flow chart. This review identified 14 tabular representations published from 1952 to 1997. Those systems provide a direct manipulation using the traditional desktop-oriented, Windows-Icon-Mouse-Pointer (WIMP) interaction paradigm. Additional levels of interactions, such as multiple mouse clicks, are required to access unique views of patient data. Large number of these displays are often duplicated to a physical copy, in part due to the simplicity and ease of reading paper charts [51].

    Figure 2. A tabular display that mimics traditional clinical flow-sheets.
    View this figure
    Figure 3. A modern dashboard utilizing waveform displays.
    View this figure

    Waveform Displays

    The review identified 9 out of 39 studies that used some form of live physiologic streams from real patients to display largely identical waveform representations. It was also noted that much of these waveform displays were integrated with other tabular and text representations. Five papers that presented waveform displays also supported interactive capabilities, including the ability to select regions of interest, filter based on patients, and generate screen captures [6,8,25,43,52]. Stylianides and colleagues (2011) present an engine for producing waveform graphics [23]; however, their system serves the purpose of animating historic physiologic data streams. CareCruiser [25], supports the interactive exploration of treatment plans using physiologic data. However, that system was not evaluated using more than one clinical user. PhysioEx [43] was evaluated using an expert evaluation methodology employing 5 domain experts, and was shown to further enhance that interactive analytic workflow by providing coordinated analysis of temporal data streams; however, using waveform displays only to guide the user with additional context.

    Despite their ability to communicate acute time-sensitive events [20], waveform representations have numerous limitations [4,31,53]. One prime disadvantage of waveform displays is the potential to negatively impact cognitive load, that is, they require humans to monitor and consume large numbers of data points as they are produced to derive trends and higher level knowledge [7,8,22]. These waveforms display can convey several features in one frame; therefore, easily disturb limited resources of the working memory capabilities [54]. The challenge of managing large volumes of data have been extensively studied in several domains, such as information overload [55], visual data mining [56], and addressing cognitive challenges related to interruptions, task performance, and decision making [55,57-59].

    Integrated methods of representing critical physiological information have been actively studied to reduce the internal mental processing requirement [20,22,32,60-62]. These integrated displays use a combination of text [33,34], graphic [3,4,63], and waveform [64,65] representations to summarize low-level information. Figure 3 [6] illustrates an example of an integrated display. Three such integrated displays were identified in the review [6,8,25]. These displays support clinicians to interactively select regions of interest while monitoring other forms of slow-changing clinical data. However, only one display allows the clinician to compare against a cohort [25]. Other studies, seeking alternatives to the waveform visual encoding, propose novel and ecological methods to improve knowledge discovery and minimize cognitive overload.

    Ecological Displays

    Classes of Visual Representations

    Ecologic displays attempt to integrate relationships existing across both workflows and semantics [66]. Among the primary goals of ecologic displays is to convey both the means-end relation, answering the particular means of arriving at that state and its ultimate consequence. From our review, we identified 2 large classes of visual representations that approach these objectives. Object-oriented displays, and metaphoric displays were seen to extend typical limitations found in text, tabular, and waveform displays by introducing novel information, such as spatial and temporal arrangements of closely related information.

    Object-Oriented Displays

    Displays that utilize and manipulate 2-dimensional graphical objects, limited to basic shapes and symmetries to produce emergent properties have been classed as object-oriented displays [2,13,35]. These displays follow demonstrated efficacy of graphical displays over traditional numeric displays observed in nuclear power station control stations [67]. Studies have shown a positive relationship with integrated displays and an overall improvement in diagnosis ability as well as a reduction in time to initiate treatment [68].

    Blike and colleagues (2000) [69] showed that subjects exposed to emergent features using novel graphics recognized a problem more rapidly, but their accuracy had not improved in comparison to the numeric display. Moreover, they showed that the shape of the graphic, illustrated in Figure 4 [13], improved detection of etiology compared to the numeric and control displays. While Blike and colleagues stated an improved reaction and fewer errors when using the object-oriented display, the display was found to be confusing and not ecological to naïve participants. Zhang and colleagues [36] reproduced the designs introduced by Blike et al, and found that anesthesiologists were able to detect simple deviations faster; however, no change was seen with detection times of more complex cardiovascular events. Other studies have reported similar conclusions [5,9,13,68,70], suggesting a link between detection and reactionary time to the format and features of the graphical display.

    In contrast, other studies that extrapolated heuristics from object-oriented displays report less convincing evidence; for instance, some report negative links when participants were presented object-oriented displays [21,37]. The etiological potential display (Figure 5) [21] attempts to extract specific features of object displays that improve detection and diagnosis. In that study, Effken and colleagues find no significance in the detection or diagnostic times, even when 3 abstract displays were tested. Two of these displays required that features of the full prototype either be reorganized or removed.

    Figure 4. Advanced graphical display for hemodynamic monitoring.
    View this figure
    Figure 5. The etiological potential display moves an object across 4 quadrants of heart strength and resistance. The object in the top right quadrant is distorted to show relative depressions in the atrium, ventricles, veins and arteries.
    View this figure
    Metaphoric Displays

    A total of 20 representations, for over half of all visual representations analyzed, belonged to the metaphoric display group. Most clinical metaphoric displays illustrate physiologic data in terms of organ-systems [20,36,44,71]. Five papers presented metaphors that involved dynamic objects that exhibited behaviors similar to organ systems [5,14,20,21,26].

    Several papers identified metaphor displays with positive outcomes. Cole and Stewart (1994) [14], introduced a visual representation (Figure 6) [50] which consists of 2 volume rectangles that compress or expand similar to the respiratory system. This design was further improved with additional data dimensions [26]. A Graphical Cardiovascular Display (Figure 7) [5] that uses a pipe-like metaphor of the cardiovascular system, was shown to enable faster detection of adverse events [5]. Wachter and colleagues (2003) applied similar approaches to develop a respiratory interface and found participants were able to identify abnormal states faster [9]. Gorges and colleagues, introduced a series of visual metaphors to communicate visual signs to bed-side clinicians [22]. These displays adopt a clock metaphor illustrated in Figure 8 [22] to convey salient features, such as temporal trends over the past 12-hours. Charabati and colleagues from the Montreal General Hospital’s department of anesthesiology introduced a gauge metaphor to highlight normal and abnormal ranges, and conducted an evaluation across 2 sites [7]. They found a combination of numeric and visual metaphors achieved the strongest advantage in detection, accuracy, and workload. Tappan and colleagues evaluated visual metaphors by appending visual objects to traditional medical monitors [19]. They reported significant improvements in detection of adverse events, with the visual metaphor having a 14.4 second advantage over traditional physiologic monitors. The visual metaphor was also found to reduce the number of missed events. However, similar to previous studies, these investigations were conducted in controlled environments.

    Not all visual metaphors, however, have seen similar success. Zhang and colleagues (2002) [36] introduced an integrated 3-dimensional balloon metaphor, building on the work of Blike and colleagues (2000) [69] with object displays. Zhang and colleagues found mixed results after evaluations, with only 63% of scenarios having shorter detection than scenarios, and situational awareness being improved in 1 of 4 scenarios. Moreover, van Amsterdam and colleagues (2013) from the University Medical Center Groningen, utilized customization features offered by vendor-based medical monitors to construct and evaluate a metaphoric display presented in Figure 9 [10]. They found, however, that visual metaphors did not improve detection or accuracy of anesthesiologists [10].

    Finally, while ecologic representations were evaluated for diagnostic accuracy and speed, the challenges surrounding cognitive errors remain only a secondary concern in research involving visual representations. Less than 8 out of 39 of papers analyzed were identified to have measured for cognitive workload [5,7,8,19,22,34,36,41]. Of the 8 papers that measured for cognitive workload, 4 papers used a quantitative measure such as the NASA-TLX score [5,8,22,34]. There are also limitations with the use of NASA-TLX, largely because it is a self-reported method of identifying perceived workload. A total of 3 of the 8 papers were evaluated with critical care clinicians, consequently, incorporating cognitive workload as a passive measure of potential cognitive error remains limited across visual representation research for clinical environments. Significantly, none of metaphoric displays supported analytic functions.

    Figure 6. Volume triangles represent multivariate clinical data using a lung-expansion metaphor.
    View this figure
    Figure 7. Graphical Cardiovascular Display, adapts a metaphor of a pipes with volume and pressure properties.
    View this figure
    Figure 8. Far-view visual metaphors for triaging vital signs.
    View this figure
    Figure 9. (a) Metaphorical anesthesia interface and (b) Metaphorical interface with trend information (tMAI).
    View this figure

    Tri-Event Parameters

    Physiological displays can be designed and developed using 3 consumption efficacy metrics derived for temporal and dynamic data streams [43]. These metrics are termed tri-event temporal parameters namely, trajectory, frequency, and duration of salient events.

    Among the tri-event parameters, trajectory was found to be the most popular, with 32 of 39 studies incorporating some form of trajectory information. However, longitudinal trajectory was found in only 9 studies, and was rare among displays that were found in anesthesiology but more common in critical care. Displays that incorporated an aspect of the tri-event temporal parameters exclusively adopted trajectory. Nine visual representations were found to have included the duration and frequency metrics. Most of the representations that included duration and frequency used glyphs (n=6) or text (n=5) to communicate episodic information. For instance, PhysioEx [43] uses the river metaphor [72] to illustrate frequency of adverse physiologic events that were analyzed by a real-time algorithm (bottom left, in Figure 10). Text also remains a popular method for communicating discrete events. Law and colleagues found text to be superior to waveform and numeric displays when communicating clinical episodes, even while clinicians reported a preference for graphical displays [73]. Where multiple views were presented, only one representation utilized interactive coordination between independent views [6].

    Figure 10. PhysioEx, a coordinated visual analytic tool for exploring clinical events across multiple temporal physiologic data streams.
    View this figure

    Conclusion

    Visual representations of physiological data have been attempted several times as witnessed by the sheer size of prior work discussed in this paper. Many have shown their potential to improve clinical care, and while largely positive results have been released, there are still concerns as to the efficacy of both in reproducibility as well as translatability to the unit. In particular, methods to identify the accuracy of actions post-treatment to the display remain as concern and open areas for further exploration.

    Few clinical visualization papers studied associations of the treatment condition to the accuracy or accrued insight by the user. It was also seen that most studies included detailed study of the time to diagnosis and its accuracy; however, many of these studies included highly controlled scenarios with highly visible graphical distortions. Additionally, few studies used real patient data to evaluate their prototypes. Hence, the frequency of events with clear and distinctive graphical patterns existing across real patient data remains untested. Detection was also another area where studies frequently report positive findings; however, in many cases these differences were marginal and found in narrow statistical ranges. It has yet to be proven whether these statistical significances are relevant in the clinical domain. Exact mechanisms inducing positive effect have yet to be studied within the prototypes studied [63,74].

    Visual representations show promise; however, they are plagued with user-preference and interaction challenges. Results spanning two decades continue to show positive influence of graphical representations when they are used in simulated studies [4]. However, many of these studies have not used standardized metrics to test distinct controlled variables, or provide evidence of precisely which features of the graphical displays afford greater comprehension to the consumer. Questions still remain as to its efficacy in clinical practice, where, the availability of all data required by the representations may be limited. There is also the limitation of graphical representation failing to maintain interpretable coherence, when provided incorrect data [2].

    Some studies have also demonstrated user involvement as an important factor which may have influenced results, in the design and development of the clinical system [45]. Future studies should focus on clinical validation as a means to identify real-life relevance. Clinical experiments are difficult in lieu of several considerations and their limitations. However, one study by Wachter et al [9], demonstrates that observational studies, although somewhat intrusive, may produce some significant qualitative results. These studies need to be expanded, and clinical trials must ultimately demonstrate their efficacy. Cognitive errors also require additional research effort, specifically by including evaluation methodologies such as the NASA-TLX score to allow end-users to self-report perceived workloads.

    Only 7 visualizations were identified to have had some element of interactive selection and filtering functions to support basic analysis tasks. While only one display was identified to support analysis across cohort populations. The general absence of analysis functionalities is an opportunity for enhancing physiologic visualizations. Physiologic data represents a unique subset, due to the dynamic and streaming nature of the data. Application of visual analysis techniques may support novel uses of physiologic visualizations, such as supporting human-driven hypothesis generation tasks.

    Finally, research in visual representations should include tri-event parameters as important design considerations to produce designs that communicate episodic information. PhysioEx was seen to incorporate all 3 parameters; however, it was limited to one view per patient [43]. These visual representations can then be used to better assess the influence of tri-event parameters on higher level workflows as well as in the progression of clinical conditions.

    Conflicts of Interest

    None declared.

    References

    1. McGregor C. Big data in neonatal intensive care. Computer 2013 Jun;46(6):54-59. [CrossRef]
    2. Sanderson PM, Watson MO, Russell WJ. Advanced patient monitoring displays: tools for continuous informing. Anesth Analg 2005 Jul;101(1):161. [CrossRef] [Medline]
    3. Drews FA, Westenskow DR. The right picture is worth a thousand numbers: data displays in anesthesia. Hum Factors 2006;48(1):59-71. [Medline]
    4. Görges M, Staggers N. Evaluations of physiological monitoring displays: a systematic review. J Clin Monit Comput 2008 Feb;22(1):45-66. [CrossRef] [Medline]
    5. Agutter J, Drews F, Syroid N, Westneskow D, Albert R, Strayer D. Evaluation of graphic cardiovascular display in a high-fidelity simulator. Anesth Analg 2003 Nov;97(5):1403-1413 [FREE Full text]
    6. Engelman D, Higgins T, Talati R, Grimsman J. Maintaining situational awareness in a cardiac intensive care unit. J Thorac Cardiovasc Surg 2014 Mar;147(3):1105-1106 [FREE Full text] [CrossRef]
    7. Charabati S, Bracco D, Mathieu P, Hemmerling T. Comparison of four different display designs of a novel anaesthetic monitoring system, the 'integrated monitor of anaesthesia (IMA)'. Br J Anaesth 2009 Nov;103(5):670-677 [FREE Full text] [CrossRef] [Medline]
    8. Anders S, Albert R, Miller A, Weinger M, Doig A, Behrens M, et al. Evaluation of an integrated graphical display to promote acute change detection in ICU patients. Int J Med Inform 2012 Dec;81(12):842-851 [FREE Full text] [CrossRef] [Medline]
    9. Wachter S, Markewitz B, Rose R, Westenskow D. Evaluation of a pulmonary graphical display in the medical intensive care unit: an observational study. J Biomed Inform 2005 Jun;38(3):239-243 [FREE Full text] [CrossRef] [Medline]
    10. van Amsterdam K, Cnossen F, Ballast A, Struys M. Visual metaphors on anaesthesia monitors do not improve anaesthetists' performance in the operating theatre. Br J Anaesth 2013 Feb 05;110(5):816-822 [FREE Full text] [CrossRef] [Medline]
    11. Kennedy R, Merry A. The effect of a graphical interpretation of a statistic trend indicator (Trigg's Tracking Variable) on the detection of simulated changes. Anaesth Intensive Care 2011 Sep;39(5):881-886 [FREE Full text] [Medline]
    12. Liu Y, Osvalder AL. Usability evaluation of a GUI prototype for a ventilator machine. J Clin Monit Comput 2004 Dec;18(5-6):365-372. [Medline]
    13. Blike GT, Surgenor SD, Whalen K. A graphical object display improves anesthesiologists' performance on a simulated diagnostic task. J Clin Monit Comput 1999 Jan;15(1):37-44. [Medline]
    14. Cole WG, Stewart JG. Human performance evaluation of a metaphor graphic display for respiratory data. Methods Inf Med 1994 Oct;33(4):390-396. [Medline]
    15. Deneault L, Lewis C, Debons A, Stein K, Dewolf A. An integrative display for patient monitoring. Presented at: IEEE International Conference on Systems, Man and Cybernetics; 1990 Nov; Los Angeles, CA.   URL: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=142161 [CrossRef]
    16. Jungk A, Thull B, Hoeft A, Rau G. Evaluation of two new ecological interface approaches for the anesthesia workplace. J Clin Monit Comput 2000;16(4):243-258. [Medline]
    17. Gurushanthaiah K, Weinger M, Englund C. Visual display format affects the ability of anesthesiologists to detect acute physiologic changes: a laboratory study employing a clinical display simulator. Anesthesiology 1995 Dec;83(6):1184-1193. [Medline]
    18. Ireland R, James HV, Howes M, Wilson A. Design of a summary screen for an ICU patient data management system. Med Biol Eng Comput 1997 Jul;35(4):397-401. [Medline]
    19. Tappan J, Daniels J, Slavin B, Lim J, Brant R, Ansermino J. Visual cueing with context relevant information for reducing change blindness. J Clin Monit Comput 2009 Aug;23(4):223-232. [CrossRef] [Medline]
    20. Michels P, Gravenstein D, Westenskow DR. An integrated graphic data display improves detection and identification of critical events during anesthesia. J Clin Monit 1997 Jul;13(4):249-259. [Medline]
    21. Effken J, Kim N, Shaw R. Making the constraints visible: testing the ecological approach to interface design. Ergonomics 1997 Jan;40(1):1-27. [CrossRef] [Medline]
    22. Görges M, Westenskow DR, Markewitz BA. Evaluation of an integrated intensive care unit monitoring display by critical care fellow physicians. J Clin Monit Comput 2012 Dec;26(6):429-436. [CrossRef] [Medline]
    23. Stylianides N, Dikaiakos M, Gjermundrød H, Panayi G, Kyprianou T. Intensive care window: real-time monitoring and analysis in the intensive care environment. IEEE Trans Inf Technol Biomed 2011 Jan;15(1):26-32. [CrossRef] [Medline]
    24. Litt H, Loonsk J. Digital patient records and the medical desktop: an integrated physician workstation for medical informatics training. Proc Annu Symp Comput Appl Med Care 1992:555-559 [FREE Full text] [Medline]
    25. Gschwandtner T, Aigner W, Kaiser K, Miksch S, Seyfang A. CareCruiser: exploring and visualizing plans, events, and effects interactively. 2011 Presented at: IEEE Pacific Visualization Symposium (PacificVis); 2011; Hong Kong p. 43-50. [CrossRef]
    26. Horn W, Popow C, Unterasinger L. Support for fast comprehension of ICU data: visualization using metaphor graphics. Methods Inf Med 2001;40(5):421-424. [Medline]
    27. Dayhoff R, Kirin G, Pollock S, Miller C, Todd S. Medical data capture and display: the importance of clinicians' workstation design. Proc Annu Symp Comput Appl Med Care 1994:541-545 [FREE Full text] [Medline]
    28. Norris P, Dawant B. Closing the loop in ICU decision support: physiologic event detection, alerts, and documentation. Proc AMIA Symp 2001:498-502 [FREE Full text] [Medline]
    29. Langner P. The value of high fidelity electrocardiography using the cathode ray oscillograph and an expanded time scale. Circulation 1952 Feb;5(2):249-256. [Medline]
    30. Burykin A, Peck T, Krejci V, Vannucci A, Kangrga I, Buchman T. Toward optimal display of physiologic status in critical care: I. Recreating bedside displays from archived physiologic data. J Crit Care 2011 Feb;26(1):105.e1-105.e9. [CrossRef] [Medline]
    31. Miller A, Scheinkestel C, Steele C. The effects of clinical information presentation on physicians' and nurses' decision-making in ICUs. Appl Ergon 2009 Jul;40(4):753-761. [CrossRef] [Medline]
    32. Kruger G, Tremper K. Advanced integrated real-time clinical displays. Anesthesiol Clin 2011 Sep;29(3):487-504. [CrossRef] [Medline]
    33. Law A, Freer Y, Hunter J, Logie R, McIntosh N, Quinn J. A comparison of graphical and textual presentations of time series data to support medical decision making in the neonatal intensive care unit. J Clin Monit Comput 2005 Jun;19(3):183-194. [CrossRef] [Medline]
    34. Ahmed A, Chandra S, Herasevich V, Gajic O, Pickering BW. The effect of two different electronic health record user interfaces on intensive care provider task load, errors of cognition, and performance. Crit Care Med 2011 Jul;39(7):1626-1634. [CrossRef] [Medline]
    35. Sainsbury D. An object-oriented approach to data display and storage: 3 years experience, 25,000 cases. Int J Clin Monit Comput 1993 Nov;10(4):225-233. [Medline]
    36. Zhang Y, Drews F, Westenskow D, Foresti S, Agutter J, Bermudez J, et al. Effects of integrated graphical displays on situation awareness in anaesthesiology. Cogn Technol Work 2002 Jun 1;4(2):82-90. [CrossRef]
    37. Kennedy RR, Merry AF, Warman GR, Webster CS. The influence of various graphical and numeric trend display formats on the detection of simulated changes. Anaesthesia 2009 Nov;64(11):1186-1191 [FREE Full text] [CrossRef] [Medline]
    38. Lowe A, Jones R, Harrison M. The graphical presentation of decision support information in an intelligent anaesthesia monitor. Artif Intell Med 2001;22(2):91. [Medline]
    39. Charbonnier S. On line extraction of temporal episodes from ICU high-frequency data: a visual support for signal interpretation. Comput Methods Programs Biomed 2005 May;78(2):115-132. [CrossRef] [Medline]
    40. Shabot M, Carlton P, Sadoff S, Nolan-Avila L. Graphical reports and displays for complex ICU data: a new, flexible and configurable method. Comput Methods Programs Biomed 1986 Mar;22(1):111-116. [Medline]
    41. Douglas JR, Ritter MJ. Implementation of an Anesthesia Information Management System (AIMS). Ochsner J 2011;11(2):102-114 [FREE Full text] [Medline]
    42. Koch S, Weir C, Westenskow D, Gondan M, Agutter J, Haar M, et al. Evaluation of the effect of information integration in displays for ICU nurses on situation awareness and task completion time: a prospective randomized controlled study. Int J Med Inform 2013 Aug;82(8):665-675. [CrossRef] [Medline]
    43. Kamaleswaran R, Collins C, James A, McGregor C. PhysioEx: visual analysis of physiological event streams. Comput Graphics Forum 2016 Jul 04;35(3):331-340. [CrossRef]
    44. Wachter S, Agutter J, Syroid N, Drews F, Weinger M, Westenskow D. The employment of an iterative design process to develop a pulmonary graphical display. J Am Med Inform Assoc 2003;10(4):363-372 [FREE Full text] [CrossRef] [Medline]
    45. Anders S, Albert R, Miller A, Weinger MB, Doig AK, Behrens M, et al. Evaluation of an integrated graphical display to promote acute change detection in ICU patients. Int J Med Inform 2012 Dec;81(12):842-851 [FREE Full text] [CrossRef] [Medline]
    46. Kilman D, Forslund D. An international collaboratory based on virtual patient records. Commun ACM 1997;40(8):110-117. [CrossRef]
    47. van Bemmel J, van Ginneken A, Stam B, van Mulligen E. Virtual electronic patient records for shared care. Stud Health Technol Inform 1998;52:37-41. [Medline]
    48. Tange H. The paper-based patient record: is it really so bad? Comput Methods Programs Biomed 1995;48(1-2):127-131. [Medline]
    49. Margulies D, McCallie J, Elkowitz A, Ribitzky R. An integrated hospital information system at children's hospital. Presented at: Proceedings of the Annual Symposium on Computer Application in Medical Care; 1990; New York p. 699.
    50. Cole W, Stewart J. Metaphor graphics to support integrated decision making with respiratory data. Int J Clin Monit Comput 1993 May;10(2):91-100. [Medline]
    51. Were M, Shen C, Bwana M, Emenyonu N, Musinguzi N, Nkuyahaga F, et al. Creation and evaluation of EMR-based paper clinical summaries to support HIV-care in Uganda, Africa. Int J Med Inform 2010 Feb;79(2):90-96 [FREE Full text] [CrossRef] [Medline]
    52. Miller A, Sanderson P. Evaluating an information display for clinical decision making in the intensive care unit. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2003 Oct 01;47(3):576-580. [CrossRef]
    53. Gather U, Imhoff M, Fried R. Graphical models for multivariate time series from intensive care monitoring. Stat Med 2002 Sep 30;21(18):2685-2701. [CrossRef] [Medline]
    54. Miller GA. The magical number seven, plus or minus two: some limits on our capacity for processing information. 1956. Psychol Rev 1994 Apr;101(2):343-352. [Medline]
    55. Eppler M, Mengis J. The concept of information overload: a review of literature from organization science, accounting, marketing, MIS, and related disciplines. Inf Soc 2004 Nov;20(5):325-344 [FREE Full text] [CrossRef]
    56. de Oliveira M, Levkowitz H. From visual data exploration to visual data mining: a survey. IEEE Trans Visual Comput Graphics 2003 Jul;9(3):378-394 [FREE Full text] [CrossRef]
    57. Bawden D, Robinson L. The dark side of information: overload, anxiety and other paradoxes and pathologies. J Inf Sci 2008 Nov 21;35(2):180-191 [FREE Full text] [CrossRef]
    58. Speier C, Valacich J, Vessey I. The influence of task interruption on individual decision making: an information overload perspective. Decis Sci 1999 Mar;30(2):337-360 [FREE Full text] [CrossRef]
    59. Endsley M. Toward a theory of situation awareness in dynamic systems. Hum Factors 1995 Mar 01;37(1):32-64 [FREE Full text] [CrossRef]
    60. Bui A, Aberle D, Kangarloo H. TimeLine: visualizing integrated patient records. IEEE Trans Inform Technol Biomed 2007 Jul;11(4):462-473 [FREE Full text] [CrossRef]
    61. Duncan R, Saperia D, Dulbandzhyan R, Shabot M, Polaschek J, Jones D. Integrated Web-based viewing and secure remote access to a clinical data repository and diverse clinical systems. Proc AMIA Symp 2001:149-153 [FREE Full text] [Medline]
    62. Meyer M, Levine W, Brzezinski P, Robbins J, Lai F, Spitz G, et al. Integration of hospital information systems, operative and peri-operative information systems, and operative equipment into a single information display. AMIA Annu Symp Proc 2005:1054 [FREE Full text] [Medline]
    63. Sanderson P. The multimodal world of medical monitoring displays. Appl Ergon 2006 Jul;37(4):501-512. [CrossRef] [Medline]
    64. Georgopoulos D, Prinianakis G, Kondili E. Bedside waveforms interpretation as a tool to identify patient-ventilator asynchronies. Intensive Care Med 2006 Jan;32(1):34-47. [CrossRef] [Medline]
    65. Enison E, Dayhoff R, Fletcher R. Graphical electrocardiogram waveforms as part of an integrated hospital system's patient record. Proc Annu Symp Comput Appl Med Care 1993:373-375 [FREE Full text] [Medline]
    66. Burns C. Putting it all together: improving display integration in ecological displays. Hum Factors 2000 Jun 01;42(2):226-241. [CrossRef] [Medline]
    67. Zinser K, Frischenschlager F. Multimedia's push into power. IEEE Spectr 1994 Jul;31(7):44-48. [CrossRef]
    68. Effken J, Loeb R, Kang Y, Lin Z. Clinical information displays to improve ICU outcomes. Int J Med Inform 2008 Nov;77(11):765-777. [CrossRef] [Medline]
    69. Blike GT, Surgenor SD, Whalen K, Jensen J. Specific elements of a new hemodynamics display improves the performance of anesthesiologists. J Clin Monit Comput 2000;16(7):485-491. [Medline]
    70. Görges M, Förger K, Westenskow D. A trend based decision support system for anesthesiologists improves diagnosis speed and accuracy. Presented at: Proceedings of the Annual Mountain West Biomedical Engineering Conference; 2006; Snowbird, UT.
    71. Albert R, Agutter J, Syroid N, Johnson K, Loeb R, Westenskow D. A simulation-based evaluation of a graphic cardiovascular display. Anesth Analg 2007 Nov;105(5):1303-11, table of contents. [CrossRef] [Medline]
    72. Havre S, Hetzler B, Nowell L. ThemeRiver: visualizing theme changes over time. IEEE Symposium on Information Visualization. InfoVis 2000:115 [FREE Full text] [CrossRef]
    73. Law A, Freer Y, Hunter J, Logie R, McIntosh N, Quinn J. A comparison of graphical and textual presentations of time series data to support medical decision making in the neonatal intensive care unit. J Clin Monitoring Computing 2005;19(3):94. [CrossRef]
    74. Drews F, Westenskow D. The right picture is worth a thousand numbers: data displays in anesthesia. Hum Factors 2006;48(1):59-71. [Medline]


    Abbreviations

    0: No Change
    App: Application
    C: Curves
    Des: Design
    Eval: Evaluation
    Exp: Experiment
    G: Glyph
    MT: Metaphoric display
    Neg: Negative
    NI: Not included
    O: Object
    OB: Object-based display
    Pos: Positive
    Sim: Simulated
    T: Text
    TB: Tabular display
    WF: waveform display


    Edited by CL Parra-Calderón; submitted 30.09.15; peer-reviewed by S Van de Velde, N Farre; comments to author 29.11.15; revised version received 23.02.16; accepted 06.09.16; published 21.11.16

    ©Rishikesan Kamaleswaran, Carolyn McGregor. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 21.11.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.