Maintenance Note

On Friday, August 31, 2018 at 12:00 pm Eastern Time, JMIR will be completing a server migration to improve site stability and user experience. We expect to be back online Friday, August 31, 2018 at 5:00 pm Eastern Time. Should any problems arise our technical team will be using the weekend to resolve them, and users will be able to access our site by Sunday, September 2, 2018 at 1:00pm Eastern Time.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 02.05.18 in Vol 6, No 2 (2018): Apr-Jun

This paper is in the following e-collection/theme issue:

    Original Paper

    Automated Modular Magnetic Resonance Imaging Clinical Decision Support System (MIROR): An Application in Pediatric Cancer Diagnosis

    1Institute of Cancer and Genomic Sciences, University of Birmingham, Birmingham, United Kingdom

    2Birmingham Children Hospital NHS Trust, Birmingham, United Kingdom

    3Physical Sciences of Imaging in Biomedical Sciences Doctoral Training Centre, University of Birmingham, Birmingham, United Kingdom

    Corresponding Author:

    Andrew Peet, PhD, FRCPCH

    Institute of Cancer and Genomic Sciences

    University of Birmingham

    Edgbaston

    Birmingham,

    United Kingdom

    Phone: 44 0121 333 8234

    Fax:44 0121 333 8241

    Email:


    ABSTRACT

    Background: Advances in magnetic resonance imaging and the introduction of clinical decision support systems has underlined the need for an analysis tool to extract and analyze relevant information from magnetic resonance imaging data to aid decision making, prevent errors, and enhance health care.

    Objective: The aim of this study was to design and develop a modular medical image region of interest analysis tool and repository (MIROR) for automatic processing, classification, evaluation, and representation of advanced magnetic resonance imaging data.

    Methods: The clinical decision support system was developed and evaluated for diffusion-weighted imaging of body tumors in children (cohort of 48 children, with 37 malignant and 11 benign tumors). Mevislab software and Python have been used for the development of MIROR. Regions of interests were drawn around benign and malignant body tumors on different diffusion parametric maps, and extracted information was used to discriminate the malignant tumors from benign tumors.

    Results: Using MIROR, the various histogram parameters derived for each tumor case when compared with the information in the repository provided additional information for tumor characterization and facilitated the discrimination between benign and malignant tumors. Clinical decision support system cross-validation showed high sensitivity and specificity in discriminating between these tumor groups using histogram parameters.

    Conclusions: MIROR, as a diagnostic tool and repository, allowed the interpretation and analysis of magnetic resonance imaging images to be more accessible and comprehensive for clinicians. It aims to increase clinicians’ skillset by introducing newer techniques and up-to-date findings to their repertoire and make information from previous cases available to aid decision making. The modular-based format of the tool allows integration of analyses that are not readily available clinically and streamlines the future developments.

    JMIR Med Inform 2018;6(2):e30

    doi:10.2196/medinform.9171

    KEYWORDS



    Introduction

    Magnetic resonance imaging (MRI) is a fast-growing clinical imaging modality and has become the modality of choice for the evaluation of disease and treatment management across multiple therapeutic areas. It has increasingly been used in oncology, central nervous system diseases, musculoskeletal disorders, and cardiovascular disease due to its superior soft-tissue imaging capabilities, lack of ionizing radiation, and noninvasive nature [1-4].

    MRI technology constantly advances with new magnetic resonance applications being pioneered, investigated, mainstreamed, and added to clinical applications and capabilities. Nevertheless, clinical interpretation remains largely by qualitative expert review. In addition, new advanced and computationally intensive medical quantitative image analysis techniques are constantly being developed and validated. These techniques have allowed the discovery of specific biomarkers of both disease and treatment response and have exposed clinicians to new information in a computable format [5-8]. However, the growing and versatile amount of magnetic resonance–derived information can form an insurmountable obstacle to the individual clinician; in particular, the use of quantitative MRI biomarkers requires further improvement in accessibility and presentation to aid decision making [9].

    In the past decade, clinical decision support (CDS) systems have increasingly gained attention, and the routine uptake of these intelligent systems is becoming more common [10-17]. Introduction of CDSs has provided clinicians and health care investigators with a platform for extraction of relevant information to aid decision making, prevent errors, and enhance health care. CDSs include a range of options from computerized alerts, reminders, and clinical guidelines to diagnostic support and clinical workflow through computer-assisted diagnosis tools (CAD) [18-26]. There are several clinically implemented or research-based CADs available for medical image analysis [23,26-29]. However, majority of them lack at least one of the following: (1) a user-friendly graphical interface to be used by clinicians in their clinical routine; (2) system performance is often not compared with radiologist diagnosis in the absence of the tool or when the tool is utilized; (3) are not MRI based; (4) are designed for one particular disease; and (5) are just a single postprocessing tool or analysis algorithm, which also provides a likelihood for a disease and does not offer decision support for the clinicians (ie, in form of only providing additional structured information for comparison with available other relevant diagnosis). These types of solutions have shown to suffer from high false positives [9].

    Availability of a user-friendly and flexible MRI CAD that encompasses a variety of medical image analysis techniques and postprocessing methods and can act as a CDS could facilitate the uptake of new advanced magnetic resonance techniques in the real-time clinical setting; it could also allow health care investigators to interrogate their data in a scientifically informative and convenient manner to determine a robust and efficient diagnosis. The aim of this study was to design, develop, and evaluate a medical image region of interest analysis tool and repository (MIROR) platform for conventional magnetic resonance data aimed at improving clinical performance through the provision of real-time diagnostic support for clinicians.

    To the best of our knowledge, there is only the International Network for Pattern Recognition of Tumours Using Magnetic Resonance Decision Support System validated and available for the analysis of magnetic resonance spectroscopy (MRS) data [30]. However, this CDS is developed for diagnosing and grading adult brain tumors and is based on MRS only. There is no CDS for both MRI and MRS analysis with a robust user interface for clinical routine use that is capable of creating and updating a validated repository for different diagnostic problems.


    Methods

    Clinical Decision Support System Design

    Features available in the presented version of the MIROR are (1) a clinician-friendly graphical user interface; (2) measurement of morphologic properties such as size, shape, volume, length dimensions, and center-of-mass location of the region of interest (ROI); (3) an integrated magnetic resonance diffusion-weighted imaging (DWI) analysis application based on intravoxel incoherent motion (IVIM) model; (4) statistical data analysis of the ROI overlaid on standard MRI images (such as T1-weighted and T2-weighted scans or the advanced quantitative maps) to provide decision support in forms of comparison with other differential diagnosis and several different image volumes to aid diagnosis and determination of prognosis; and (5) a self-archiving repository of the extracted data and features. Availability of the latter 2 options in combination with the first 3 will move the designed tool from a CAD toward becoming a CDS for MRI data. MIROR also allows investigators to further grow, advance, and combine different analysis techniques and types of imaging sequences to extend the tool to a more sophisticated decision support, dependent on their individual center’s needs to better inform diagnosis. MIROR’s self-archiving, evolving repository is the core of its decision support. This unique feature of the MIROR distinguishes it from pervious CADs and CDSs. First, the repository’s continuous development allows for improvement in the predication accuracy for the available biomarkers and disease in the database; second, it permits provision of a decision support system compatible to additional disease types by means of importing and appending the repository.

    We used a modular and open architecture design [23] in the design and implementation of MIROR to be able to adapt to the constant increase and development in the MRI sequences; it will also make room for consequent advances in the related analysis applications and allow future development of additional new workflows. Additionally, we used Mevislab software (v. 2.7.1, MeVis AG- Fraunhofer-MEVIS) [31,32], a research-based rapid prototyping platform for medical image processing, for development of the MIROR to achieve the latter. Post processing, quantitative and statistical analysis functionalities embedded in MIROR were either developed using Python (v. 2.7, embedded within Mevislab) or were imported from the Mevislab library. The MIROR repository was developed using Python. Each individual independent module of MIROR was developed, evaluated, and tested by different groups within the team in their own life cycle and schedules before addition to the final product. A hierarchical structure of MIROR infrastructure is represented in Figure 1. MIROR self-archiving, evolving repository is the core of its decision support. This feature of the MIROR distinguishes it from pervious computer-assisted diagnosis tools (CADs) and CDSs.

    Based on the recommendations of the American medical informatics association [33], an evidence-adaptive approach was employed in the design of the MIROR by utilizing its knowledge base to derive from—and reflect on—the most up-to-date evidence from the research literature and practice-based sources [34]. The statistical and quantitative analysis module embedded in MIROR are developed based on the literature and local practice-based research and will continue to update in future releases. MIROR is an evolving database of available diagnosis data gathered from routine clinical practice. Outcomes of this repository data analysis can inform future clinical investigations, reflect on the clinical practice, and consequently impact on the MIROR statistical and quantitative analysis module. Conversely, practice-based experience can inform the choice of MRI sequence and parametric maps to be used for analysis and clinical evaluation (Figure 2).

    MIROR can import all file formats supported by the National Library of Medicine Insight Segmentation and Registration Toolkit, such as digital imaging and communications in medicine (DICOM) files, Neuroimaging Informatics Technology Initiative files, JPEG as well as text files, and comma spreadsheets. Built-in Mevislab modules and Python were used to create this functionality. Using the module raw images, postprocessed quantitative maps and data files can be imported from the clinical data warehouses, such as hospital picture archiving and communication system (PACS), the local servers, or MIROR for future analysis and visualization (Figure 1).

    Comparing the MIROR architecture with previously developed CDSs with a clinical data base and domain expert knowledge base, MIROR does not connect to hospital electronic health record (EHR) system or any Internet-based database or medical knowledge representations or guidelines [10,35,36]. Having said so, one should note that imaging data can be imported to MIROR through connection to hospital PACS and therefore can be considered as a semi-integrated CDSs [37]. Moreover, currently available active and robust CDSs benefit from EHR data with very large and historical dataset that changes continuously and contains hidden knowledge. MIROR was designed based on a similar architecture applied to a repository containing a constantly updated independent database. The updating of the database allows the advanced MRI biomarkers to be revised whenever new data are available.

    One of the main strengths of MIROR is its ability to allow for integration of new advanced and computationally intensive quantitative analyses that are not readily available to be used in routine clinical practice under the Advance Quantitative analysis module (Figure 1). In this study, the analysis of multi b-value (b=diffusion weightings) magnetic resonance DWI is embedded in MIROR. The analysis was developed using the well-established and not clinically available intravoxel incoherent motion (IVIM) model, which has been shown to have clinical value in many different tumor types [38-40] as well as in other pathologies [41,42]. Although IVIM provides a similar measure to clinically available apparent diffusion coefficient (ADC), derivation of the additional parameters allows the separation of the perfusion contribution from the true diffusion, resulting in a greater insight to the underlying tissue microenvironment [43-45].

    Figure 1. Flow diagram showing hierarchical structure of the medical image region of interest analysis tool and repository (MIROR) infrastructure. Dashed blue lines indicate direct connection of the module output to the front-end display, solid lines are the connections between internal clinical decision support (CDS) modules, and green dashed lines represent the feedback system to the repository. PACS: picture archiving and communication system; ROI: region of interest; MRI: magnetic resonance imaging. MRI: magnetic resonance imaging; DICOM: digital imaging and communications in medicine.
    View this figure
    Figure 2. Medical Image Region of interest analysis tool and Repository (MIROR) evidence-adaptive protocol. An evidence-adaptive approach was utilized in the design of the MIROR by utilizing its knowledge base to derive from and reflect on the most up-to-date evidence from the research literature and practice-based sources.
    View this figure

    This analysis allows the computation of tissue diffusion coefficient (IVIM-D), pseudo-diffusion coefficient (IVIM-D*), and perfusion fraction (IVIM-f) [46]. However, as a relatively new analysis method, it is currently not available as part of the scanner software packages. The MIROR IVIM module, which is developed using python programming, loads the raw DWI DICOM file and based on it determines whether the IVIM analysis is feasible, based on the number of the b values. The output from the module is the IVIM parameter maps, which can then be used for further ROI-based statistical analysis.

    The MIROR-embedded statistical analysis module provides specialists with instruments for (1) analyzing and interpreting individual patient MRI data and (2) comparing it with results of previous cases directly from the MIROR-evolving repository via powerful statistical techniques to future inform the investigation (Figure 1). MIROR provides statistical analysis of the advanced parametric maps or the standard MRI sequence image’s ROI (ie, diffusion maps produced in MIROR, other imported advance quantitative maps, or T1-weighted and T2-weighted scans). It measures the volume of the defined ROI, creates a histogram for it, performs statistical analysis on the ROI values, and extracts and stores histogram parameters such as entropy, median, mean, different quantiles, skewness, and kurtosis in its repository. Unlike previously reported CAD and CDS tools, using MIROR the medical expert has multiple options, including (1) selection of the population of interest from the repository to work only with data from a specific condition, (2) choosing the MRI biomarker/variable (eg, diffusion, perfusion), and (3) the statistical variable of interest from a complete set of basic and advanced features that cover both clinical and research needs (basic statistics mean, median, variance, standard deviation, quantile, histogram analysis, etc).

    Note that MIROR is a nonregion-specific MRI CDS, and its novel quantitative image analysis and statistical analysis modules are designed to analyze any region of the body and aid in resolving different demanding diagnostic problems.

    The frontend of the MIROR is a clinician’s user-friendly graphical interface that displays MRI images as well as quantified parametric maps and allows clinicians to define their ROI. It also provides real-time morphological and statistical results for compassion with the repository data so as to aid clinical evaluation of the disease.

    Medical Image Region of Interest Analysis Tool and Repository Application to Pediatric Tumor Evaluation

    MIROR is currently being developed, evaluated, and used at Birmingham Children’s Hospital to determine its role in facilitating noninvasive diagnosis in children presenting with solid body tumors in clinical practice.

    Solid masses in children represent a diagnostic dilemma, as neoplastic and non-neoplastic lesions can appear similar on conventional imaging. Although in some cases the clinical history and physical examination findings indicate a likely diagnosis, the majority of cases require further evaluation with MRI to assess the extent of the lesion and make a specific diagnosis. It is often difficult to determine whether a lesion is benign or malignant or identify specific tumor type based on conventional MRI alone.

    Table 1. Body tumor patient cohort demographics.
    View this table

    This study evaluated the impact of information provided by MIROR in aiding clinicians to distinguish between benign and malignant solid body pediatric tumor types using DWI. Accuracy testing involved examination of MIROR for a cohort of real patient cases with recent visits to Birmingham Children’s Hospital and comparison of the MIROR outcome with the radiologist’s initial opinion and final diagnosis derived based on the opinion of the clinical multidisciplinary team of experts together with pathology.

    Magnetic Resonance Imaging Data

    The body tumor patient cohort studied consisted of children (aged 0-16 years) with solid tumors, undergoing diagnostic MRI with multi b-value DWI at Birmingham Children’s Hospital from 2012 to September 2016. A total of 48 children were enrolled, of whom 37 had malignant tumors and the rest had benign tumors. Details of the malignant and benign body tumors along with patients’ demographics are presented in Table 1.

    We performed the MRI on a Siemens Avanto 1.5 T MRI scanner (Siemens Healthcare, Erlangen, Germany). The diffusion-weighted MRI protocol used an echo-planar imaging sequence in an axial acquisition plane with a field-of-view (FOV) 221 to 350 × 172 to 317 mm2, matrix size 122 to 192 × 128 to 192, slice thickness of 5 mm, and in-plane resolution of 1.56 × 1.56 mm2. For all subjects, 6 b-values: 0, 50, 100, 150, 600, and 1000 s/mm2 were acquired in 3 orthogonal directions with TR/TE=3200 to 9900/92 ms and number of averages=3. The signal to noise of the MRI dataset was approximately 30 (SD 10) for b1000 and 60 (SD 10) for b0 images.

    Depending on their ability to cooperate, children were awake, sedated, or under general anesthesia. MRI acquired with different diffusion weightings (b-values) was used to compute ADC (computed from b0 and b1000), IVIM-D, IVIM-D*, and IVIM-f maps. These values are a quantitative measure of diffusion related to tissue cellularity [45,47] and can be useful for tumor characterization. Clinicians currently only use ADC maps in a qualitative manner to help tumor characterization, commenting on restriction of diffusion as a possible marker of malignancy. Advanced quantitative diffusion parameters (ie, IVIM-D, IVIM-D*, and IVIM-f) and means for direct and real-time statistical analysis of these variables are unavailable clinically, despite the growing body of evidence for their potential value in noninvasive diagnosis.

    Discrimination Between Benign and Malignant Tumor Types

    To discriminate between malignant and benign tumors, the authors made use of a leave-one-out cross-validation method combined with a {displaystyle C_{30}^{100}approx 3times 10^{25}.} threshold-based classification approach to determine the potential of individual parameters determined by MIROR. To achieve this, one case of the cohort was assigned in turn as the validation case, with the remaining ones used for training to inform the outcome. The selected validation case was then iteratively changed until all cases had been evaluated exactly once. On the basis of the threshold approach, if the value of the case under study lay within 1 standard deviation of the mean of a statistical parameter of the training tumor group (ie, benign or malignant), it was assigned to that particular group. However, if the value of the parameter under study fell within both tumor group regions, we made use of second-adjusted classification based on distance from the mean of the statistical parameter of the groups in the training set. To further evaluate the significance of the statistical parameters and information provided by MIROR, k-nearest neighbor (KNN) and support vector machine (SVM) pattern recognition techniques, followed by leave-one-out cross-validation, were utilized to test the accuracy of derived data in distinguishing between tumor types, using all parameters as classification features. KNN was chosen for its simplicity and performance on basic recognition problems; it has been a ubiquitous classification method with good scalability. SVM outperforms conventional pattern recognition methods, especially when the number of training data is small and number of input variables is large [48].

    To account for the data skewness and imbalanced distribution of the 2 groups, synthetic minority oversampling technique has been used to allow for building a larger decision region that contains nearby instances of the minority class [49] when KNN and SVM are used.

    Feature selection was performed before classification by means of calculating the significance level of the histogram derived parameters between the tumor groups. Then, benign and malignant tumors’ histogram permanents (ie, Median; 2nd, 5th, 10th, 15th, 25th, 75th, 85th, 90th, and 98th centile values; kurtosis; skewness; and entropy) were compared using the nonparametric Mann-Whitney U test. The authors used the Bonferroni correction method. Parameters showing a significant difference between the 2 groups were used for classification. All statistical analyses were performed using SPSS Statistics (v. 23, Chicago, Illinois) software.

    All patients were consented for research to the UK Children’s Cancer and Leukemia Group, Functional Imaging Group database, a UK National Health Service Research Ethics committee-approved study (Reference number 04/MRE04/41, Health Research Authority East Midlands—Derby, UK, Ethical Review Board Chair, Dr Peter Korczak). Informed participation and publication consent was given by parents/guardians.

    MIROR is aimed at improving clinical practice through the provision of real-time diagnostic support. To ensure achievement of the latter, each individual application and module has been tested in such an environment by allowing clinicians to interrogate it about the most important clinical questions and provide feedback. We used an iterative process of design, testing, and revision of the MIROR by a diverse team, including medical informatics experts, clinical content experts, and end users to ensure reliable translation of the tool to clinical practice. Experts in MRI and medical imaging, including PhD researchers specializing in MRI and data processing, a physician, and a senior consultant radiologist pilot tested MIROR iteratively during the development and refinement of the tool.


    Results

    MRI datasets and ADC produced by the scanner were imported to MIROR from a local PACS. An ROI was drawn around the entire solid tumor for each case on a high-resolution image by one clinician, which was then checked by another (KM and KF) before being transferred to a matched parametric map (eg, ADC and IVIM maps). The entire tumor volume, including cystic and necrotic areas, was included in the ROI to determine representative data for heterogeneous tumors [50]. A histogram of the drawn ROI was constructed; the mean, median, 2nd-98th percentile values, skewness, kurtosis, and entropy of the histograms were calculated for all tumors, recorded, and stored on a database. Figure 3 demonstrates a screenshot of MIROR for all of the described stages. Advance IVIM parametric maps D and f are also shown in Figure 4.

    Figure 5 represents an example of MIROR’s decision support module for evaluating benign and malignant cases.

    To establish differences between the malignant and benign lesions histogram parameters, the authors compared the histogram-derived parameters in the repository. Analysis showed apparent differences between malignant and benign tumors, with lower ADC values and higher skewness and kurtosis in malignant lesions. There was no significant difference between 85th (P=.12), 90th (P=.22), 95th (P=.82), or 98th centile (P=.41) ADC values between benign and malignant tumors (Table 2). Malignant tumors demonstrated statistically significantly lower mean (P=.03), median (P=.005), 2nd (P=.04), 5th (P=.02), 10th (P=.01), 15th (P=.005), 25th (P=.004), and 75th centile (P=.03) ADC values, higher kurtosis (P<.01), more positively skewed histograms (P<.001), and higher entropy (P=.03). These results are in agreement with similar studies published for adults [51-53].

    The feasibility of MIROR to provide distinctive surplus information, which would further aid diagnosis, was evaluated using histogram-derived parameters with the statistically significant differences between the 2 groups. The accuracy of individual statistical parameters to discriminate between specific benign or malignant tumors is presented in Table 3. Due to the high correlation between the ADC centiles, only 15% and 75% centiles were used for classification.

    Figure 3. Medical image region of interest analysis tool and repository (MIROR) user-interface patient view. This figure represents data for a malignant tumor case. Here, the region of interest (ROI) is drawn on a high-resolution image and overlaid on the corresponding parametric apparent diffusion coefficient (ADC) map. Measurement of morphologic properties of the ROI, zooming, scaling, rotating of the estimated object surface, and histogram analysis of the overlaid ROI on voxel-by-voxel parametric maps, is supported to enhance quality assessment.
    View this figure
    Figure 4. Medical image region of interest analysis tool and repository (MIROR) user-interface intravoxel incoherent motion (IVIM) maps tabs. This figure represents data for a malignant tumor case. Here, the region of interest (ROI) is drawn on a high-resolution image and overlaid on the corresponding parametric map IVIM-D and IVIM-f. Measurement of morphologic properties of the ROI, zooming, scaling, rotating of the estimated object surface and histogram analysis of the overlaid ROI on voxel-by-voxel parametric maps, is supported to enhance quality assessment.
    View this figure
    Figure 5. Medical image region of interest analysis tool and repository (MIROR) user-interface analysis tab. This figure represents MIROR use as a decision support system for benign and malignant cases. Here, the green histogram line represents the index case under examination; the red line and gray area represent the mean and standard deviation of the selected tumor group for comparison, respectively. The box plot compares median, skewness, entropy, and 25th percentile values of the index case with the tumor types in the database.
    View this figure

    In the studied cohort, skewness, kurtosis, entropy, different percentiles, mean, and median provided important distinctive additional information, which was not available by using qualitative approaches only. Using kurtosis, entropy and 15th percentile for threshold-based classification, 100% of malignant cases were correctly assigned. Mean, median, and skewness had an accuracy of 0.97 in classifying malignant cases. In the benign category, kurtosis, entropy, and 75th percentile achieved full accuracy. Mean and median had an accuracy of 0.91 in classifying benign tumors. Overall, kurtosis and entropy had the highest sensitivity and specificity (sensitivity=1, specificity=1) in discriminating between benign and malignant tumors. Note that the threshold classification was performed based on a single feature input and a 2-step classification process, with a less strict rule in second layer to reclassify the cases in the ambiguous group. Using all of the above extracted features, more advanced pattern recognition techniques, and 10-fold cross-validation, an accuracy of 0.89 (sensitivity=0.97, specificity=0.5, area under the curve [AUC]=0.78) and 0.93 (sensitivity=0.97, specificity=0.58, AUC=0.84) was obtained by SVM and KNN, respectively. Figure 6 shows a comparison of the classifiers using 10-fold cross-validation receiver operating characteristic analysis.

    Table 2. Comparison of apparent diffusion coefficient (ADC) histogram parameters between malignant and benign pediatric tumors using the Mann-Whitney U test.
    View this table
    Table 3. Accuracy of individual statistical parameters along with sensitivity and specificity of the analysis obtained by medical image region of interest analysis tool and repository (MIROR) to discriminate between benign or malignant tumors using 2-step threshold classifications.
    View this table

    To further evaluate MIROR in terms of its added clinical value, radiologist initial diagnosis from the first MRI scans was compared with the final diagnosis obtained from histopathology, and the outcome of the 3 statistical analyses based on MIROR provided information for classifying the tumor types for this cohort of patients (Figure 7). For the 2-step thresholding classification, outcome of the first classification layer is presented to include the ambiguous group. The ambiguous groups for KNN and SVM were identified by thresholding their predication probabilities at above 0.8 and above 0.5 for the accurate and ambiguous assignment of cases, respectively.

    A higher amount of uncertainty was observed in the initial diagnosis of the benign group. The benign group diagnostic uncertainty rate decreased when we used the information provided by MIROR. Moreover, the false diagnosis rate for both the malignant and benign groups was reduced compared with the radiologist’s initial report with all 3 analysis methods. Additional statistical information provided to clinicians by MIROR can allow for a better and more informed noninvasive discrimination of benign and malignant body tumors in children.

    Net reclassification improvement (NRI) [54] for KNN, SVM, and the 2-step thresholding methods using the histogram parameters were calculated to evaluate the level of improvement achieved by these methods compared with the radiologist’s initial reading . The same is presented in Figure 8. KNN had the highest incremental value (NRI=0.35) among all the methods. Histogram parameters on average had a NRI of 0.19 in comparison with the radiologist’s initial read.

    Figure 6. Receiver operating characteristic (ROC) curves of support vector machine and k-nearest neighbor (KNN) in discriminating benign from malignant tumors using medical image region of interest analysis tool and repository (MIROR)-derived parameters. Area under the curve (AUC) was 0.78 for support vector machine (SVM) and 0.84 for KNN.
    View this figure
    Figure 7. Radiologist’s initial diagnosis compared with final diagnosis after histopathology for different tumor types, along with the comparison between Medical Image Region of interest analysis tool and Repository (MIROR) performance evaluated by support vector machine (SVM), k-nearest neighbor (KNN), and 2-step threshold classification methods.
    View this figure
    Figure 8. Net reclassification improvement (NRI) for k-nearest neighbor (KNN), support vector machine (SVM), and the 2-step thresholding methods compared with radiologist’s initial read.
    View this figure

    Discussion

    Continuous developments of magnetic resonance systems have transformed this domain from a pure imaging system to a sophisticated precise metric system that generates a substantial amount of information and data. The complex structure of the clinical data generated often does not ease the difficulty in discriminating between different diagnoses and promotes adoption of intelligent CDSs [55,56]. CDSs allow clinicians and specialists to get insight into the data, test hypotheses, draw conclusions, and directly interact with all the available information. One should note that CDSs should aim to facilitate optimal human performance by harnessing the most advanced imaging and analysis techniques in conjunction with the end user’s own decision-making skills and abilities.

    Radiologists are moving toward quantitative imaging techniques that are difficult to apply and complex to interpret [16,57]. MIROR is a real-time CDS, which can guide clinicians through the implementation and analysis of advanced and new imaging techniques and allow for these new methodologies to find clinical acceptance through translational applications.

    MIROR as a diagnostic tool allows its users to extract specific region morphological features, request specific quantified metrics and features (as a biomarker), and compare with relevant findings available in its repository to gain maximal statistical power with regard to outcome prediction for the input case into the support system. MIROR can direct users to refine their search patterns looking for particular diagnoses, even if they themselves are not immediately aware of the significance of these findings.

    Use of modular programming in the development of MIROR enforces logical boundaries between magnetic resonance analysis applications, thereby improving maintainability [58,59]. Modularity has also allowed development and validation of individual analysis techniques in separate studies to ensure achievement of the important feature of any CDSs, which is its accuracy and appropriateness of the system’s result.

    In terms of its added clinical value and its impact on providing clinical evidence, MIROR will assist clinicians to better understand the pathophysiological difference between the different tumor types and provide information that could help them to better understand the mechanisms of diseases to improve the diagnosis and prognosis of tumors. MRI biomarkers provide information on both the tumor and its interaction with its environment and can potentially provide new information, which is not available from histology or tumor genetics. Analysis of cancer imaging big data will allow uncovering the relation and structure of cancer disease from an angle that has not previously been viewed.

    Although we concentrate on developing these advanced MRI methods as a noninvasive diagnostic aid, they provide important information on tissue properties. Apparent diffusion coefficient shows a strong inverse correlation to cellularity—a key feature of tumors and tumor aggressiveness. Likewise, there is an increasing understanding that IVIM-f is related to tumor vascularity, which is again an important pathophysiological property of the tumor. Making these advanced MRI techniques available to clinicians in their multidisciplinary team meetings, where imaging and histopathology are evaluated together for individual cases, is an important goal and will allow an improved understanding of pathophysiology for these tumors in vivo and ex vivo.

    Limitations

    As a part of future research, we plan to work on functionality and intelligent scaling of quantitative applications of MIROR by further enhancing its statistical capabilities and extension to more embedded advanced quantitative analysis modules such as magnetic resonance spectroscopy and integration of real-time interactive machine learning to optimize the use of available data in MIROR based on the guidelines [60].

    It should be noted that a major limitation of the study was that MIROR was trained and tested with a rather limited set of data analyzed retrospectively. A major focus of future work will be the validation of the tool based on a prospective dataset in real time and in a multicenter clinical setting to reinforce the credibility, usability, and efficiency of the proposed CDSs applications.

    Conclusions

    The proposed CDS is a diagnostic tool and repository that allows the interpretation and analysis of magnetic resonance images to be more accessible and comprehensive for clinicians. The process and experiences described here provide a model for development of the other CDSs attempting to perform a nonregion-specific quantitative analysis of MRI data. MIROR aims to increase clinician’s skillset by introducing newer techniques and up-to-date findings to their repertoire and make information from previous cases available to aid decision making. The modular-based format of the tool allowed integration of analyses that are not readily available clinically, and streamlines future developments. Pipelines for new analysis applications are available or already in development and will be shortly available under the MIROR platform.

    Acknowledgments

    This study is funded through an NIHR Research Professorship, 13-0053. The authors acknowledge funding from the CRUK and EPSRC Cancer Imaging Programme at the Children’s Cancer and Leukaemia Group in association with the MRC and Department of Health (England; C7809/A10342) and the Engineering and Physical Sciences Research Council (EPSRC, EP/F50053X/1). The authors would also like to acknowledge the magnetic resonance research radiographers at Birmingham Children’s Hospital for scanning the patients in this study.

    Authors' Contributions

    NZ developed application of the software, implemented suggested interface components, enabled deployment of the system, identified problems requiring alteration and implemented them upon agreed revisions, conducted data analysis, and drafted the manuscript. EM implemented diffusion application algorithm, tested MIROR, and made suggestions for revisions of MIROR. KM evaluated MIROR, extracted patient data, assisted in drafting the manuscript, and made suggestions for revisions of MIROR. KF, KM, and AP provided guidance and recommendations for modification of the system for this clinical application and participated in early setup of the project. AP conceived of the study, wrote the original grant proposal, supervised development, made suggestions for revisions, and provided final approval of MIROR. All authors read, provided suggestions for revising, and approved the final manuscript.

    Conflicts of Interest

    None declared.

    References

    1. Smith KA, Carrino JA, Carrino J. MRI-guided interventions of the musculoskeletal system. J Magn Reson Imaging 2008 Feb;27(2):339-346. [CrossRef] [Medline]
    2. Morton G, Ishida M, Schuster A, Hussain S, Schaeffter T, Chiribiri A, et al. Perfusion cardiovascular magnetic resonance: comparison of an advanced, high-resolution and a standard sequence. J Cardiovasc Magn Reson 2012 Jun 09;14:34 [FREE Full text] [CrossRef] [Medline]
    3. Hamirani YS, Kramer CM. Cardiac MRI assessment of myocardial perfusion. Future Cardiol 2014 May;10(3):349-358 [FREE Full text] [CrossRef] [Medline]
    4. Kwon T, Jeong IG, Yoo S, Lee J, Hong S, You D, et al. Role of MRI in indeterminate renal mass: diagnostic accuracy and impact on clinical decision making. Int Urol Nephrol 2015 Apr;47(4):585-593. [CrossRef] [Medline]
    5. Padhani AR, Liu G, Koh DM, Chenevert TL, Thoeny HC, Takahara T, et al. Diffusion-weighted magnetic resonance imaging as a cancer biomarker: consensus and recommendations. Neoplasia 2009 Feb;11(2):102-125 [FREE Full text] [Medline]
    6. Patterson DM, Padhani AR, Collins DJ. Technology insight: water diffusion MRI--a potential new biomarker of response to cancer therapy. Nat Clin Pract Oncol 2008 Apr;5(4):220-233. [CrossRef] [Medline]
    7. Waldman AD, Jackson A, Price SJ, Clark CA, Booth TC, Auer DP, National Cancer Research Institute Brain Tumour Imaging Subgroup. Quantitative imaging biomarkers in neuro-oncology. Nat Rev Clin Oncol 2009 Aug;6(8):445-454. [CrossRef] [Medline]
    8. O'Connor JP, Jackson A, Parker GJ, Roberts C, Jayson GC. Dynamic contrast-enhanced MRI in clinical trials of antivascular therapies. Nat Rev Clin Oncol 2012 Feb 14;9(3):167-177. [CrossRef] [Medline]
    9. Mendelson DS, Rubin DL. Imaging informatics: essential tools for the delivery of imaging services. Acad Radiol 2013 Oct;20(10):1195-1212 [FREE Full text] [CrossRef] [Medline]
    10. Shahmoradi L, Farzanehnejad AR, Arji G. Guideline-based clinical decision support systems as an inseparable tool for better cancer care management. Iran J Public Health 2016 Jul;45(7):962-963 [FREE Full text] [Medline]
    11. Turgeon J, Michaud V. Clinical decision support systems: great promises for better management of patients' drug therapy. Expert Opin Drug Metab Toxicol 2016 Sep;12(9):993-995. [CrossRef] [Medline]
    12. Wright A, Sittig DF. A four-phase model of the evolution of clinical decision support architectures. Int J Med Inform 2008 Oct;77(10):641-649 [FREE Full text] [CrossRef] [Medline]
    13. Berrouiguet S, Barrigón ML, Brandt SA, Ovejero-García S, Álvarez-García R, Carballo JJ, MEmind Study Group, et al. Development of a Web-Based Clinical Decision Support System for Drug Prescription: Non-Interventional Naturalistic Description of the Antipsychotic Prescription Patterns in 4345 Outpatients and Future Applications. PLoS One 2016;11(10):e0163796 [FREE Full text] [CrossRef] [Medline]
    14. Johnson AE, Ghassemi MM, Nemati S, Niehaus KE, Clifton DA, Clifford GD. Machine Learning and Decision Support in Critical Care. Proc IEEE Inst Electr Electron Eng 2016 Feb;104(2):444-466 [FREE Full text] [CrossRef] [Medline]
    15. Schiff GD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors? N Engl J Med 2010 Mar 25;362(12):1066-1069. [CrossRef] [Medline]
    16. Stivaros SM, Gledson A, Nenadic G, Zeng XJ, Keane J, Jackson A. Decision support systems for clinical radiological practice -- towards the next generation. Br J Radiol 2010 Nov;83(995):904-914 [FREE Full text] [CrossRef] [Medline]
    17. Wang X, Bie R, Sun Y, Wu Z, Zhou M, Cao R, et al. The architecture of an automatic eHealth platform with mobile client for cerebrovascular disease detection. JMIR Mhealth Uhealth 2013 Aug 09;1(2):e20 [FREE Full text] [CrossRef] [Medline]
    18. Billis AS, Papageorgiou EI, Frantzidis CA, Tsatali MS, Tsolaki AC, Bamidis PD. A decision-support framework for promoting independent living and ageing well. IEEE J Biomed Health Inform 2015 Jan;19(1):199-209. [CrossRef] [Medline]
    19. Tsolaki E, Kousi E, Svolos P, Kapsalaki E, Theodorou K, Kappas C, et al. Clinical decision support systems for brain tumor characterization using advanced magnetic resonance imaging techniques. World J Radiol 2014 Apr 28;6(4):72-81 [FREE Full text] [CrossRef] [Medline]
    20. Njie GJ, Proia KK, Thota AB, Finnie RK, Hopkins DP, Banks SM, Community Preventive Services Task Force. Clinical decision support systems and prevention: a community guide cardiovascular disease systematic review. Am J Prev Med 2015 Nov;49(5):784-795 [FREE Full text] [CrossRef] [Medline]
    21. Pesce F, Diciolla M, Binetti G, Naso D, Ostuni VC, Di Noia T, et al. Clinical decision support system for end-stage kidney disease risk estimation in IgA nephropathy patients. Nephrol Dial Transplant 2016 Jan;31(1):80-86. [CrossRef] [Medline]
    22. Bosch JG, Nijland F, Mitchell SC, Lelieveldt BP, Kamp O, Reiber JH, et al. Computer-aided diagnosis via model-based shape analysis: automated classification of wall motion abnormalities in echocardiograms. Acad Radiol 2005 Mar;12(3):358-367. [CrossRef] [Medline]
    23. Pietka E, Kawa J, Badura P, Spinczyk D. Open architecture computer‐aided diagnosis system. Expert Syst 2010;27(1):17-39. [CrossRef]
    24. Ho TW, Huang CW, Lin CM, Lai F, Ding JJ, Ho YL, et al. A telesurveillance system with automatic electrocardiogram interpretation based on support vector machine and rule-based processing. JMIR Med Inform 2015 May 07;3(2):e21 [FREE Full text] [CrossRef] [Medline]
    25. Pietka E, Gertych A, Pospiechâ Euro Kurkowska S, Cao F, Huang HK, Gilzanz V. Computer-assisted bone age assessment: graphical user interface for image processing and comparison. J Digit Imaging 2004 Sep;17(3):175-188 [FREE Full text] [CrossRef] [Medline]
    26. Litjens G, Debats O, Barentsz J, Karssemeijer N, Huisman H. Computer-aided detection of prostate cancer in MRI. IEEE Trans Med Imaging 2014 May;33(5):1083-1092. [CrossRef] [Medline]
    27. Sonka M, Bosch JG, Lelieveldt BP, Mitchell SC, Reiber JH. Computer-aided diagnosis via model-based shape analysis: cardiac MR and echo. Int Congr Ser 2003 Jun;1256:1013-1018. [CrossRef]
    28. Dorrius MD, Jansen-van der Weide MC, van Ooijen PM, Pijnappel RM, Oudkerk M. Computer-aided detection in breast MRI: a systematic review and meta-analysis. Eur Radiol 2011 Aug;21(8):1600-1608 [FREE Full text] [CrossRef] [Medline]
    29. Kim YJ, Park JW, Kim JW, Park C, Gonzalez JP, Lee SH, et al. Computerized automated quantification of subcutaneous and visceral adipose tissue from computed tomography scans: development and validation study. JMIR Med Inform 2016 Feb 04;4(1):e2 [FREE Full text] [CrossRef] [Medline]
    30. Pérez-Ruiz A, Julià-Sapé M, Mercadal G, Olier I, Majós C, Arús C. The INTERPRET Decision-Support System version 3.0 for evaluation of magnetic resonance spectroscopy data from human brain tumours and other abnormal brain masses. BMC Bioinformatics 2010 Nov 29;11:581 [FREE Full text] [CrossRef] [Medline]
    31. Merkel B, Harz MT, Konrad O, Hahn HK, Peitgen HO. A novel software assistant for clinical analysis of MR spectroscopy with MeVisLab. In: Proceedings of SPIE, Volume 6915, 69152R. 2008 Presented at: Medical Imaging; 2008; San Diego, CA, United States   URL: https:/​/www.​researchgate.net/​profile/​Markus_Wenzel2/​publication/​252297720_A_novel_software_assistant_for_clinical_analysis_of_MR_Spectroscopy_with_MeVisLab/​links/​53cf8a3f0cf2fd75bc59d6ea/​A-novel-software-assistant-for-clinical-analysis-of-MR-Spectroscopy-with-MeVisLab.​pdf [CrossRef]
    32. Egger J, Tokuda J, Chauvin L, Freisleben B, Nimsky C, Kapur T, et al. Integration of the OpenIGTLink network protocol for image-guided therapy with the medical platform MeVisLab. Int J Med Robot 2012 Sep;8(3):282-290 [FREE Full text] [CrossRef] [Medline]
    33. Sim I, Gorman P, Greenes RA, Haynes RB, Kaplan B, Lehmann H, et al. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc 2001;8(6):527-534 [FREE Full text] [Medline]
    34. de Clercq PA, Blom JA, Korsten HH, Hasman A. Approaches for creating computer-interpretable guidelines that facilitate decision support. Artif Intell Med 2004 May;31(1):1-27. [CrossRef] [Medline]
    35. Thomas KW, Dayton CS, Peterson MW. Evaluation of internet-based clinical decision support systems. J Med Internet Res 1999;1(2):E6 [FREE Full text] [CrossRef] [Medline]
    36. Nejat F, El Khashab M, Rutka JT. Initial management of childhood brain tumors: neurosurgical considerations. J Child Neurol 2008 Oct;23(10):1136-1148 [FREE Full text] [CrossRef] [Medline]
    37. Sen A, Banerjee A, Sinha AP, Bansal M. Clinical decision support: converging toward an integrated architecture. J Biomed Inform 2012 Oct;45(5):1009-1017 [FREE Full text] [CrossRef] [Medline]
    38. Federau C, Meuli R, O'Brien K, Maeder P, Hagmann P. Perfusion measurement in brain gliomas with intravoxel incoherent motion MRI. AJNR Am J Neuroradiol 2014 Feb;35(2):256-262 [FREE Full text] [CrossRef] [Medline]
    39. Cho GY, Moy L, Zhang JL, Baete S, Lattanzi R, Moccaldi M, et al. Comparison of fitting methods and b-value sampling strategies for intravoxel incoherent motion in breast cancer. Magn Reson Med 2015 Oct;74(4):1077-1085 [FREE Full text] [CrossRef] [Medline]
    40. Hectors SJ, Wagner M, Besa C, Bane O, Dyvorne HA, Fiel MI, et al. Intravoxel incoherent motion diffusion-weighted imaging of hepatocellular carcinoma: is there a correlation with flow and perfusion metrics obtained with dynamic contrast-enhanced MRI? J Magn Reson Imaging 2016 Oct;44(4):856-864 [FREE Full text] [CrossRef] [Medline]
    41. Wirestam R, Brockstedt S, Lindgren A, Geijer B, Thomsen C, Holtås S, et al. The perfusion fraction in volunteers and in patients with ischaemic stroke. Acta Radiol 1997 Nov;38(6):961-964. [Medline]
    42. Luciani A, Vignaud A, Cavet M, Nhieu JT, Mallat A, Ruel L, et al. Liver cirrhosis: intravoxel incoherent motion MR imaging--pilot study. Radiology 2008 Dec;249(3):891-899. [CrossRef] [Medline]
    43. Chandarana H, Kang SK, Wong S, Rusinek H, Zhang JL, Arizono S, et al. Diffusion-weighted intravoxel incoherent motion imaging of renal tumors with histopathologic correlation. Invest Radiol 2012 Dec;47(12):688-696. [CrossRef] [Medline]
    44. Lemke A, Laun FB, Klauss M, Re TJ, Simon D, Delorme S, et al. Differentiation of pancreas carcinoma from healthy pancreatic tissue using multiple b-values: comparison of apparent diffusion coefficient and intravoxel incoherent motion derived parameters. Invest Radiol 2009 Dec;44(12):769-775. [CrossRef] [Medline]
    45. Klauss M, Lemke A, Grünberg K, Simon D, Re TJ, Wente MN, et al. Intravoxel incoherent motion MRI for the differentiation between mass forming chronic pancreatitis and pancreatic carcinoma. Invest Radiol 2011 Jan;46(1):57-63. [CrossRef] [Medline]
    46. Meeus EM, Novak J, Withey SB, Zarinabad N, Dehghani H, Peet AC. Evaluation of intravoxel incoherent motion fitting methods in low-perfused tissue. J Magn Reson Imaging 2017 May;45(5):1325-1334 [FREE Full text] [CrossRef] [Medline]
    47. Humphries PD, Sebire NJ, Siegel MJ, Olsen OE. Tumors in pediatric patients at diffusion-weighted MR imaging: apparent diffusion coefficient and tumor cellularity. Radiology 2007 Dec;245(3):848-854. [CrossRef] [Medline]
    48. Abe S. Support Vector Machines for Pattern Classification, 2nd ed. London; New York: Springer; 2010.
    49. Zarinabad N, Wilson M, Gill SK, Manias KA, Davies NP, Peet AC. Multiclass imbalance learning: improving classification of pediatric brain tumors from magnetic resonance spectroscopy. Magn Reson Med 2017 Jun;77(6):2114-2124 [FREE Full text] [CrossRef] [Medline]
    50. Woo S, Cho JY, Kim SY, Kim SH. Histogram analysis of apparent diffusion coefficient map of diffusion-weighted MRI in endometrial cancer: a preliminary correlation study with histological grade. Acta Radiol 2014 Dec;55(10):1270-1277. [CrossRef] [Medline]
    51. Xu XQ, Hu H, Su GY, Liu H, Hong XN, Shi HB, et al. Utility of histogram analysis of ADC maps for differentiating orbital tumors. Diagn Interv Radiol 2016;22(2):161-167 [FREE Full text] [CrossRef] [Medline]
    52. Suo ST, Chen XX, Fan Y, Wu LM, Yao QY, Cao MQ, et al. Histogram analysis of apparent diffusion coefficient at 3.0 T in urinary bladder lesions: correlation with pathologic findings. Acad Radiol 2014 Aug;21(8):1027-1034. [CrossRef] [Medline]
    53. Bull JG, Saunders DE, Clark CA. Discrimination of paediatric brain tumours using apparent diffusion coefficient histograms. Eur Radiol 2012 Feb;22(2):447-457. [CrossRef] [Medline]
    54. Jewell ES, Maile MD, Engoren M, Elliott M. Net reclassification improvement. Anesth Analg 2016 Mar;122(3):818-824. [CrossRef] [Medline]
    55. Buckler AJ, Bresolin L, Dunnick NR, Sullivan DC, Aerts HJ, Bendriem B, et al. Quantitative imaging test approval and biomarker qualification: interrelated but distinct activities. Radiology 2011 Jun;259(3):875-884 [FREE Full text] [CrossRef] [Medline]
    56. Sullivan DC. Imaging as a quantitative science. Radiology 2008 Aug;248(2):328-332. [CrossRef] [Medline]
    57. Peh WC. Pitfalls in Diagnostic Radiology. Verlag Berlin Heidelberg: Springer; 2015:978-973.
    58. Hughes J. Why functional programming matters. Comput J 1989;32(2):98-107. [CrossRef]
    59. Parnas DL. On the criteria to be used in decomposing systems into modules. Commun ACM 1972;15(12):1053-1058. [CrossRef]
    60. Luo W, Phung D, Tran T, Gupta S, Rana S, Karmakar C, et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res 2016 Dec 16;18(12):e323 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    ADC: apparent diffusion coefficient
    AUC: area under the curve
    CADs: computer-assisted diagnosis tools
    CDS: clinical decision support
    DICOM: digital imaging and communications in medicine
    DWI: diffusion-weighted imaging
    IVIM: intravoxel incoherent motion
    KNN: k-nearest neighbor
    MIROR: medical image region of interest analysis tool and repository
    MRI: magnetic resonance imaging
    PACS: picture archiving and communication system
    ROI: region of interest
    SVM: support vector machine


    Edited by G Eysenbach; submitted 25.10.17; peer-reviewed by CS Hung, JJ Ding; comments to author 17.12.17; revised version received 10.01.18; accepted 26.01.18; published 02.05.18

    ©Niloufar Zarinabad, Emma M Meeus, Karen Manias, Katharine Foster, Andrew Peet. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 02.05.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.