This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
We recently demonstrated that quality of spirometry in primary care could markedly improve with remote offline support from specialized professionals. It is hypothesized that implementation of automatic online assessment of quality of spirometry using information and communication technologies may significantly enhance the potential for extensive deployment of a high quality spirometry program in integrated care settings.
The objective of the study was to elaborate and validate a Clinical Decision Support System (CDSS) for automatic online quality assessment of spirometry.
The CDSS was done through a three step process including: (1) identification of optimal sampling frequency; (2) iterations to build-up an initial version using the 24 standard spirometry curves recommended by the American Thoracic Society; and (3) iterations to refine the CDSS using 270 curves from 90 patients. In each of these steps the results were checked against one expert. Finally, 778 spirometry curves from 291 patients were analyzed for validation purposes.
The CDSS generated appropriate online classification and certification in 685/778 (88.1%) of spirometry testing, with 96% sensitivity and 95% specificity.
Consequently, only 93/778 (11.9%) of spirometry testing required offline remote classification by an expert, indicating a potential positive role of the CDSS in the deployment of a high quality spirometry program in an integrated care setting.
High quality spirometry testing across health care levels is pivotal for proper management of patients with prevalent chronic respiratory disorders, namely asthma and chronic obstructive pulmonary disease (COPD) [
We have recently reported the effectiveness of a Web-based application for remote offline expert support to enhance the quality of spirometry in primary care. High quality testing improved in a sustainable manner with the remote support [
In the Basque Country (Spain), the ongoing regional deployment of the Web-based offline support program from specialists to primary care will cover the entire population, 2.2 million inhabitants, by the end of 2014 [
Ideally, extensive deployment of a high quality spirometry program in the community should offer accessibility to standardized raw spirometric data through a technological architecture providing interoperability across health care tiers. To this end, a Clinical Document Architecture for spirometry using Health Level Seven v3 standards was recently made available by the Catalan Health Department [
We hypothesize that elaboration and validation of a clinical decision support system (CDSS) for online automatic assessment and certification of quality of spirometry in primary care may represent a pivotal step toward regional adoption of the high quality spirometry program with an integrated care approach.
The current study is part of the refinement of the ongoing deployment of the high quality spirometry program in Catalonia [
The process was done using the 24 standard flow-volume and volume-time curves from the pulmonary waveform generator recommended by the American Thoracic Society/European Respiratory Society (ATS/ERS) [
The construction of an initial version of the CDSS was carried out using the 24 standard spirometry curves [
The CDSS combines the different aspects assessed on the spirometry curve in one score with three different categories: (1) grade 0, rejected due to unacceptable morphology of the spirometry curve; (2) grade 1, acceptable for further classification according to
Quality scores for spirometric maneuvers according to ATS/ERS standardization [
Scores | Maneuvers |
Aa | 3 acceptable maneuvers, and best 2 matched with differences in FVCband/or FEV1<150 ml |
B | 3 acceptable maneuvers, and best 2 matched with differences in FVCband/or FEV1 c<200 ml |
C | 2 acceptable maneuvers, and best 2 matched with differences in FVC and/or FEV1 c<250 ml |
D | 1 acceptable maneuver |
F | No acceptable maneuvers |
aHigh quality spirometries, A and B scores, correspond to A, 3 acceptable maneuvers with differences in FVC and/or FEV1<150 ml; and B, 3 acceptable maneuvers with differences in FVC and/or FEV1<200 ml; C, to high variability among maneuvers; D, only one acceptable maneuver; and F no acceptable maneuver.
bFVC = forced vital capacity
cFEV1= forced expiratory volume in the first second
Flow of the process followed to elaborate and validate the Clinical Decision Support System (CDSS). ATS=American Thoracic Society; FS=forced spirometry.
The CDSS systematically assessed 27 different characteristics of each spirometry curve, as displayed in
List of criteria of the forced spirometry curve explored by the CDSS.
Forced spirometry curve | Criteriai |
BEVa trad | Back extrapolation >0.15 L or < 5% of FVCg |
EOTVb trad | End of test criteria, volume < 0.025 L in time ≥1 s |
Texc | Time of end FVCg(Tex>6 s) |
EOTVb new (5 criteria) | a) EOTVb< 0.025 L or Texc>6 s; |
Peak_Valley_Single | High local maximum (peak) and minimum (valley) in FVecurve |
Peak_Valley_Combined | High local maximum (peak) and minimum (valley) in FVecurve close to FEV1 h |
VTd end | Irregularity or oscillation at the end of FTmcurve |
FVe_slope_single | Variation of FVeslope or high FVeslope |
FVe_slope_combined | Variation of FVeslope and high FVeslope |
FVeSlope_Test_Combo | Irregularity and variation of FVeslope or high FVeslope |
FVeSlope_Test_Combo_Area Under Rectj | Irregularity or variation of FVeslope and high FVeslope |
FVeSlope_Test_Combo4 | Irregularity and variation of FVeslope and high FVeslope |
Diff_singlek | Irregular concavity-convexity before the PEFfvalue in FVecurve |
Diff_combinedl | Irregular slope and irregular concavity-convexity before the PEFfvalue in FVecurve |
PEFf TimeUp | Time to archive PEFf< 130 milliseconds |
PEFf TimeDown | Time to archive PEFf> 0.25 milliseconds |
PEFf Cut | PEFfis not a peak in FVecurve (is plane), volume (Fn=PEFf) > 15 % FVCg |
PEFf Cut2 FEV1 h | PEFfis not a peak in FVecurve (is plane), volume (Fn=PEFf) > 17.5 % FEV1 h |
PEFf DoublePeak | PEFfbimodal in FVecurve |
PEFf Slow | Volume to archive PEFf< 20% FVCg |
aBEV = back extrapolation
bEOTV = end of test criteria, volume
cTex = Time to end FVC
dVT = volume/time curve
eFV = flow/volume curve
fPEF = peak expiratory flow
gFVC = forced vital capacity
hFEV1= forced expiratory volume in the first second
iThe list includes the classical parameters used by ATS/ERS guidelines [
jRect = rectum
kDiff single= irregular concavity-convexity before the PEFf value in flow volumen curve concavity or convexity exists if the extracted signal metric
lDiff_combined = irregular slope and irregular concavity-convexity before the peak expiratory flow value in flow volume curve
mFT = flow/time curve
nF=flow
The refined version of the CDSS was compared with a database of 778 curves from 291 patients from one of the primary care centers in Barcelona. The spirometry testing was done using a spirometer (Sibel 120, SIBELMED, Barcelona Spain). Again, the score generated by the CDSS was compared with the one obtained from the same expert evaluator.
The use of the two patient databases, for refinement and validation purposes, was approved by the Ethical Committee of the Hospital Clínic i Provincial de Barcelona.
The ATS database [
Equations for data analysis. F=flow; V=volume; i=1,…,N; N=length of the sequence; true positive (TP) corresponds to curves classified as grade 0 by both CDSS and the evaluator; true negative (TN) corresponds to curves classified as grade 1 by the CDSS and the by the evaluator; false positive (FP) indicates curves classified as grade 0 by the CDSS, but classified in grade 1 by the evaluator; and, false negative (FN) corresponds to curves classified as grade 1 by the CDSS, but as grade 0 by the evaluator.
The sampling frequency that provided the highest sensitivity and specificity for the analysis carried out with the 24 standard spirometry curves recommended by the ATS [
Both sensitivity and specificity of the CDSS were initially calculated with the 24 standard spirometry curves recommended by the ATS [
The validation study using 778 curves from 291 patients showed the following distribution of spirometry curves, 419/778 maneuvers (53.8%) were appropriately classified as bad curves (grade 0); 266/778 maneuvers (34.2%) were appropriately classified as good curves (grade 1); and only 93/778 maneuvers (11.9%) needed an offline review by a lung function expert to assess quality of the test (grade 2; see
The current research has generated and validated a CDSS that shows the ability to classify a reasonable percentage of spirometry curves, 685/778 (88.1%) as either acceptable (grade 1) or bad maneuvers (grade 0). Only 93/778 (11.9%) of the curves were classified as undefined (grade 2) and were candidates for offline remote validation by an expert. Moreover, we observed that both sensitivity and specificity of the CDSS were very high. Consequently, the results seem to indicate that a vast majority of spirometry testing carried out by nonspecialized professionals in primary care can be reliably assessed online, and the high quality spirometry program partly based on remote automatic evaluation of the testing could be considered ready for regional scalability. Obviously, further steps toward extensive deployment of the program must be planned with caution. A proper monitoring of the potential for generalization of the current results and the need for further refinements of the current CDSS should be taken into account.
The results of the current research overcome some of the limitations of the existing computer-based algorithms generating automatic feedback, as reported in [
In the new scenario, as indicated by the business process management notation (BPMN) diagram (
Previous reports have indicated the potential of telemedicine to enhance both quality and diagnostic potential of spirometry testing carried out by nonexpert professionals [
We acknowledge that high quality spirometry programs combine several different dimensions, namely: (1) professional coaching [
We acknowledge two principal limitations of the study. First, we included only one expert observed (FB). The CDSS should be reassessed in the future with the inclusion of at least 3 different experts. Moreover, the current study evaluates the CDSS in an isolated manner. But, further assessment of the whole clinical process as defined in the BPMN (see
To our knowledge, the current study constitutes the first successful attempt to validate an automatic CDSS for large scale online assessment of quality of spirometry testing. The incorporation of the CDSS into the Web-based application for remote assistance to primary care professionals [
The results indicate a high potential of the CDSS for discrimination between good and poor quality results of spirometry testing, but they require further independent validation before specific plans for implementation are materialized.
The algorithm for computing maneuver acceptability, using the 27 set of criteria.
Three examples with curves classified as Grade 0, 1 and 2. The Business Process Model Notation (BPMN) diagram displays the use of the CDSS for quality control in primary care within a coordinated care scenario. The results of the protocol undertaken to identify the optimal sampling frequency during the first iterative process are shown here.
For each FS curve, the results generated by the CDSS are compared with those provided by the expert professional. It is of note, that only the expiratory portion of the FS manouevres was taken into account for analysis.
American Thoracic Society
business process management notation
clinical decision support system
chronic obstructive pulmonary disease
European Respiratory Society
The authors thank Jordi Giner of Hospital de la Santa Creu i Sant Pau, in Barcelona for providing the validation database. This project was supported by Inforegió (AGAUR) 2008; NEXES (Supporting Healthier and Independent Living for Chronic Patients and Elderly, CIP-ICT-PSP-2007-225025); FIS PI09/90634. Servicios Innovadores de Atencion Integrada para Pacientes Crónicos - PITES- ISCIII 2010-12; EC-FP7 Programme, Synergy-COPD, GA nº 270086; TAMESIS (TEC2011-22746, Spanish Government) CIBER of Bioengineering, Biomaterials and Nanomedicine; Research Fellowship Grant FPU AP2009-0858 from the Spanish Government; and, Catalan Master Plan of Respiratory Diseases (PDMAR).
None declared.