TY  - JOUR
AU  - Zheng, Chengyi
AU  - Ackerson, Bradley
AU  - Qiu, Sijia
AU  - Sy, Lina S
AU  - Daily, Leticia I Vega
AU  - Song, Jeannie
AU  - Qian, Lei
AU  - Luo, Yi
AU  - Ku, Jennifer H
AU  - Cheng, Yanjun
AU  - Wu, Jun
AU  - Tseng, Hung Fu
PY  - 2024
DA  - 2024/9/10
TI  - Natural Language Processing Versus Diagnosis Code–Based Methods for Postherpetic Neuralgia Identification: Algorithm Development and Validation
JO  - JMIR Med Inform
SP  - e57949
VL  - 12
KW  - postherpetic neuralgia
KW  - herpes zoster
KW  - natural language processing
KW  - electronic health record
KW  - real-world data
KW  - artificial intelligence
KW  - development
KW  - validation
KW  - diagnosis
KW  - EHR
KW  - algorithm
KW  - EHR data
KW  - sensitivity
KW  - specificity
KW  - validation data
KW  - neuralgia
KW  - recombinant zoster vaccine
AB  - Background: Diagnosis codes and prescription data are used in algorithms to identify postherpetic neuralgia (PHN), a debilitating complication of herpes zoster (HZ). Because of the questionable accuracy of codes and prescription data, manual chart review is sometimes used to identify PHN in electronic health records (EHRs), which can be costly and time-consuming. Objective: This study aims to develop and validate a natural language processing (NLP) algorithm for automatically identifying PHN from unstructured EHR data and to compare its performance with that of code-based methods. Methods: This retrospective study used EHR data from Kaiser Permanente Southern California, a large integrated health care system that serves over 4.8 million members. The source population included members aged ≥50 years who received an incident HZ diagnosis and accompanying antiviral prescription between 2018 and 2020 and had ≥1 encounter within 90‐180 days of the incident HZ diagnosis. The study team manually reviewed the EHR and identified PHN cases. For NLP development and validation, 500 and 800 random samples from the source population were selected, respectively. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F-score, and Matthews correlation coefficient (MCC) of NLP and the code-based methods were evaluated using chart-reviewed results as the reference standard. Results: The NLP algorithm identified PHN cases with a 90.9% sensitivity, 98.5% specificity, 82% PPV, and 99.3% NPV. The composite scores of the NLP algorithm were 0.89 (F-score) and 0.85 (MCC). The prevalences of PHN in the validation data were 6.9% (reference standard), 7.6% (NLP), and 5.4%‐13.1% (code-based). The code-based methods achieved a 52.7%‐61.8% sensitivity, 89.8%‐98.4% specificity, 27.6%‐72.1% PPV, and 96.3%‐97.1% NPV. The F-scores and MCCs ranged between 0.45 and 0.59 and between 0.32 and 0.61, respectively. Conclusions: The automated NLP-based approach identified PHN cases from the EHR with good accuracy. This method could be useful in population-based PHN research. 
SN  - 2291-9694
UR  - https://medinform.jmir.org/2024/1/e57949
UR  - https://doi.org/10.2196/57949
DO  - 10.2196/57949
ID  - info:doi/10.2196/57949
ER  -