Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 14.04.20 in Vol 8, No 4 (2020): April

This paper is in the following e-collection/theme issue:

    Original Paper

    A Novel, Integrative Approach for Evaluating Progression in Multiple Sclerosis: Development of a Scoring Algorithm

    1Adelphi Values Ltd, Macclesfield, United Kingdom

    2Novartis Pharma AG, Basel, Switzerland

    3Adelphi Real World Ltd, Macclesfield, United Kingdom

    4Center of Clinical Neuroscience, Neurological University Clinic Carl Gustav Carus, TU Dresden, Dresden, Germany

    Corresponding Author:

    Tjalf Ziemssen, MD

    Center of Clinical Neuroscience

    Neurological University Clinic Carl Gustav Carus

    TU Dresden

    Fetscherstr. 74

    Dresden, 01307

    Germany

    Phone: 49 351458446

    Email: Ziemssen@web.de


    ABSTRACT

    Background: There is an unmet need for a tool that helps to evaluate patients who are at risk of progressing from relapsing-remitting multiple sclerosis to secondary progressive multiple sclerosis (SPMS). A new tool supporting the evaluation of early signs suggestive of progression in multiple sclerosis (MS) has been developed. In the initial stage, concepts relevant to progression were identified using a mixed method approach involving regression on data from a real-world observational study and qualitative research with patients and physicians. The tool was drafted in a questionnaire format to assess these variables.

    Objective: This study aimed to develop the scoring algorithm for the tool, using both quantitative and qualitative research methods.

    Methods: The draft scoring algorithm was developed using two approaches: quantitative analysis of real-world data and qualitative analysis based on physician interviews and ranking and weighting exercises. Variables that were included in the draft tool and regarded as most clinically relevant were selected for inclusion in a multiple logistic regression. The analyses were run using physician-reported data and patient-reported data. Subsequently, a ranking and weighting exercise was conducted with 8 experienced neurologists as part of semistructured interviews. Physicians were presented with the variables included in the draft tool and were asked to rank them in order of strength of contribution to progression and assign a weight by providing a percentage of the overall contribution. Physicians were also asked to explain their ranking and weighting choices. Concordance between physicians was explored.

    Results: Multiple logistic regression identified age, MS disease activity, and Expanded Disability Status Scale score as the most significant physician-reported predictors of progression to SPMS. Patient age, mobility, and self-care were identified as the strongest patient-reported predictors of progression to SPMS. In physician interviews, the variables ranked and weighted as most important were stability or worsening of symptoms, intermittent or persistent symptoms, and presence of ambulatory and cognitive symptoms. Across all physicians, the level of concordance was 0.278 (P<.001), indicating a low to moderate, but statistically significant, level of agreement. Variables were categorized as high (n=8), moderate (n=8), or low (n=10) importance based on the findings from the different approaches described above. Accordingly, the respective questions in the tool were assigned a weight of “three,” “two,” or “one” to inform the draft scoring algorithm.

    Conclusions: This study further confirms the need for a tool to provide a consistent, comprehensive approach across physicians to support the early evaluation of signs indicative of progression to SPMS. The novel and comprehensive approach to develop the draft scoring algorithm triangulates data obtained from ranking and weighting exercises, qualitative interviews, and a real-world observational study. Variables that go beyond the clinically most obvious impairment in lower limbs have been identified as relevant subtle/sensitive signs suggestive of progressive disease.

    JMIR Med Inform 2020;8(4):e17592

    doi:10.2196/17592

    KEYWORDS



    Introduction

    Background

    Onset of secondary progressive disease course is associated with an unfavorable and severe long-term outcome in multiple sclerosis (MS) [1], and there are no distinct biomarkers or clinical criteria to detect the transition to secondary progressive multiple sclerosis (SPMS). Diagnosis of SPMS is usually retrospective in nature and based on the identification of progression independent of relapses [2], often relying on patients’ recollection of worsening of their clinical status as well as the thoroughness of physicians’ inquiries at the regular visits [3]. There is a period of diagnostic uncertainty, which lasts for an average of 3 years [4]. Lack of treatment options, psychological burden imposed on the patients, and concerns regarding reimbursement are additional challenges toward making a definitive diagnosis [5,6].

    With the advent of newer and highly effective therapies, recognizing early indicators of progressive disease may represent a window of opportunity for intervention [4]. A tool that helps to assess the signs of progression may support an early identification of patients who are at a higher risk of transitioning to SPMS. In the past, several studies have evaluated various clinical and magnetic resonance imaging (MRI) variables predictive of the risk of secondary progression based exclusively on empirical or quantitative assessments of different study cohorts [1,7-13]. Some of those studies further developed models or algorithms, predicting the risk of conversion to SPMS—Skoog et al (MS prediction) [12], Manouchehrinia et al (SPMS nomogram) [11], and Lorscheider et al (calculators) [10]. The parameters identified as relevant for conversion are not consistent across the different studies probably because of the differences in their respective study settings, used datasets, and methodologies.

    Objective

    We conducted a comprehensive research study using a mixed methods approach for developing a new tool to support the early evaluation of signs of progressive disease. As a first step, the tool content was developed in the form of a questionnaire based on the results obtained from regression analysis on data from a real-world observational study and insights obtained from the open-ended, qualitative, concept elicitation interviews with patients and physicians [14]. Here, we describe the next stage of the research, which aimed to develop the scoring algorithm for the tool by determining the relevance and importance of each item included in the questionnaire, using a mixed methods approach.


    Methods

    Scoring Algorithm Development

    The draft scoring algorithm was developed using two approaches: quantitative analysis of real-world data and qualitative analysis based on physician interviews and ranking and weighting exercises (Figure 1). Quantitative methods involved retrospective analysis on data from a global cross-sectional study that collected information from physicians (neurologists) and their consulting MS patients on demographics, clinical history, current symptomatology, treatment history, and quality of life [14]. The study was run without set hypothesis before data collection but involved a large number of MS patients (n=3294) in a real-world setting, across countries, reflecting clinical practice and physician views. In the previous study, univariate analysis was conducted on variables included in the observational study [14]. Multivariate regression analysis was used in this study to determine variables associated with being early relapsing-remitting multiple sclerosis (RRMS) or early SPMS. In an iterative approach, these findings were used alongside qualitative research to inform the development of the draft tool content. The development and content of the draft tool (in the form of a questionnaire) have been described in detail previously by Ziemssen et al [14].

    Figure 1. Overview of the development of the scoring algorithm. SPMS: secondary progressive multiple sclerosis.
    View this figure

    Assigning Rank and Weights

    Physician ranking and weighting exercises were conducted as part of a qualitative interview. Eight physicians in Germany (n=4) and the United States (n=4), all neurologists, were recruited into this study by specialist recruitment agencies. Physicians were required to meet prespecified eligibility criteria (Multimedia Appendix 1). Each physician participated in a 45-min, face-to-face, semistructured, qualitative interview, conducted by a trained interviewer. First, physicians were presented with the list of variables included in the tool (Multimedia Appendix 2) and were asked to rank them in order of how strongly they contribute to SPMS progression. Then, physicians were asked to provide a “weight” for each variable by dividing 100 plastic tokens among the variables to indicate the contribution each variable should have to make up the total percentage score. Throughout the tasks, physicians were encouraged to “think aloud” and provide a rationale for the decisions that they made. Following completion of each task, physicians were asked to further explain their rankings or weightings or to clarify any decisions that they had not already commented on. In addition, physicians were asked to comment on the ease of completion of the tasks and to report if any important variables were missing. Mean and range weighting and ranking positions were produced for each variable.

    All interviews were audio-recorded and transcribed verbatim. Physicians’ rationales for ranking and weighting choices were analyzed using thematic analysis on Atlas.ti software [15]. Furthermore, the level of agreement between physicians for the ranking of variables was investigated at the individual country level (Germany and the United States) and for all physicians combined. Kendall coefficient of concordance was used to assess the agreement between the ranked concepts (from most important to least important). The test statistic, Kendall W, is calculated between 0 and 1, where 0 indicates no agreement between raters and 1 indicates complete agreement.

    Variables were categorized by researchers as high, moderate, or low in importance, based on the review of the findings from quantitative regression analysis, the ranking and weighting exercise, and the qualitative physicians’ rationale for the ranks and weights. A scoring algorithm was then developed to produce a total score for the draft tool.


    Results

    Regression Analysis

    A total of 11 physician-reported variables and nine patient-reported variables were identified for inclusion in multiple logistic regression analyses. Age (odds ratio [OR] 1.04; P<.001), MS disease activity (OR 1.68; P<.05), and Expanded Disability Status Scale (EDSS) score (OR 1.79; P<.001) were identified as the most significant physician-reported predictors of progression to SPMS (Figure 2). Patient age (OR 1.05; P<.001), mobility (OR 4.46; P<.001), and self-care (OR 2.39; P<.001) were identified as the strongest patient-reported predictors of progression to SPMS (Figure 3).

    Figure 2. Multivariate regression analysis: variables that are predictors of progression to secondary progressive multiple sclerosis. Disease activity is Physician-reported multiple sclerosis disease activity based on the physician’s overall perception of the patient’s disease activity, ranging from “no activity to high activity” (no specific definition of disease activity was provided to the physicians); an odds ratio >1 implies a higher risk of secondary progressive multiple sclerosis; the blue box highlights the significant predictors. EDSS: Expanded Disability Status Scale; MRI: magnetic resonance imaging; MS: multiple sclerosis; PRF: patient record form; SPMS: secondary progressive multiple sclerosis; T2: transverse relaxation time. Black dots indicate odds ratio (point estimate); black line indicates the 95% confidence interval.
    View this figure
    Figure 3. Multivariate regression analysis: patient self-completion form variable. An odds ratio >1 implies a higher risk of secondary progressive multiple sclerosis; the blue box highlights the significant predictors. PSC: patient self-completion. Black dots indicate odds ratio (point estimate); black line indicates the 95% confidence interval.
    View this figure

    Qualitative Interviews

    Demographics

    Physicians had a range of demographic characteristics and clinical experience. The sample consisted of 5 male and 3 female physicians, all of whom were neurologists. The mean age of the sample was 50.2 years (range 38-69). US physicians had been in their role for an average of 20 years (range 7-41), whereas German physicians had been in their role for an average of 8 years (range 3-21). The physicians were employed in a range of settings, including private practice (5/8), hospital-based care (2/8), and academia (2/8). On average, physicians saw 18 RRMS patients and 3 SPMS patients per week; German physicians saw more RRMS and SPMS patients per week than the US physicians. On average, physicians estimated that 30.6% (range 7%-70%) of their workload was dedicated to patients with MS.

    Ranking and Weighting

    The average ranking and weighting was calculated for each variable, and the top 10 ranked and weighted variables were identified. Findings from physician-completed ranking and weighting exercises were consistent in that 7 of the top 10 variables were present in both the ranked and weighted list. The top 10 variables included improvement, stability, or worsening of symptoms; intermittence or persistence of symptoms; ambulatory symptoms; cognitive symptoms; EDSS score; mobility; and presence or absence of relapse (Table 1). Variables that were in the top 10 for both the ranking and weighting exercises are italicized. Lower ranking indicates greater importance. Higher weighting indicates greater importance.

    Table 1. Top 10 ranked and weighted variables.
    View this table
    Categorizing Variables

    On the basis of the review of the findings from quantitative regression analysis, the ranking and weighting exercise, and the qualitative physicians’ rationale for rankings and weightings, eight variables were categorized by researchers as highly important in identifying progression to SPMS. These included variables describing the nature of the symptoms (intermittent vs persistent, stable vs worsening, and the absence or presence of relapses) and the presence of ambulatory, mobility, and cognitive symptoms, in addition to the EDSS score and time since diagnosis. Physicians explained that the variables rated as high importance were often indicators of progression to SPMS (Figure 4).

    Eight variables were categorized as moderately important indicators of progression to SPMS, as determined by the qualitative findings and physician’ rankings and weightings. Moderately important variables included those relating to the characteristics of relapse (recovery from the most recent relapse, number of relapses in the past 6 months, and symptoms during relapse), the presence of specific symptoms (motor, coordination and balance, and speech), an objective clinical measure of progression (signs of new activity based on MRI scans), and the impact on daily activities.

    Physicians explained that variables of moderate importance could be early signs of progression to SPMS but were not specific enough to be considered as highly important indicators (Figure 5).

    A total of 10 variables were categorized to be low indicators of progression to SPMS, as determined by physician rankings and weightings. These included fatigue, visual symptoms, bladder and bowel symptoms, pain, specific impacts (hobbies and leisure time, self-care, and work), and whether an MRI had been performed. Physicians explained that variables of low importance were subjective, general symptoms of MS, not relevant enough to MS and too unspecific for the progression to SPMS (Figure 6).

    The majority of physicians reported that they found the task challenging, given the complex nature of identifying progression to SPMS. One physician suggested including medication history, and another physician suggested removing ambulatory symptoms as it is similar to impact on mobility.

    Across all 8 physicians, the level of concordance was 0.278 (P<.001), indicating a low to moderate, but statistically significant, level of agreement. Physicians demonstrated slightly greater concordance within countries (United States: 0.42, P=.02; Germany: 0.385, P=.04; Table 2).

    Figure 4. Variables of high importance in progression to secondary progressive multiple sclerosis. Ranking out of 26 variables included. Lower ranking indicates greater importance. EDSS: Expanded Disability Status Scale; RRMS: relapsing-remitting multiple sclerosis; SPMS: secondary progressive multiple sclerosis.
    View this figure
    Figure 5. Variables of moderate importance in progression to secondary progressive multiple sclerosis. Ranking out of 26 variables included. Lower ranking indicates greater importance. MRI: magnetic resonance imaging; SPMS: secondary progressive multiple sclerosis.
    View this figure
    Figure 6. Variables of low importance in progression to secondary progressive multiple sclerosis. Ranking out of 26 variables included. Lower ranking indicates greater importance. SPMS: secondary progressive multiple sclerosis.
    View this figure
    Table 2. Regional concordance across physicians.
    View this table
    Scoring Algorithm

    On the basis of the results from the review of previous findings outlined by Ziemssen et al [14], regression analysis of observational study data, physician ranking and weighting, and the associated rationales, questions were weighted as follows:

    • 3 for variables that were found to be important
    • 2 for variables that were found to be moderately important
    • 1 for variables that were found to be less important.

    These weightings were integrated accordingly in the scoring algorithm. Absence of relapse; presence of motor, ambulatory, and cognitive symptoms; and the persistent worsening of any symptom were assigned the highest weights in the scoring algorithm.

    Question scores were multiplied by question weight to produce a total score for that question. The weightings and maximum score for each section are shown in Multimedia Appendix 3. The standardized total score was calculated by summing the score for each section and reweighting to a score divided by 100.


    Discussion

    Overview

    Disability progression is known to be a continuous process that starts very early in the disease course, identifiable even at Disability Status Scale score of 2 [16]. This is also evident from the similar rates of brain atrophy seen at the earliest vs later stages of MS [17]. In addition, cognitive impairment is also seen early in the disease course, including patients with clinically isolated syndrome [18]. Therefore, it is important to identify the signs of progression early, as the timing will determine the extent of therapeutic benefits and affect long-term outcome [19]. Currently, no established tools are available for use in routine clinical practice to support real-time “systematic” comprehensive assessment to help assess the subtle signs of progression [20]. During the stage 1 of our study, physicians confirmed the unmet need for such a tool in routine clinical practice and highlighted that a digital tool generating a score or a graphical output would be preferred and useful for clinical practice [14].

    Principal Findings

    The quantitative and qualitative approaches employed in this study informed categorization of the variables in the draft questionnaire as of “high,” “moderate,” and “low” importance. As expected, differences between the categories were not pronounced, but it is noteworthy that the variables rated as highly important were consistent and presented substantial overlap, thus providing confidence in the categorization. In line with previous studies, ambulation, mobility, and EDSS score were identified, not unsurprisingly, as the most “obvious” parameters associated with progression across all approaches. Interestingly, cognition emerged as an additional highly relevant symptom associated with progression. This is consistent with the previous reports showing that cognition is impaired very early in the disease course, even before physical disability might be obvious. The cognitive impairment affects multiple functionalities and can negatively impact patients’ lives. In addition, cognition has been reported to be predictive of disease evolution [19,21], whereas cognitive reserve can be a buffer to disease progression, reflecting the ability to compensate for progressive injury and as a marker of neuronal network efficiency [22-24].

    A 10-year follow-up study in patients with RRMS reports that patients with cognitive impairment are at a higher risk of reaching important milestone EDSS compared with cognitively preserved patients, and better cognitive performance at baseline was significantly predictive of lower SPMS conversion rates [25]. However, the ranking and weighting of variables by experienced neurologists clearly identified and confirmed the nature of the symptoms (eg, persistent worsening of any symptom) as the most important indicator of progression in MS, even more than a specific symptom itself, similar to the previous qualitative assessments with both physicians and patients [14].

    Existing research into predictors of SPMS has been primarily quantitative, based on single-center or large-scale observational cohort studies [11,12]. Hence, variables identified as significant predictors are those typically based on objective, clinical observations collected as part of those specific electronic medical records applied and accessible in those registries [10-12]. Although the global cross-sectional study described in this paper involved a large number of MS patients in a real-world setting and reflected clinical practice and physician views, specific limitations were identified. Namely, more frequently consulting patients were more likely to participate, physicians were included only if they saw a minimum number of patients and were willing to take part, data accuracy relied on the reporting accuracy of the physician, and analyses were limited to the variables and information collected in the cross-sectional study. Furthermore, regression analysis, when using cross-sectional data, cannot prove a causal relationship but will be able to show an association between the outcome and study group that is independent of confounding factors.

    Our study overcomes some of the limitations identified from these earlier studies, in that a more comprehensive approach was taken to identify the variables, also considering the descriptive and qualitative patient data assessed in daily practice and further ascertaining the importance of a particular variable for progression using a mixed methods approach. This enabled each variable to be classified by the level of contribution to progression thereby characterizing a sensitive algorithm that provides a score indicating the likelihood of progression for easy adoption in routine clinical practice. More importantly, none of the earlier studies evaluate progression at the current moment with such accuracy; rather, they provide a risk or likelihood of progression in the next few years or in the future.

    Findings from the previous qualitative interviews with physicians showed a lack of consistency in the diagnosis and time taken to determine SPMS. The level of concordance in ranking and weighting among physicians in this study was low to moderate but statistically significant and with greater level concordance among physicians within countries (United States or Germany). The variation seen in this study confirms the lack of clear consensus and, hence, the unmet need for a universal standardized method, or tool, that supports the identification of patients at risk of progression. Despite this variation, the fact that there was a significant agreement between physicians on the importance of variables supports feasibility and the value of the data in developing an algorithm for the tool by identifying prevailing common concepts driving the physician to determine that the patient has progressed to SPMS.

    As we used a mixed methods approach, some of the variables included in the tool were not collected in the RWE study and, thus, were categorized solely based on the ranking and weighting exercise and qualitative insights complimenting the findings from the regression analysis. The sample size for the qualitative assessment might have affected the level of agreement, and eventually, a more accurate representation of the level of agreement may have been achieved with a larger sample as any outliers in this sample had a large impact on the overall concordance statistic. However, as between and within differences in determining SPMS were also identified in earlier work and the MS neurologists in this study were all well experienced, it is unlikely that the level of agreement would have been a lot stronger with a larger sample size. By the inclusion of different geographies, we tried to cover for some of the differences in the prevailing health care systems and approaches adopted for the overall management of the disease.

    Subsequent work confirmed the validity of the scoring algorithm derived from these analyses in a real-world setting and determined cutoffs to accurately differentiate between RRMS and SPMS patients with high specificity and sensitivity, in addition to evaluation of other measurement properties including interrater reliability [26]. The final validated MS Progression Discussion tool can be accessed on the Web [27].

    Conclusions

    This study confirms the need for a tool to support the early evaluation of signs of progression to SPMS. The novel and comprehensive approach to develop the draft scoring algorithm triangulates data obtained from ranking and weighting exercises, qualitative interviews, and a real-world observational study. Variables that go beyond the clinically most obvious impairment in lower limbs have been identified as relevant subtle or sensitive signs suggestive of progressive disease. and have been integrated in the algorithm. The tool might, therefore, contribute to a more comprehensive physician-patient interaction in evaluating a patient’s current disease status and level of progression. Future work will aim to validate this scoring algorithm longitudinally in a real-world setting and its suitability for longitudinal monitoring of disease symptoms and its impacts.

    Acknowledgments

    Funding support was provided by Novartis Pharma AG, Basel, Switzerland. Adelphi Values (funded by the research sponsor, Novartis) designed and conducted this research along with data collection and management. Both the sponsor and Adelphi Values were involved in data interpretation and in the preparation, review, and approval of the study report as well as the publication. The authors would like to gratefully acknowledge all the participating physicians for their contribution and insights on the draft tool. The authors also acknowledge Sivaram Vedantam of Novartis Healthcare Pvt Ltd for medical writing support, which included literature search, drafting of article, and revising the article as per author comments, and Uma Kundu of Novartis Healthcare Pvt Ltd for scientific editorial review, preparation of submission, and journal revisions and resubmission.

    Conflicts of Interest

    CT, SB, EJ, and JP are full-time employees of Adelphi Values Ltd, United Kingdom, a health care research consultancy. BB was an employee of Adelphi Values at the time of this study. DP, FD, and DT are employees of Novartis Pharma AG, Basel. TZ has nothing to disclose for the submitted work. He has received personal compensation for participating on advisory boards, trial steering committees, and data and safety monitoring committees, as well as for scientific talks and project support, from Almirall, Bayer, British American Tobacco, Biogen, Celgene, Sanofi Genzyme, Merck, Novartis, Roche, Vitaccess, and Teva, outside of the submitted work.

    Multimedia Appendix 1

    Physician eligibility criteria.

    DOCX File , 23 KB

    Multimedia Appendix 2

    Variables rated by physicians.

    DOCX File , 24 KB

    Multimedia Appendix 3

    Scoring algorithm—question weights and total scores.

    DOCX File , 26 KB

    References

    1. Scalfari A, Neuhaus A, Daumer M, Muraro PA, Ebers GC. Onset of secondary progressive phase and long-term evolution of multiple sclerosis. J Neurol Neurosurg Psychiatry 2014 Jan;85(1):67-75. [CrossRef] [Medline]
    2. Lublin FD. New multiple sclerosis phenotypic classification. Eur Neurol 2014;72(Suppl 1):1-5 [FREE Full text] [CrossRef] [Medline]
    3. Inojosa H, Proschmann U, Akgün K, Ziemssen T. A focus on secondary progressive multiple sclerosis (SPMS): challenges in diagnosis and definition. J Neurol 2019 Jul 30 [Epub ahead of print]. [CrossRef] [Medline]
    4. Katz Sand I, Krieger S, Farrell C, Miller AE. Diagnostic uncertainty during the transition to secondary progressive multiple sclerosis. Mult Scler 2014 Oct;20(12):1654-1657. [CrossRef] [Medline]
    5. Davies F, Wood F, Brain KE, Edwards M, Jones R, Wallbank R, et al. The transition to secondary progressive multiple sclerosis: an exploratory qualitative study of health professionals' experiences. Int J MS Care 2016;18(5):257-264 [FREE Full text] [CrossRef] [Medline]
    6. O'Loughlin E, Hourihan S, Chataway J, Playford ED, Riazi A. The experience of transitioning from relapsing remitting to secondary progressive multiple sclerosis: views of patients and health professionals. Disabil Rehabil 2017 Sep;39(18):1821-1828. [CrossRef] [Medline]
    7. Bischof A, Papinutto N, Zhang X, Rajesh A, Sacco S, Kirkish G, et al. Accelerated cord atrophy precedes conversion to secondary progressive disease in relapsing multiple sclerosis. Neurol 2019 Apr;92(15) [FREE Full text]
    8. Housley WJ, Pitt D, Hafler DA. Biomarkers in multiple sclerosis. Clin Immunol 2015 Nov;161(1):51-58. [CrossRef] [Medline]
    9. Lim CK, Bilgin A, Lovejoy DB, Tan V, Bustamante S, Taylor BV, et al. Kynurenine pathway metabolomics predicts and provides mechanistic insight into multiple sclerosis progression. Sci Rep 2017 Feb 3;7:41473 [FREE Full text] [CrossRef] [Medline]
    10. Lorscheider J, Buzzard K, Jokubaitis V, Spelman T, Havrdova E, Horakova D, MSBase Study Group. Defining secondary progressive multiple sclerosis. Brain 2016 Sep;139(Pt 9):2395-2405. [CrossRef] [Medline]
    11. Manouchehrinia A, Zhu F, Piani-Meier D, Lange M, Silva DG, Carruthers R, et al. Predicting risk of secondary progression in multiple sclerosis: A nomogram. Mult Scler 2019 Jul;25(8):1102-1112. [CrossRef] [Medline]
    12. Skoog B, Tedeholm H, Runmarker B, Odén A, Andersen O. Continuous prediction of secondary progression in the individual course of multiple sclerosis. Mult Scler Relat Disord 2014 Sep;3(5):584-592. [CrossRef] [Medline]
    13. Tremlett H, Zhao Y, Devonshire V. Natural history of secondary-progressive multiple sclerosis. Mult Scler 2008 Apr;14(3):314-324. [CrossRef] [Medline]
    14. Ziemssen T, Tolley C, Bennett B, Kilgariff S, Jones E, Pike J, et al. A mixed methods approach towards understanding key disease characteristics associated with the progression from RRMS to SPMS: Physicians' and patients' views. Mult Scler Relat Disord 2019 Nov 18;38:101861 [FREE Full text] [CrossRef] [Medline]
    15. ATLAS.ti: The Qualitative Data Analysis & Research Software.   URL: https://atlasti.com/ [accessed 2020-01-20]
    16. Kremenchutzky M, Rice GP, Baskerville J, Wingerchuk DM, Ebers GC. The natural history of multiple sclerosis: a geographically based study 9: observations on the progressive phase of the disease. Brain 2006 Mar;129(Pt 3):584-594. [CrossRef] [Medline]
    17. de Stefano N, Giorgio A, Battaglini M, Rovaris M, Sormani MP, Barkhof F, et al. Assessing brain atrophy rates in a large population of untreated multiple sclerosis subtypes. Neurology 2010 Jun 8;74(23):1868-1876. [CrossRef] [Medline]
    18. Anhoque CF, Domingues SC, Teixeira AL, Domingues RB. Cognitive impairment in clinically isolated syndrome: A systematic review. Dement Neuropsychol 2010;4(2):86-90 [FREE Full text] [CrossRef] [Medline]
    19. Cerqueira JJ, Compston DA, Geraldes R, Rosa MM, Schmierer K, Thompson A, et al. Time matters in multiple sclerosis: can early treatment and long-term follow-up ensure everyone benefits from the latest advances in multiple sclerosis? J Neurol Neurosurg Psychiatry 2018 Aug;89(8):844-850 [FREE Full text] [CrossRef] [Medline]
    20. Inojosa H, Schriefer D, Ziemssen T. Clinical outcome measures in multiple sclerosis: A review. Autoimmun Rev 2020 Mar 12:102512. [CrossRef] [Medline]
    21. Lovera J, Kovner B. Cognitive impairment in multiple sclerosis. Curr Neurol Neurosci Rep 2012 Oct;12(5):618-627 [FREE Full text] [CrossRef] [Medline]
    22. Amato MP, Razzolini L, Goretti B, Stromillo ML, Rossi F, Giorgio A, et al. Cognitive reserve and cortical atrophy in multiple sclerosis: a longitudinal study. Neurology 2013 May 7;80(19):1728-1733. [CrossRef] [Medline]
    23. Schwartz CE, Quaranto BR, Healy BC, Benedict RH, Vollmer TL. Cognitive reserve and symptom experience in multiple sclerosis: a buffer to disability progression over time? Arch Phys Med Rehabil 2013 Oct;94(10):1971-1981. [CrossRef] [Medline]
    24. Ziemssen T, Derfuss T, de Stefano N, Giovannoni G, Palavra F, Tomic D, et al. Optimizing treatment success in multiple sclerosis. J Neurol 2016 Jun;263(6):1053-1065 [FREE Full text] [CrossRef] [Medline]
    25. Moccia M, Lanzillo R, Palladino R, Chang KC, Costabile T, Russo C, et al. Cognitive impairment at diagnosis predicts 10-year multiple sclerosis progression. Mult Scler 2016 Apr;22(5):659-667. [CrossRef] [Medline]
    26. Ziemssen T, Piani-Meier D, Bennett B, Johnson C, Tinsley K, Trigg A, et al. A physician-completed digital tool for evaluating disease progression (multiple sclerosis progression discussion tool): validation study. J Med Internet Res 2020 Feb 12;22(2):e16932 [FREE Full text] [CrossRef] [Medline]
    27. Neuro-Compass. MSProDiscuss: MS Progression Discussion Tool   URL: https://www.neuro-compass.education/en-gb/msprodiscuss/ [accessed 2020-01-20]


    Abbreviations

    EDSS: Expanded Disability Status Scale
    MRI: magnetic resonance imaging
    MS: multiple sclerosis
    OR: odds ratio
    RRMS: relapsing-remitting multiple sclerosis
    RWE: real-world evidence
    SPMS: secondary progressive multiple sclerosis


    Edited by G Eysenbach; submitted 23.12.19; peer-reviewed by E D'Amico, M Moccia; comments to author 21.01.20; revised version received 14.02.20; accepted 22.02.20; published 14.04.20

    ©Chloe Tolley, Daniela Piani-Meier, Sarah Bentley, Bryan Bennett, Eddie Jones, James Pike, Frank Dahlke, Davorka Tomic, Tjalf Ziemssen. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 14.04.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.