Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 30.03.20 in Vol 8, No 3 (2020): March

This paper is in the following e-collection/theme issue:

    Original Paper

    Towards an Adoption Framework for Patient Access to Electronic Health Records: Systematic Literature Mapping Study

    1Department of Medical Informatics, Amsterdam Public Health, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands

    2Department of Research & Development, ChipSoft, Amsterdam, Netherlands

    Corresponding Author:

    Hugo J T van Mens, MSc

    Department of Medical Informatics

    Amsterdam Public Health

    Amsterdam University Medical Centers, University of Amsterdam

    Meibergdreef 9, PO Box 22700

    Amsterdam, 1100 DE


    Phone: 31 20 566 51 88



    Background: Patient access to electronic health records (EHRs) is associated with increased patient engagement and health care quality outcomes. However, the adoption of patient portals and personal health records (PHRs) that facilitate this access is impeded by barriers. The Clinical Adoption Framework (CAF) has been developed to analyze EHR adoption, but this framework does not consider the patient as an end-user.

    Objective: We aim to extend the scope of the CAF to patient access to EHRs, develop guidance documentation for the application of the CAF, and assess the interrater reliability.

    Methods: We systematically reviewed existing systematic reviews on patients' access to EHRs and PHRs. Results of each review were mapped to one of the 43 CAF categories. Categories were iteratively adapted when needed. We measured the interrater reliability with Cohen’s unweighted kappa and statistics regarding the agreement among reviewers on mapping quotes of the reviews to different CAF categories.

    Results: We further defined the framework’s inclusion and exclusion criteria for 33 of the 43 CAF categories and achieved a moderate agreement among the raters, which varied between categories.

    Conclusions: In the reviews, categories about people, organization, system quality, system use, and the net benefits of system use were addressed more often than those about international and regional information and communication technology infrastructures, standards, politics, incentive programs, and social trends. Categories that were addressed less might have been underdefined in this study. The guidance documentation we developed can be applied to systematic literature reviews and implementation studies, patient and informal caregiver access to EHRs, and the adoption of PHRs.

    JMIR Med Inform 2020;8(3):e15150




    Patient access to electronic health records (EHRs) is becoming increasingly common and is even a legal right in many countries. EHRs have been associated with increased patient engagement and improved health care quality outcomes [1-8]. However, there are also barriers to patients’ access to EHRs. For example, some patients have difficulties logging in to patient portals and personal health records (PHRs), which facilitate access, due to complicated security procedures [1-8]. A framework is needed to assess the determinants and outcomes of PHR and EHR adoption that facilitates this access. This framework should consider patients and informal caregivers as users rather than health care providers alone. This framework would enable the comparison and aggregation of evidence, and provide an overview of any important factors involved, which can then be used as a guide in implementations and health care policies as well as to address the gaps in knowledge.

    “The Clinical Adoption Framework (CAF) is a general evaluation framework to assess the success of health information system (HIS) adoption in healthcare organisations” [9,10]. PHRs and EHRs are types of HISs, and thus this framework is also applicable to them. “As shown in Figure 1, it addresses the micro level, which encompasses the dimensions of quality, use and net benefits of the HIS; the meso level, consisting of the dimensions people, organisation and implementation; and the macro level, incorporating the dimensions healthcare standards, legislation, policy and governance, funding and incentives, and societal, political and economic trends” [10]. Within each dimension, several categories were distinguished, for example “01. Functionality”, “02. Performance”, and “03. Security” are categories of the dimension “HIS quality” at the micro level. “It is hence an integrated framework that covers a wide range of aspects involved in HIS adoption” [10]. The CAF was developed and validated through consultation with health information technology professionals, comparisons with other survey instruments, and a meta-review of 50 systematic reviews on HIS implementation [11]. Categories, dimensions, and levels of the CAF were originally described by Lau, Price, and Kashevjee [9]. Throughout the categories, dimensions, and levels there are feedback loops, which are indicated by the arrows in Figure 1, that resembles the interplay between the factors and nondeterministic characteristics of HIS adoption and the outcomes of HIS use [9,11]. The CAF was applied in over 30 studies [12-18].

    Figure 1. Clinical Adoption Framework with levels, dimensions, and categories. HIS: health information system. Originally published in [10, 18].
    View this figure

    The CAF is a complex framework consisting of 43 categories that belong to 15 dimensions (illustrated as small boxes in Figure 1), which are further separated into the 3 previously mentioned micro, meso, and macro levels [9]. The CAF was considered difficult to apply as there was no guidance documentation with explicit descriptions and rules regarding its use [11]. Consequently, studies [15-18] that have applied the CAF differed in their interpretations and applications. Furthermore, HIS adoption increasingly involves sharing medical data with patients and informal caregivers. Therefore, patients and caregivers should also be considered when understanding successful HIS implementation, because they might value different factors than health care providers. This patient and caregiver perspective was not explicitly taken into account during the development of the CAF.

    The primary objective of this study was to extend the CAF to make it useful for evaluating patients' access to EHRs and the adoption of PHRs. The second objective was to improve the consistent application of the CAF in literature and implementation studies. For this purpose, we aimed to assess the interrater reliability of applying the framework, discuss which areas of the CAF could be improved, and develop guidance documentation.


    We systematically reviewed existing systematic review papers on determinants and outcomes of patients’ access to their personal health data. Results from each review paper were mapped to categories in the CAF, which was adapted when needed to reach consensus. The protocol for this review study was developed using the first 6 review papers [19-24], which were the most recent publications at the beginning of this review study. We used 13 subsequent review papers [25-37] in this study to refine the CAF and to assess the interrater reliability. The review protocol was registered at PROSPERO under CRD42018084542 [38]. We then reported the results of adapting the CAF, including its reliability, to make it suitable for an evaluation of the adoption of PHRs and patients’ access to EHRs. The results of the review study on the determinants and outcomes of patients’ access to medical records were reported separately [10].

    To improve the CAF and its definitions, one reviewer (HM) extracted quotes from the literature that described determinants and outcomes for the adoption of EHRs and PHRs, and another reviewer (RD) verified these extracted quotes. The two reviewers independently mapped the extracted quotes. The interrater reliability for the agreement on the mapping was calculated with Cohen’s unweighted kappa [39,40]. Each quote was mapped to two CAF categories: one for the determinant of the quote and the other for the outcome. Within each category, the quotes were classified into metrics by thematic analysis, as illustrated in Figure 2. The metrics and categorizations were iteratively revised to ensure consistent and meaningful categories for summarizing results, which was similar to the process described in Bassi, Lau, and Lesperance [15]. The mapping to two categories is visualized in Figure 2. For example, in the quote “Online record access and service users tended to be slightly older (t-test, P<0.001)” [31], the determinant metric could be “Age” and the outcome metric “Adoption”. Age would be classified as “21. Personal characteristics”, under the dimension “People” at the meso level, while adoption would be classified as “07. Use behaviour/pattern”, under the dimension “Use” at the micro level. For the sake of the review, we added the category “44. Other” to denote when a quote could not be classified using the CAF.

    Figure 2. Example of how a quote is mapped to a determinant metric and outcome metric. CAF: Clinical Adoption Framework.
    View this figure

    The results of this mapping and the differences in quote interpretation and CAF categorization were discussed among the two reviewers to achieve consensus. When necessary to achieve consensus, the definitions of the CAF were adapted and extended with inclusion and exclusion criteria to make them clearer. We presented the number of definitions for categories that were introduced, extended, or unchanged in each level. For agreements and disagreements between reviewers on mapping quotes to categories, we calculated the number of times each unique combination was agreed or disagreed upon (ie, number of times there was agreement on one certain category or disagreement between two specific categories). We counted the number of quotes that were classified into each category by a reviewer as well as how many quotes could not be mapped to the CAF. The level of agreement between reviewers on mapping quotes to each category indicated how ambiguous or well-defined the category was. This process resulted in defined categories of the CAF with inclusion and exclusion criteria and a list of metrics that we distinguished. Statistical analysis was carried out in R version 3.5.0 (R Foundation for Statistical Computing, Vienna, Austria) with RStudio 1.1.453 (RStudio Inc, Boston, MA). The R script can be found in Multimedia Appendix 1.


    In this section, we first list the definitions that were unchanged, extended, or introduced. Second, we discuss the interrater reliability and the spread of mapping quotes to CAF categories.

    Adaptation of CAF Categories and Found Metrics

    Definitions were introduced to the CAF for the 19 micro level categories, because they were missing in the original publication of the CAF. For example, the category “01. Functionality” of the dimension “System quality” was defined with the inclusion criteria “Actual or missing features/functionalities of the HIS and their quality” and the exclusion criteria “If adoption or use of the HIS in general, without a particular functionality, then choose 07. Use behaviour/pattern”. Thus, the exclusion criteria were made explicit for when a quote must be classified in another category. For the 24 meso- and macro level categories, the definitions from Lau, Price, and Kashevjee [9] were used, either unchanged (9 categories) or extended (15 categories), to cover cases of patient and informal caregiver use and disambiguate the categories with refinements and exclusion criteria. For example, the definition for the category “21. Personal characteristics” of the dimension “People” was extended with “socio-economic status, ethnicity, computer skills, (health) literacy, health status” and “Behaviour”. These are factors that were found to be important for the adoption of a HIS by patients and caregivers, and were not included in the original CAF category definition. Table 1 shows the number of categories for each level and how many were introduced, extended, or unchanged. Table 2 shows the categories that were changed and provides an example for each level. The resulting definitions for disambiguation in each category are listed in Table A in Multimedia Appendix 2. The metrics of each category can be found in Table B in Multimedia Appendix 2.

    Table 1. The number of categories with introduced, extended, and unchanged definitions per level.
    View this table
    Table 2. Categories where inclusion and exclusion criteria were added.
    View this table

    Interrater Reliability and Spread

    From the 13 reviews [25-37], we extracted 624 quotes. Each of the 624 quotes were mapped twice (ie, to a determinant and an outcome category) resulting in 1248 mappings. We achieved a percentage agreement of 67.0% (418) and a kappa of 0.58 for the determinant category, and a percentage agreement of 62.5% (390) and a kappa of 0.55 for the outcome category. As shown in Table C and Table D in Multimedia Appendix 2, the three categories that were least ambiguous, based on their high agreement scores, were “16. Efficiency”, “21. Personal characteristics”, and “13. Patient safety”. In contrast, categories “09. Intention to use”, “04. Content”, and “30. Project” showed low agreement scores. Some disagreements between two categories occurred more often than others. For example, a feature relating to secure messaging or access to medical records was interpreted by one reviewer as “’01. Functionality” and the other reviewer as “’07. Use behaviour/pattern” 94 times. This happened for instance with the quote “Patients experienced easier communication and interactive discussion with their physician after reading the medical file” [37]. There was one quote that did not fit into any one of the categories: “Two articles proposed achieving data exchange by setting up (Regional) Health Information Exchanges that can standardize data and facilitate exchange among different organizations.” [24]. This result referred to infrastructure that exists outside of an organization to facilitate data exchange between organizations and would, therefore, fall into the macro level.


    Principal Findings

    The definitions of the CAF categories were extended to be applicable to patient access to EHRs and the adoption of PHRs. This was achieved by adding factors that were found in the reviewed literature on patient access to EHRs, but were not present in the CAF yet, as was illustrated in the example of “21. Personal characteristics” in the results section. In addition, we developed guidance documentation in the form of inclusion criteria, exclusion criteria, and a list of metrics found. The interrater reliability of the reviewers applying the adapted CAF was moderate. However, we found the CAF to be a highly suitable and comprehensive framework to address patients’ access to EHRs, as we could achieve consensus on the mappings through discussion, and almost all results could be categorized in the CAF. The original content for the definitions of the CAF were unchanged and only extended with additional inclusion and exclusion criteria for disambiguation and for the application to patients’ access to EHRs. The number of agreements and disagreements and percentage of agreements varied among the CAF categories, just like the number of quotes that were mapped to each category. Some categories were not found at all in the reviews, especially those on the macro level.

    Strengths and Weaknesses of the Study

    We showed how the CAF can be applied to studies evaluating patient access to EHRs and PHRs. Despite many publications on the application of the CAF, we are the first, to our knowledge, to provide measures on the interrater reliability. However, the unweighted Cohen’s kappa does not consider that categories actually reflect an order and results within each review are all correlated and come from the same study. Nonetheless, the moderate agreement indicates that the extended CAF is applicable in a consistent way. Because this study was a systematic review of systematic reviews, we have not investigated how to apply these results in primary implementation studies. The categories that were mapped to a lesser extent might have been underdefined, especially those at the macro level. It is possible that these categories may not have been reported in the literature, but also the literature may not have addressed the topics from those categories, or those categories could have been reported in other types of literature such as in policy, law, or gray literature, rather than scientific medical literature. Those categories with relatively high disagreement should also be further evaluated and redefined. Furthermore, the CAF could be used in studies to present their results in a more structured and standardized way. This will improve the ability to compare the results of different studies.

    Results in Relation to Other Studies

    The variability in the application of the CAF categories found in previous studies [15-18] can be explained by ambiguities that were addressed by the inclusion and exclusion criteria of this study. In addition, we found that mapping to a determinant and an outcome CAF category, instead of only one, decreased some of the ambiguity. Only one result, concerning regional information exchange, could not be mapped in the original CAF. This shows that overall the CAF is sufficiently comprehensive. However, we believe that the infrastructure that is available in the environment of an organization forms a missing category in the framework. This category could be introduced in the framework at the macro level to incorporate regional information and communication technologies (ICT) infrastructure, which might be more advanced in some regions than in others.

    Implications of the Study

    This adapted framework can be used in other reviews and in implementation studies of HISs, especially when the HIS has patients and informal caregivers as users. The definitions and metrics provided will still be of value to implementation studies by pointing out several aspects and metrics that have to be considered when carrying out HIS implementations. Furthermore, the results of this study fulfill part of the need for more guidance documentation when applying the CAF [11]. Our definitions with inclusion and exclusion criteria as well as the metrics found may contribute to a more consistent application of the framework. We recommend addressing specific relationships between determinants and outcomes using this framework, as we did by mapping quotes from the literature to two CAF categories.


    The scope of the CAF was extended to the adoption of PHRs, in addition to EHRs, by health care providers, patients, and informal caregivers. Further definitions and inclusion and exclusion criteria disambiguate and guide the application of each category. We found moderate interrater reliability in applying the framework and variance among the categories in the framework. Future research should address the application of the CAF in primary implementation studies and studies focusing on macro level topics such as international and regional ICT infrastructures, standards, politics, incentive programs, and social trends.

    Authors' Contributions

    All authors were involved in the systematic review. RC, NK, and RN reviewed two of the first three reviews and contributed to the revision of the CAF definitions. RD and HM reviewed all reviews and continued revising the CAF iteratively. HM prepared the text and carried out the analyses with feedback from RC, NK, and RN. RC, NK, RN, and RD critically reviewed the manuscript. All authors agreed to submit the manuscript.

    Conflicts of Interest

    RN, RD, and HM work at ChipSoft B V. RN is manager of the Department of Research and Development (R&D). RD was an intern at Datawarehouse, Department of R&D. HM is software developer of Care Portal, Department of R&D. ChipSoft is a software vendor that develops the health information system HiX. HiX Care Portal is a web application that includes a patient portal. RC and NK declare that they have no known conflicts of interest and are not affiliated with any vendor, patient portal, or personal health record.

    Multimedia Appendix 1

    R script with syntax for analysis.

    TXT File , 14 KB

    Multimedia Appendix 2

    Supplementary tables.

    DOCX File , 61 KB


    1. Ross SE, Lin C. The effects of promoting patient access to medical records: a review. J Am Med Inform Assoc 2003;10(2):129-138 [FREE Full text] [CrossRef] [Medline]
    2. Ammenwerth E, Schnell-Inderst P, Hoerbst A. The impact of electronic patient portals on patient care: a systematic review of controlled trials. J Med Internet Res 2012 Nov 26;14(6):e162 [FREE Full text] [CrossRef] [Medline]
    3. Prey JE, Polubriaginof F, Kuperman GJ, Tiase V, Collins SA, Vawdrey DK. International perspectives on sharing clinical data with patients. Int J Med Inform 2016 Feb;86:135-141. [CrossRef] [Medline]
    4. Beard L, Schein R, Morra D, Wilson K, Keelan J. The challenges in making electronic health records accessible to patients. J Am Med Inform Assoc 2012;19(1):116-120 [FREE Full text] [CrossRef] [Medline]
    5. Saparova D. Motivating, influencing, and persuading patients through personal health records: a scoping review. Perspect Health Inf Manag 2012;9:1f [FREE Full text] [Medline]
    6. Goldzweig C, Orshansky G, Paige N, Towfigh AA, Haggstrom D, Miake-Lye I, et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013 Nov 19;159(10):677-687. [CrossRef] [Medline]
    7. Farrelly S, Brown GE, Flach C, Barley E, Laugharne R, Henderson C. User-held personalised information for routine care of people with severe mental illness. Cochrane Database Syst Rev 2013 Oct 05(10):CD001711. [CrossRef] [Medline]
    8. Goldzweig CL, Orshansky G, Paige NM, Towfigh AA, Haggstrom DA, Miake-Lye I, et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013 Nov 19;159(10):677-687. [CrossRef] [Medline]
    9. Lau F, Price M, Keshavjee K. From benefits evaluation to clinical adoption: making sense of health information system success in Canada. Healthc Q 2011;14(1):39-45. [CrossRef] [Medline]
    10. van Mens HJT, Duijm RD, Nienhuis R, de Keizer NF, Cornet R. Determinants and outcomes of patient access to medical records: Systematic review of systematic reviews. Int J Med Inform 2019 Sep;129:226-233. [CrossRef] [Medline]
    11. Craven CK, Doebbeling B, Furniss D, Holden RJ, Lau F, Novak LL. Evidence-based health informatics frameworks for applied use. Stud Health Technol Inform 2016;222:77-89. [Medline]
    12. Lau F, Kuziemsky C. Handbook of eHealth evaluation: an evidence-based approach. Victoria: University of Victoria; 2016.
    13. Ahmadi H, Darvishi M, Nilashi M, Almaee A, Ibrahim O, Zolghadri A. Evaluating the critical factors for electronic medical record adoption using fuzzy approaches. International Journal of Innovation and Scientific Research 2014;9(2):268-284.
    14. Ahmadi H, Salahshour Rad M, Nilashi M, Ibrahim O, Almaee A. Ranking the Micro Level Critical Factors of Electronic Medical Records Adoption Using Topsis Method. HIIJ 2013 Nov 30;2(4):19-32. [CrossRef]
    15. Bassi J, Lau F, Lesperance M. Perceived impact of electronic medical records in physician office practices: a review of survey-based research. Interact J Med Res 2012 Jul 28;1(2):e3 [FREE Full text] [CrossRef] [Medline]
    16. Gagnon M, Payne-Gagnon JA, Sicotte C, Langué-Dubé JA, Motulsky A. Connecting primary care clinics and community pharmacies through a nationwide electronic prescribing network: A qualitative study. J Innov Health Inform 2015 Oct 05;22(3):359-367 [FREE Full text] [CrossRef] [Medline]
    17. Lau F, Partridge C, Randhawa G, Bowen M. Applying the clinical adoption framework to evaluate the impact of an ambulatory electronic medical record. Stud Health Technol Inform 2013;183:15-20. [Medline]
    18. Lau F, Price M, Boyd J, Partridge C, Bell H, Raworth R. Impact of electronic medical record on physician practice in office settings: a systematic review. BMC Med Inform Decis Mak 2012 Feb 24;12:10 [FREE Full text] [CrossRef] [Medline]
    19. Rathert C, Mittler JN, Banerjee S, McDaniel J. Patient-centered communication in the era of electronic health records: What does the evidence say? Patient Educ Couns 2017 Jan;100(1):50-64. [CrossRef] [Medline]
    20. Thompson MJ, Reilly JD, Valdez RS. Work system barriers to patient, provider, and caregiver use of personal health records: A systematic review. Appl Ergon 2016 May;54:218-242. [CrossRef] [Medline]
    21. Turner K, Klaman SL, Shea CM. Personal health records for people living with HIV: a review. AIDS Care 2016 Sep;28(9):1181-1187. [CrossRef] [Medline]
    22. Bush RA, Connelly CD, Fuller M, Pérez A. Implementation of the Integrated Electronic Patient Portal in the Pediatric Population: A Systematic Review. Telemed J E Health 2016 Feb;22(2):144-152 [FREE Full text] [CrossRef] [Medline]
    23. Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res 2015 Jun 23;17(6):e148 [FREE Full text] [CrossRef] [Medline]
    24. Otte-Trojel T, de Bont A, Rundall TG, van de Klundert J. What do we know about developing patient portals? a systematic literature review. J Am Med Inform Assoc 2016 Apr;23(e1):e162-e168 [FREE Full text] [CrossRef] [Medline]
    25. Coughlin SS, Prochaska JJ, Williams LB, Besenyi GM, Heboyan V, Goggans DS, et al. Patient web portals, disease management, and primary prevention. Risk Manag Healthc Policy 2017;10:33-40 [FREE Full text] [CrossRef] [Medline]
    26. Hemsley B, Rollo M, Georgiou A, Balandin S, Hill S. The health literacy demands of electronic personal health records (e-PHRs): An integrative review to inform future inclusive research. Patient Educ Couns 2018 Jan;101(1):2-15. [CrossRef] [Medline]
    27. Kelly MM, Coller RJ, Hoonakker PL. Inpatient portals for hospitalized patients and caregivers: a systematic review. J Hosp Med 2018 Jun 01;13(6):405-412 [FREE Full text] [CrossRef] [Medline]
    28. Kneale L, Demiris G. Lack of diversity in personal health record evaluations with older adult participants: a systematic review of literature. J Innov Health Inform 2017 Jan 15;23(4):881 [FREE Full text] [CrossRef] [Medline]
    29. Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res 2015 Feb 20;17(2):e40 [FREE Full text] [CrossRef] [Medline]
    30. Kruse CS, Bolton K, Freriks G. The effect of patient portals on quality outcomes and its implications to meaningful use: a systematic review. J Med Internet Res 2015 Feb 10;17(2):e44 [FREE Full text] [CrossRef] [Medline]
    31. Mold F, de Lusignan S, Sheikh A, Majeed A, Wyatt JC, Quinn T, et al. Patients' online access to their electronic health records and linked online services: a systematic review in primary care. Br J Gen Pract 2015 Mar;65(632):e141-e151 [FREE Full text] [CrossRef] [Medline]
    32. Powell KR. Patient-perceived facilitators of and barriers to electronic portal use: a systematic review. Comput Inform Nurs 2017 Nov;35(11):565-573. [CrossRef] [Medline]
    33. Price M, Bellwood P, Kitson N, Davies I, Weber J, Lau F. Conditions potentially sensitive to a personal health record (PHR) intervention, a systematic review. BMC Med Inform Decis Mak 2015 Apr 18;15:32 [FREE Full text] [CrossRef] [Medline]
    34. Sakaguchi-Tang DK, Bosold AL, Choi YK, Turner AM. Patient portal use and experience among older adults: systematic review. JMIR Med Inform 2017 Oct 16;5(4):e38 [FREE Full text] [CrossRef] [Medline]
    35. Sartain SA, Stressing S, Prieto J. Patients' views on the effectiveness of patient-held records: a systematic review and thematic synthesis of qualitative studies. Health Expect 2015 Dec;18(6):2666-2677 [FREE Full text] [CrossRef] [Medline]
    36. Showell C. Barriers to the use of personal health records by patients: a structured review. PeerJ 2017;5:e3268 [FREE Full text] [CrossRef] [Medline]
    37. Vermeir P, Degroote S, Vandijck D, Van Tiggelen H, Peleman R, Verhaeghe R, et al. The patient perspective on the effects of medical record accessibility: a systematic review. Acta Clin Belg 2017 Jun;72(3):186-194. [CrossRef] [Medline]
    38. van Mens HJT, Nienhuis R, Duijm RD, Cornet R, de Keizer NF. National Institute for Health Research.: PROSPERO; 2018. An overview of determinants and outcomes of providing patients access to and control over their own medical data: a systematic review of reviews   URL:
    39. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 2016 Jul 02;20(1):37-46. [CrossRef]
    40. Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968 Oct;70(4):213-220. [CrossRef] [Medline]


    CAF: Clinical Adoption Framework
    EHR: electronic health record
    HIS: health information system
    ICT: information and communication technologies
    PHR: personal health record.

    Edited by G Eysenbach; submitted 24.06.19; peer-reviewed by F Lau, H Perry; comments to author 29.09.19; revised version received 30.09.19; accepted 15.12.19; published 30.03.20

    ©Hugo J T van Mens, Ruben D Duijm, Remko Nienhuis, Nicolette F de Keizer, Ronald Cornet. Originally published in JMIR Medical Informatics (, 30.03.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.