This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
With the widespread use of mobile technologies, mobile information systems have become crucial tools in health care operations. Although the appropriate use of mobile health (mHealth) may result in major advances in expanding health care coverage (increasing decision-making speeds, managing chronic conditions, and providing suitable health care in emergencies), previous studies have argued that current mHealth research does not adequately evaluate mHealth interventions, and it does not provide sufficient evidence regarding the effects on health.
The aim of this study was to facilitate the widespread use of mHealth systems; an accurate evaluation of the systems from the users’ perspective is essential after the implementation and use of the system in daily health care practices. This study extends the expectation-confirmation model by using characteristics of individuals, technology, and tasks to identify critical factors affecting mHealth continuance and performance from the perspective of health care professionals (HCPs).
A questionnaire survey was used to collect data from HCPs who were experienced in using mHealth systems of a Taiwanese teaching hospital. In total, 282 questionnaires were distributed, and 201 complete and valid questionnaires were returned, thus indicating a valid response rate of 71.3% (201/282). The collected data were analyzed using WarpPLS version 5.0 (ScriptWarp Systems).
The results revealed that mHealth continuance (
The identified critical factors influencing mHealth continuance and performance can be used as a useful assessment tool by hospitals that have implemented mHealth systems to facilitate the use and infusion of the systems. Furthermore, the results can help health care institutions that intend to introduce or develop mHealth applications to identify critical issues and effectively allocate limited resources to mHealth systems.
With the widespread use of mobile technologies, mobile information systems (ISs) have become crucial tools in health care operations. In recent times, smart health (sHealth) has become a critical strategy that is promoted by the government and medical industry; however, the successful implementation of sHealth depends on the development of mobile health (mHealth) [
In reality, health care professionals (HCPs) often require high-quality communication and information resources, including communication capabilities, hospital information systems (HISs), information resources, and clinical software applications, at the point of care to facilitate rapid decision making with a low error rate, improve the quality of data management and accessibility, and improve practice efficiency and knowledge [
Varshney [
Many studies have reported that when appropriately used, mHealth systems facilitate rapid decision making with low error rates, thereby improving the quality of data management and accessibility and improving practice efficiency and knowledge [
Bhattacherjee [
Some studies have emphasized examining the determinants of mHealth in the assimilation or integration stage, where the mHealth services or systems are stable and have been incorporated into routine practices [
The performance of mHealth ISs should be evaluated based on user satisfaction and the specific outcomes of their continued use from users’ perspectives as performance evaluation is a major concern of the effects of ITs or ISs [
To provide comprehensive understanding and insights into the postimplementation stage of mHealth systems (or at IS infusion stage), this study proposed an extended ECM research model for investigating key factors affecting the continuance and performance of mHealth services in Taiwan by incorporating the ECM proposed by Bhattacherjee [
Therefore, the research model (
Research framework. H1: The confirmation of mHealth systems significantly affects perceived usefulness; H2: The confirmation of mHealth systems significantly affects user satisfaction; H3: The perceived usefulness of mHealth systems significantly affects user satisfaction; H4: User satisfaction with mHealth systems significantly affects mHealth continuance; H5: The perceived usefulness of mHealth systems significantly affects mHealth continuance; H6: The continuance of mHealth significantly affects individual performance; H7: The individual characteristics of HCPs significantly affect mHealth continuance; H8: The technology characteristics of mHealth significantly affect mHealth continuance; H9: The task characteristics of HCPs significantly affect mHealth continuance; mHealth: mobile health.
Measurement and operational definitions of variables.
Construct | Operational definition | Source | Measurement items | |
Confirmation | Users’ perception of the congruence between expectation of mHealtha use and its actual performance | [ |
4 | |
Perceived usefulness | Users’ perception of the expected benefits of mHealth use | [ |
5 | |
User satisfaction | Users’ affect with (feelings about) mHealth use | [ |
3 | |
mHealth continuance | Users’ intention to continue using mHealth | [ |
3 | |
Habits | The extent to which an individual tends to use the mHealth automatically | [ |
4 | |
Innovativeness | Willingness to try out any new technology | [ |
4 | |
Availability | The ability of accessing patient information when required | [ |
3 | |
Portability | The degree of ease associated with transporting the mHealth | [ |
3 | |
Maturity | The existence of a level of system quality that is perceived as satisfactory and the perceived need for system improvement by the user. | [ |
3 | |
Time critical | The urgency when accessing information through the mHealth | [ |
3 | |
Interdependence | The degree to which completing tasks using mHealth requires interaction with other people | [ |
3 | |
Mobility | The extent to which a task is being performed in different locations using the mHealth | [ |
3 | |
Individual performance | The use of mHealth can help health care practitioner improve efficiency, effectiveness, and quality of medical activities | [ |
6 |
amHealth: mobile health.
The second part investigates the effect of characteristics of technology, individual, and task on mHealth continuance; this part is based on the framework proposed by O’Connor et al [
In this study, confirmation refers to the users’ perception of the congruence between expectation associated with the use of mHealth systems and their actual performance. Perceived usefulness refers to the users’ perception of the expected benefits of mHealth use. User satisfaction refers to users’ affect (feelings) regarding previous mHealth use. As shown in the ECM, confirmation has a direct effect on perceived usefulness and user satisfaction [
H1: The confirmation of mHealth systems significantly affects perceived usefulness.
H2: The confirmation of mHealth systems significantly affects user satisfaction.
H3: The perceived usefulness of mHealth systems significantly affects user satisfaction.
H4: User satisfaction with mHealth systems significantly affects mHealth continuance.
H5: The perceived usefulness of mHealth systems significantly affects mHealth continuance.
H6: The continuance of mHealth significantly affects individual performance.
Most factors mentioned in the framework proposed by O’Connor et al [
Goodhue and Thompson [
H7: The individual characteristics of HCPs significantly affect mHealth continuance.
H7a: Personal innovativeness significantly affects mHealth continuance.
H7b: Individual habits significantly affect mHealth continuance.
H8: The technology characteristics of mHealth significantly affect mHealth continuance.
H8a: The availability of mHealth significantly affects mHealth continuance.
H8b: The portability of mHealth significantly affects mHealth continuance.
H8c: The maturity of mHealth significantly affects mHealth continuance.
H9: The task characteristics of HCPs significantly affect mHealth continuance.
H9a: Task time criticality significantly affects mHealth continuance.
H9b: Task interdependence significantly affects mHealth continuance.
H9c: Task mobility significantly affects mHealth continuance.
The questionnaire was designed in 2 stages. The first stage involved the establishment of measurement items. We collected results from literature reviews to obtain a comprehensive list of measurement items. All measures for each construct were obtained from existing validated instruments, and they were modified to ensure the appropriateness for mHealth. A total of 4 variables, namely confirmation, perceived usefulness, satisfaction, and mHealth continuance, which were derived from the ECM, were measured using 15 items adapted from Bhattacherjee [
The second stage of questionnaire design involved the evaluation and selection of the measurement scale. A content validity index (CVI) was used to evaluate the questionnaire content according to a threshold value of 0.8 for item selection suggested by Petrick [
The respondents of this study were the HCPs of the target hospital with approximately 120 doctors and 500 nurses in southern Taiwan. Since 2009, the case hospital has developed and implemented mHealth systems, a combination of mobile ISs and medical devices, for satisfying HCPs’ needs of clinical patient care, particularly for providing more timely communication of HCPs and direct data input at source, reducing possible medical errors, and accessing up-to-date medical records. The mobile ISs can connect and access all required and integrated patient-related information from hospital ISs, including various developed systems (computerized physician order entry system, laboratory ISs, nursing ISs, pharmacy ISs, picture archiving and communication system, electronic medical records, patient referral system, and others) for supporting inpatient, outpatient, and emergency services in a hospital, through a secure wireless network infrastructure. The mobile ISs can be installed on various mobile devices, including a mobile nursing cart equipped with a Tablet PC (specifically for nurses), mobile medical cart equipped with a Tablet PC (specifically for physicians), mobile phones, and iPad for satisfying the mobile needs of HCPs, particularly in the inpatient and emergency services.
Since 2014, some health apps of the case hospital have been developed and installed on mobile phones and iPads for providing instant access to the results of medical examinations, tests, and reports and receiving immediate notifications from high-risk reminder systems for clinical laboratory critical value alerts; however, those apps only provide relatively specific and limited information for patient care because of the limitations of small screen size, less computation power, and data key-in problems of intelligent mobile devices. Therefore, HCPs in the case hospital prefer accessing full patient care information through mobile ISs installed on the mobile nursing cart, mobile medical cart, and tablet PC. HCPs who had at least one year of experience in mHealth apps and were active and voluntary users of mHealth, using mobile ISs through mobile devices in clinical practices, were requested to participate. After obtaining approval from the Institutional Review Board (IRB NO.105B-009), the questionnaires were distributed to qualified HCPs under the assistance of the nursing department and hospital administration department. The duration of data collection was from February 1 to March 1 in 2016.
The survey was administered to 282 respondents, and 201 valid responses were returned, which indicated a response rate of 71.3% (201/282). Voluntary participation might explain the relatively high response rate. The demographic data (see
Demographic data (n=201).
Measure or category | Statistics | |
<30 | 97 (48.3) | |
31-40 | 88 (43.8) | |
41-50 | 12 (6.0) | |
51-60 | 4 (2.0) | |
Male | 12 (6.0) | |
Female | 189 (94.0) | |
Junior college | 54 (26.9) | |
Bachelor | 144 (71.6) | |
Master (or higher) | 3 (1.5) | |
Medical (Physicians) | 12 (6.0) | |
Nursing (Clinical nurses) | 189 (94.0) | |
1-3 | 146 (72.6) | |
3-6 | 43 (21.4) | |
6-9 | 7 (3.5) | |
>9 | 5 (2.5) | |
1 | 46 (22.9) | |
1-5 | 145 (72.1) | |
5-10 | 10 (5.0) |
The collected data were analyzed using the partial least square (PLS) technique, which can offer extensive, scalable, and flexible causal-modeling capabilities [
Model fit and quality indices.
Quality indices | Statistics | Criteria ( |
Result |
Average path coefficient (APC) | 0.237 ( |
<.05 | Fit |
Average R-squared (ARS) | 0.529 ( |
<.05 | Fit |
Average adjusted R-squared (AARS) | 0.521 ( |
<.05 | Fit |
Average block variance inflation factor (AVIF) | 2.246 | Acceptable if ≤5, ideally ≤3.3 | Fit |
Average full collinearity VIF (AFVIF) | 2.324 | Acceptable if ≤5.0, ideally ≤3.3 | Fit |
Tenenhaus Goodness of Fit (GoF) | 0.649 | Small ≥.1, medium ≥.25, large ≥.36 | Fit |
R-squared contribution ratio (RSCR) | 0.989 | Acceptable if ≥ .9, ideally=1.0 | Fit |
We further evaluated the psychometric properties of the instrument regarding reliability, convergent validity, and discriminate validity. According to the method used by Hair et al [
The structural research model was analyzed using WarpPLS 5.0 and the bootstrap resampling method [
Results of the reliability and validity of the research model.
Construct | COa | PUb | SATc | INNd | HABe | AVAf | TCg | INTh | MCi | PERj | MOBk | PORTl | MATm | AVEn (>.5) | CRo (>.7) | Cronbach alpha (>.7) |
CO | 0.898 | —p | — | — | — | — | — | — | — | — | — | — | — | 0.806 | 0.943 | .919 |
PU | 0.718 | 0.869 | — | — | — | — | — | — | — | — | — | — | — | 0.755 | 0.939 | .918 |
SAT | 0.690 | 0.657 | 0.894 | — | — | — | — | — | — | — | — | — | — | 0.798 | 0.922 | .874 |
INN | 0.265 | 0.338 | 0.309 | 0.867 | — | — | — | — | — | — | — | — | — | 0.751 | 0.924 | .889 |
HAB | 0.586 | 0.500 | 0.502 | 0.320 | 0.910 | — | — | — | — | — | — | — | — | 0.829 | 0.951 | .931 |
AVA | 0.509 | 0.522 | 0.584 | 0.306 | 0.504 | 0.855 | — | — | — | — | — | — | — | 0.730 | 0.890 | .815 |
TC | 0.450 | 0.511 | 0.475 | 0.361 | 0.387 | 0.513 | 0.887 | — | — | — | — | — | — | 0.787 | 0.917 | .864 |
INT | 0.344 | 0.433 | 0.358 | 0.332 | 0.280 | 0.517 | 0.707 | 0.918 | — | — | — | — | — | 0.843 | 0.942 | .906 |
MC | 0.528 | 0.564 | 0.534 | 0.320 | 0.534 | 0.586 | 0.529 | 0.503 | 0.914 | — | — | — | — | 0.836 | 0.939 | .901 |
PER | 0.628 | 0.664 | 0.638 | 0.406 | 0.535 | 0.610 | 0.541 | 0.521 | 0.695 | 0.895 | — | — | — | 0.802 | 0.960 | .950 |
MOB | 0.302 | 0.406 | 0.286 | 0.245 | 0.326 | 0.410 | 0.476 | 0.582 | 0.463 | 0.444 | 0.947 | — | — | 0.897 | 0.946 | .886 |
PORT | 0.352 | 0.355 | 0.403 | 0.200 | 0.297 | 0.574 | 0.452 | 0.434 | 0.410 | 0.462 | 0.271 | 0.828 | — | 0.686 | 0.868 | .771 |
MAT | 0.440 | 0.477 | 0.506 | 0.249 | 0.340 | 0.653 | 0.547 | 0.517 | 0.518 | 0.606 | 0.337 | 0.621 | 0.899 | 0.809 | 0.927 | .881 |
aCO: confirmation.
bPU: perceived usefulness.
cSAT: satisfaction.
dINN: innovativeness.
eHAB: habits.
fAVA: availability.
gTC: time critical.
hINT: interdependence.
iMC: mobile health continuance.
jPER: performance.
kMOB: mobility.
lPORT: portability.
mMAT: maturity.
nAVE: average variance extracted.
oCR: composite reliability.
pThe omitted correlation coefficients between constructs in the upper diagonal matrix are equal to the values in lower diagonal matrix.
Results of the model validity. H1: The confirmation of mHealth systems significantly affects perceived usefulness; H2: The confirmation of mHealth systems significantly affects user satisfaction; H3: The perceived usefulness of mHealth systems significantly affects user satisfaction; H4: User satisfaction with mHealth systems significantly affects mHealth continuance; H5: The perceived usefulness of mHealth systems significantly affects mHealth continuance; H6: The continuance of mHealth significantly affects individual performance; H7: The individual characteristics of HCPs significantly affect mHealth continuance; H8: The technology characteristics of mHealth significantly affect mHealth continuance; H9: The task characteristics of HCPs significantly affect mHealth continuance; mHealth: mobile health.
Consistent with previous ECM-related studies [
This study showed that perceived usefulness (beta=.128;
This study made an empirical validation on the framework of mHealth infusion proposed by O’Connor et al; however, only mobility, habit, and maturity were found to be salient predicators in mHealth infusion. Previous studies have highlighted that mobility is the primary reason for the applications of technological innovation in hospitals [
Furthermore, the perceived usefulness and user satisfaction of mHealth systems have been considered as critical factors affecting technology continuance in ECM-related studies [
The results indicated that mHealth continuance (beta=.703;
Furthermore, in this study, we evaluated the individual performance of the HCPs derived from mHealth continuance by using 6 items (
Individual performance derived from mobile health continuance.
Items | Mean (SD) |
Using mHealtha can effectively improve information exchange between me and the health care team | 4.10 (0.60) |
Using mHealth can effectively facilitate my communication with patients and their families | 4.10 (0.60) |
Using mHealth allows me to provide efficient patient care | 3.94 (0.60) |
Using mHealth enhances the quality of patient care | 3.91 (0.63) |
Using mHealth improves my professional image | 3.86 (0.63) |
Using mHealth facilitates my work completeness | 3.83 (0.62) |
Average score | 3.96 (0.61) |
amHealth: mobile health.
The key building block for sHealth care is mHealth, and the appropriate use of mHealth may result in major advances in expanding health care coverage, improving decision making, managing chronic conditions, and providing suitable health care during emergencies [
The results revealed that mHealth continuance was mainly affected by perceived usefulness, technology maturity, individual habits, task mobility, and user satisfaction, whereas individual performance was influenced by mHealth continuance. User satisfaction was affected by confirmation and perceived usefulness of mHealth, whereas perceived usefulness was affected by confirmation. This study showed that the ECM remained valid in the mHealth context from the HCPs’ perspective. Among the identified factors that influenced mHealth continuance in this study, task mobility, individual habits, and technology maturity affected mHealth continuance more significantly than the factors (perceived usefulness and user satisfaction) derived from the ECM. To increase the intention of health professionals toward mHealth continuance, characteristics related to task mobility, user habits, and technology maturity and users’ perceptions related to perceived usefulness and user satisfaction must be given attention.
We found that the users’ intention toward mHealth continuance increased when the focus was on the design and implementation issues of the mHealth applications to satisfy the actual needs of users. This implied that if mHealth applications provided high quality of system and satisfactory support to meet the needs of the HCPs, the users will have a relatively high intention toward mHealth continuance. We further suggested the evaluation of task mobility, technology maturity, and individual habits and provision of satisfactory support related to the fit between the aforementioned factors while introducing mHealth applications. Consistent with the results of previous ECM-related studies [
This study has made theoretical and practical contributions to the evaluation of mHealth systems. First, the study proposed an innovative integration model that extended the ECM with antecedents of IS infusion (including the characteristics of individuals, technology, and tasks) to identify the critical factors influencing mHealth continuance and performance from the perspective of HCPs. The extended ECM provided a comprehensive research model for investigating mHealth continuance or IS continuance. Second, the inclusion of characteristics of individual, technology, and task not only provided a reasonable framework but also highlighted that other studies can incorporate various critical factors depending on research contexts and situations. Third, the identified critical and salient factors that affected mHealth continuance and performance can be used as assessment tools by hospitals that have implemented mHealth to facilitate mHealth use and infusion. (4) The results can also help health care institutions that intend to introduce or develop mHealth applications in identifying critical issues and effectively allocating limited resources to mHealth systems.
We suggest focus areas for additional research and future studies on this topic. First, scholars can use the research model derived in this study, apply it to various research contexts, and compare the findings. Second, others can conduct an in-depth case study with the findings obtained from this study. To expand the research scope at the IS infusion stage, future studies should pay attention to the investigated factors (personal innovativeness, availability, portability, timeliness, and interdependence) that were insignificant factors in this study. This is reasonable as those insignificant factors may have different (mixed) results because of the difference of research contexts, user groups, and application systems as mentioned in a summary of technology acceptance model studies [
This study has several limitations. First, this study was conducted only at a regional hospital in Taiwan; thus, the findings obtained from this research may not be immediately transferrable to other countries with different participant demographics and cultures. Second, a cross-sectional survey design was used for this study; thus, the inherent limitations of the survey methodology were inevitable. Furthermore, this study sample comprised voluntary participants. However, as the survey approach is commonly used in the field, the use of this method may not have adversely affected the results.
Questionnaire for mobile health continuance and performance.
average adjusted R-squared
average full collinearity variance inflation factor
average path coefficient
average R-squared
average variance extracted
average block variance inflation factor
composite reliability
content validity index
expectation-confirmation model
electronic health
Goodness of Fit
health care professional
hospital information system
information communication technology
information system
information technology
mobile health
personal computer
partial least square
R-squared contribution ratio
smart health
task-technology fit
The authors sincerely extend their gratitude and recognition to the National Science Council of Taiwan for funding this study (MOST 104-2410-H-041-007).
None declared.