Download

Original Research

Engagement and Outcomes with Mobile Health Technology among Patients Hospitalized with Acute Venous Thromboembolism

Horatio Holzer, MD*, 1*, Eric R. Goodlev, MD*, Julie M. Pearson, MPH, BSN, RN, Sally Engelman, MPH, Dana Sperber, MSW, Andrew S. Dunn, MD, MPH, FRCP, Beth Raucher, MD, SM

Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA

Abstract

Mobile health (mHealth) technology can improve engagement and self-management, though few studies have assessed the factors associated with engagement of mHealth among hospitalized patients. We implemented a multifaceted transitions of care (TOC) intervention consisting of a novel patient-facing smartphone application (app), text message medication reminders, and access to a patient navigator for patients hospitalized with venous thromboembolism. Overall, application uptake (36%) and engagement were low. Patients who downloaded the app were young (50.5 vs 66.1 years, P ˂ 0.01) and had a lower burden of disease (Charlson score 3.97 vs 5.65, P = 0.048). Similarly, patients who engaged with the app were young (48.5 vs 57.6 years, P = 0.049) and had a lower burden of disease (Charlson score 3.12 vs 7.14, P = 0.033). Our findings suggest that design and implementation of mHealth applications will be challenging for hospitalized populations characterized by old age, numerous comorbidities, and high acuity.

Résumé

La technologie de santé mobile peut favoriser la participation et l’autogestion des patients, mais peu d’études évaluent les facteurs associés au recours à la santé mobile parmi les patients hospitalisés. Nous avons réalisé une intervention à multiples facettes de transition des soins qui comprend une application pour téléphone intelligent novatrice à l’intention des patients, des rappels par texto de prise de médicaments et un accès à une infirmière-pivot pour les patients hospitalisés en raison d’une thromboembolie veineuse. Dans l’ensemble, l’adhésion (36 %) et la participation à l’application sont faibles. Les patients qui ont téléchargé l’application sont plus jeunes (50,5 ans par rapport à 66,1 ans; p ˂ 0,01) et leur fardeau de la maladie est moins lourd (score de Charlson de 3,97 par rapport à 5,65; p = 0,048). De même, les patients qui utilisent l’application sont plus jeunes (48,5 ans par rapport à 57,6 ans; p = 0,049) et leur fardeau de la maladie est moins lourd (score de Charlson de 3,12 par rapport à 7,14; p = 0,033). Nos résultats laissent supposer qu’il sera difficile de concevoir et de mettre en œuvre des applications de santé mobile pour les populations hospitalisées caractérisées par un âge avancé, de nombreuses affections comorbides et une acuité élevée.

Key words: mobile applications, mobile health, patient care management, transition of care, venous thromboembolism

Corresponding Author: Horatio Holzer: horatio.holzer@mountsinai.org

*Horatio Holzer and Eric R. Goodlev contributed equally to the project and manuscript preparation and would like to be recognized as shared first authors.

Submitted: 26 July 2021; Accepted: 19 October 2021; Published: 17 May 2022

Doi: http://dx.doi.org/10.22374/cjgim.v17i2.575

All articles published in DPG Open Access journals
This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)(https://creativecommons.org/licenses/by-nc/4.0/).

Background

Patient-facing mobile health (mHealth) applications have shown promise in improving engagement and self-management among patients with chronic conditions including inflammatory bowel disease,1 cystic fibrosis,2 and asthma.3 However, few studies have assessed mHealth implementation among patients hospitalized with acute conditions. Compared to outpatients with chronic disease, acutely ill, hospitalized patients are more likely to be older, have a higher burden of disease, and suffer from acute cognitive impairment.4 Each of these factors may negatively impact uptake and utilization of mHealth technology.

Acute venous thromboembolism (VTE), including deep vein thrombosis and pulmonary embolism, affects up to 600,000 patients in the United States annually.5 Patients hospitalized with acute VTE are at high risk for complications, and this risk is further heightened during transitions of care (TOC). VTE patients are typically elderly and have a high burden of chronic conditions.6 These factors make the TOC from the hospital to the community challenging, contributing to a significant risk of recurrence, complication, and readmission.

We aimed to assess whether mHealth technology could be effectively implemented in acutely ill, hospitalized patients with a new diagnosis of VTE, and to identify factors associated with uptake and engagement. We developed a novel VTE Transitions of Care Bundle using a patient-facing mHealth application and a dedicated patient navigator. Secondary aims were to reduce rates of recurrent VTE, bleeding complications, and hospital readmissions.

Materials and Methods

Project design and setting

The project was conducted at a 1134-bedded academic hospital located in New York City. Patients were eligible for inclusion if they were over 18 years of age and had a primary or secondary diagnosis of acute VTE requiring anticoagulation, including deep vein thrombosis, pulmonary embolism, or visceral venous thrombosis. Initially, only patients admitted to the hospital medicine service who were discharged directly home were eligible for study inclusion. Due to lower than anticipated study enrollment, the protocol was expanded to include patients admitted to hematology or oncology and geriatrics services, patients in the emergency department or observation unit, and patients discharged to subacute rehabilitation facilities.

Exclusion criteria included pregnancy, non-English-speaking patients, patients discharged without anticoagulants, residence outside of the New York City metropolitan area, and patients residing in long-term care facilities. Neither smartphone ownership nor capability to use a smartphone were required for enrollment as the patient navigator was available for all enrolled patients. Patients who did not have access to the mobile app interacted with the patient navigator via telephone or traditional short message service (SMS) messaging.

Potential subjects were identified through a daily review of hospital records of patients initiated on treatment-dose anticoagulation and through referral from attending providers. Patients were approached by the patient navigator for enrollment and to provide written informed consent prior to discharge. A retrospective cohort of patients admitted with acute VTE between September 30, 2015 and May 1, 2016 served as the baseline group. This cohort was comprised of patients with a diagnosis code for VTE (see Appendix A) from the same hospital and service lines as the study group.

The project was approved by the Institutional Review Board at the Icahn School of Medicine of Mount Sinai.

Interventions

The intervention consisted of the following components: (1) a novel patient-facing application (HealthFlo), (2) text message medication adherence reminders, (3) access to a patient navigator, and (4) population health management via a patient navigator dashboard to track medication adherence based on patient app engagement data (Figure 1). Patients were enrolled in the study from May 1, 2016 to December 31, 2017.

Figure 1. Interface between patient, HealthFLO app, and patient navigator. App provides secure communication channel and daily SMS medication reminders. Communication also occurs outside of the app (telephone, SMS text).

SMS, short messaging service.

Patient navigator

The patient navigator was a social worker whose role was to facilitate study enrollment, app engagement, and clinical follow-up during the TOC period. The navigator visited potential study participants at the bedside to demonstrate the Healthflo app and help install the app on the participant’s phone. Following discharge, the patient navigator remained in contact with the patient via HealthFlo’s secure messaging feature. The navigator contacted patients who did not have access to a smartphone or who declined to download the app via telephone or non-app-based SMS messaging. Those who downloaded the application were sent automated daily that read, “Did you take your blood thinner today?” Patients were able to respond either yes or no. The patient navigator tracked reported medication adherence based on these responses, tracked patient-level data using an online dashboard, and recorded all interactions with patients. When patients stopped responding or became nonadherent, the patient navigator contacted the patient for further care coordination.

The HOSPITAL readmission risk score was used to risk-stratify patients into low-, intermediate-, and high-risk groups.7 For patients in the intermediate- or high-risk groups, the patient navigator called patients after each follow-up appointment and made outreach efforts weekly once via text message, app-based messaging, or telephone. These care management services were offered to patients whether they downloaded the app or not.

HealthFlo app

HealthFlo is a patient-facing application developed by the study team for use on iOS and Android-compatible devices. The app features secure messaging, patient education materials, and daily medication reminders. The Healthflo app was initially piloted as two separate mobile applications: one that was used for secure messaging and another that provided patient education regarding VTE. To improve the user experience and in response to user feedbacks, the apps were merged into a single app midway through the study.

Focus groups

Focus groups were held with members of the study team (JP and SE) and five participants. Questions focused on patients’ experiences during the hospital discharge process and with the patient navigator and Healthflo app. Participants were compensated with a $25 gift card.

Study definitions and outcomes

The enrollment rate was defined as the number of patients enrolled in the study divided by the total number of patients who met inclusion criteria and were approached via the protocol.

Application uptake was defined as the percentage of enrolled patients who downloaded the smartphone app. Application engagement was defined as the percentage of enrolled patients who downloaded the app and used either the secure messaging system or medication reminders at least once.

Meaningful interactions were defined as interactions in which care coordination, patient care, or quality of care was impacted as a direct result of interacting with the patient navigator. Categories and definitions of meaningful interactions are summarized in Table 1. The secure text messages were analyzed and coded using the same definition of meaningful interactions by two members of the study team (SE and JP). Non-app-based communications were collected and recorded by the patient navigator, and similarly analyzed and coded by SE and JP according to the definitions in Table 1. In addition, data on the frequency of HealthFlo use and response rates to the SMS medication reminders from the app development team were recorded.

Table 1. Categories and definitions of meaningful use.

Category Definition
Scheduling and transportation for outpatient visits Arranging or scheduling outpatient doctor visits; arranging to and fro transportation for provider visits
Counseling, advocacy, emotional support Emotional counseling, support; goals of care conversation
Social services support Housing-related services; SNAP or meal delivery services; Insurance or Medicaid; arrangement of in-home medical services
VTE Meds VTE medication assistance
Medication and DME Non-VTE medication assistance, DME assistance
Improving communication between patient and provider Facilitating communication between patient and provider

Categories and definitions of meaningful use for scoring patient interactions. DME, durable medical equipment; SNAP, supplemental nutrition assistance program; VTE, venous thromboembolism.

The Charlson Comorbidity Index8 and the HOSPITAL readmission scores7 were used to characterize the comorbidity burden and readmission risk, respectively.

Secondary outcomes included all-cause hospital readmission within 30 days, major bleeding complications, overall complication rate, all-cause mortality, and length of stay. The overall complication rate was defined as a composite of the 18 Hospital-acquired Conditions (HACs) defined by Center for Medicare and Medicaid Services (CMS) and 82 additional conditions identified by Premier, Inc.9

Data sources and collection

Administrative data (Premier Inc.)10 were used to obtain risk-adjusted outcomes data on hospital readmission, length of stay, mortality, and complications. To obtain rates of bleeding complications, the Epic (Epic System Corporation, Madison, WI) electronic medical record was queried using ICD 9 and 10 codes (Appendix B). The baseline comparison group was identified through billing data from the Premier administrative database using ICD 9 codes for acute VTE from September 30, 2015 to May 1, 2016. Univariate statistical tests were used to assess for differences in outcomes between groups (paired-t tests and Chi-Square tests for continuous and nominal variables, respectively). SPSS (version 22.0)11 was used for all analyses. Qualitative data were independently reviewed by two members of the study team (SE and JP), and any discrepancies in categorization were resolved through discussion with the entire study team.

Results

The enrollment rate was 69.5% (98 of 141). Of the 43 patients who declined, the most common reasons given were the patient reported being overwhelmed or not interested (27.9%), the patient reported that family was comfortable managing his or her care (27.9%), or the patient reported comfort managing his or her own care (25.6%). Nine patients who enrolled in the study were subsequently excluded from analysis, leaving a total of 89 subjects in the intervention group. Reasons for exclusion were as follows: discharge disposition changed (1 patient), the study group was unable to access administrative data from Premier (6 patients), duplicate enrollment (1 patient), or the patient withdrew consent (1 patient).

The baseline group included 128 patients, and the intervention group included 89 patients, of which 72 were admitted to the hospitalist service. The age, gender, and Charlson Comorbidity Index8 score of the baseline and intervention groups were similar. The baseline group had a greater percentage of black patients than the intervention group (Table 2).

Table 2. Demographic information and comorbidities.

Variable Baseline (N = 128) Intervention (N = 89) P
Age (mean) 60.9 59.9 0.676
Gender
Female
Male
60.9% (78)
39.1% (50)
61.8% (55)
38.2% (34)
0.898
Race
Black
White
Other
49.2% (63)
28.9% (37)
21.9% (28)
27.0% (24)
21.3% (19)
51.7% (46)
0.001
Comorbidity Baseline (%) Intervention (%) P
Myocardial infarction 12.5 9.0 0.417
Congestive heart failure 28.1 29.2 0.861
Peripheral vascular disease 19.5 9.0 0.033
Cerebrovascular disease 7.0 9.0 0.598
Dementia 3.9 4.5 0.831
Chronic obstructive pulmonary disease 45.3 36.0 0.169
Rheumatologic disease 10.9 9.0 0.640
Peptic ulcer disease 4.7 2.2 0.348
Mild liver disease 8.6 2.2 0.053
Diabetes without complications 32.0 27.0 0.423
Diabetes with complications 13.3 16.9 0.465
Paraplegia or hemiplegia 1.6 0.0 0.236
Renal disease 28.9 20.2 0.148
Any malignancy 26.6 25.8 0.906
Moderate liver disease 4.7 12.4 0.039
Metastatic cancer 7.8 3.4 0.175
HIV 0.0 0.0

Demographic information and comorbidities in the baseline and intervention groups.

HIV, Human immunodeficiency virus.

Application uptake

Application uptake was noted for 32 of 89 (36%) patients in the intervention group. The most common reasons patient gave for not downloading the app were “no smartphone” (45.6% of patients with no uptake), “doesn’t use apps” (24.6%), “not interested” (7.0%), or “technological issue” (22.8%), which included forgotten password, lack of phone memory, and download error.

Patients with application uptake had a lower Charlson Comorbidity score than those who did not download the app (3.97 vs 5.65, P = 0.048). The average age of patients with application uptake was 50.5 years compared to 66.1 years among patients who did not download the app (P ˂ 0.01). There was a trend toward increased likelihood of app uptake among patients at lower risk of readmission as identified by the HOSPITAL Readmission score (45.2% of low-risk patients vs 27.7% of intermediate- or high-risk patients, P = 0.084).

Application engagement

Of patients who downloaded the app, 66% actively used the app as measured by use of the SMS text reminders or secure messaging. The average age of patients who engaged with the app was lower than for patients who did not engage with the app (48.5 and 57.6 years, respectively; P = 0.049). Patients who engaged with the app had a lower Charlson score than patients who did not engage with the app (3.12 vs 7.14, P = 0.033).

Secondary outcomes

There was no significant difference in all-cause 30-day -hospital readmission, mortality, complication rates, or length of stay between the baseline and intervention groups (Table 3). The readmission rate was similar in patients who did not download the app compared to the group who downloaded app (16.7% vs 13.3%, P = 0.76). None of the patients who used the secure messenger to communicate with the patient navigator were readmitted (0/8) compared to 4 of 22 (18.2%) patients who did not use the secure messenger (P = 0.267).

Table 3. Utilization and clinical outcomes.

Variable Baseline (N = 128) Intervention (N = 89) P
Length of stay (days) 6.38 (SD = 8.80) 6.19 (SD = 4.78) 0.852
LOS O/E 1.38 (SD = 1.28) 1.39 (SD = 0.69) 0.970
Readmission rate 13.6% (16/118) 15.5% (13/84) 0.702
Readmission O/E 0.98 (SD = 2.82) 1.19 (SD = 3.10) 0.610
Complication rate 15.1% (18/119) 10.6% (9/85) 0.346
Complication O/E 1.49 (SD = 4.44) 1.28 (SD = 4.89) 0.756
Mortality rate 0.8% (1/119) 0.00% (0)
Mortality O/E 0.03 (SD = 0.27) 0.00 (SD = 00) 0.304

Outcomes in study parameters among the baseline and intervention groups.

LOS, length of stay; O/E, observed over expected; SD, standard deviation.

There was no significant difference in bleeding complications within 30 days in the baseline (14.1%) and intervention groups (11.2%, P = 0.541).

Meaningful interactions with the patient navigator

There were 41 recorded meaningful interactions with the patient navigator that were non-app based (i.e., via telephone or regular text message), compared to 82 recorded app-based meaningful interactions (Table 4).

Table 4. Summary of app-based and non-app-based meaningful interactions with the patient navigator.

Category Non-App-based communication App-based communication
Scheduling and transportation for outpatient visits 37% (15) 21% (17)
Counseling, advocacy, emotional support 22% (9) 39% (32)
Social services support 15% (6) 2% (2)
VTE Meds 5% (2) 12% (10)
Medication and DME 7% (3) 1% (1)
Improving communication between patient and provider 15% (6) 24% (20)

Summary of app-based and non-app-based meaningful interactions with the patient navigator. Percentages are rounded and do not sum to 100%.

DME, durable medical equipment.

Overall, 18 of 32 (56%) patients who downloaded the application used app-based text message communication to interact with the patient navigator. Five users had more than 10 interactions with the patient navigator through the app-based text messaging system.

Discussion

Uptake and engagement with the mHealth technology were low in our study on patients hospitalized with acute VTE. The app was downloaded by only 36% of patients, despite in-person facilitation by a patient navigator, and only 66% of patients who downloaded the app engaged with it. Age and burden of disease were predictors of poor application uptake and engagement. Low application uptake and engagement may have limited the clinical benefit in the intervention group.

These findings are concerning as patients hospitalized with acute conditions, including VTE, are often elderly and have a higher burden of disease and acuity. If these factors are associated with low mHealth uptake and engagement, use of this technology may be limited in conditions associated with old age and high burden of disease, such as chronic obstructive lung disease and heart failure. By contrast, the technology may be more readily applicable to patients hospitalized with conditions such as asthma or sickle cell disease. This study suggests that nontechnological interventions, such as better care navigation, may be of greater benefit in older populations with high burden of disease.

Prior studies have identified similar challenges in applying mHealth technology to older and acutely ill populations. Pugliese et al.12 provided a tablet-based therapy platform for patients with acute stroke and found low rates of uptake and engagement. Similarly, Wolf et al.13 provided a mobile symptom-tracking tool and electronic health diary to patients after diagnosis of acute coronary syndrome and found that only 39% of the patients used the mHealth tool at least once after the index hospitalization.

In contrast, the SUPPORT trial found good uptake and engagement with a smartphone-based interactive patient support tool for patients hospitalized with myocardial infarction.14 Application users demonstrated improved self-reported medication adherence compared to nonusers. However, the patients in the SUPPORT trial were young and had fewer comorbid conditions than patients in our study. mHealth technology has been more successfully deployed among young hospitalized patients. In a pilot randomized trial among patients hospitalized for complications of anorexia nervosa, Neumayr et al.15 reported that 17 out of 18 participants logged in to a smartphone app daily for 14 consecutive days after discharge. The median age of the patients in the intervention group was 20 years.

Studies of mHealth technology in younger, healthier, nonhospitalized patients with chronic conditions have shown comparatively good uptake and engagement, as well as improved outcomes.1,3,1621 The HealthPROMISE study provided a mobile platform for patients with inflammatory bowel disease to track quality of life measures and demonstrated lower rates of uncontrolled anxiety and fatigue among application users.1,16 In pediatric patients with chronic asthma, the AsthmaCare mHealth application demonstrated good uptake as well as patient-reported improvements in care management.3 The findings of our study and prior studies suggest that age, burden of disease, and medical acuity may be negatively associated with mHeath uptake and engagement.

The most common reasons cited by our study participants for not downloading the app was lack of access to a smartphone and lack of comfort using smartphone applications. Successful real-world applications of mHealth technology in older, acutely ill population will have to contend with the low number (30–42%) of adults over 65 years of age who report owning a smartphone.22

Interestingly, only 4 out of 57 patients who did not download the app reported doing so because of a lack of interest in mHealth technology. In combination with our high recruitment rate of 70%, the data suggest that there may be general interest in using mHealth technology among this population. This finding is supported by Pugilese et al.,12 who found high levels of interest but low levels of uptake of mHealth technology among patients with acute stroke. Given the observed barriers to uptake of mHealth technology in our population of older patients with a high burden of disease, our findings suggest that future interventions will need to engage the target population to design a usable, intuitive, and relevant smartphone application.

Our data suggest a possible beneficial impact among patients who did engage with the app. For example, we found a trend of fewer readmissions for patients who used the app’s secure messaging function to communicate with the patient navigator compared to than those who did not, though the small number of events limits any definitive conclusions. In addition, we identified more app-based than non-app-based meaningful interactions with the patient navigator, despite having fewer patients with access to the app-based text messenger. These data suggest that access to the mHealth application lowered barriers to communication and may promote counseling, support, and communication with providers. Similarly, Wolf et al.13 found improvements in self-efficacy among patients hospitalized with acute coronary syndrome who engaged with an mHealth app; however, the rate of uptake and engagement with the app was low.

During the focus group sessions, several patients reported feeling more connected with their health-care team after both telephone- and app-based communication from the navigator; however, the low number of patients in the focus groups limits the generalizability of these findings. In addition, during a focus group session held early during the study, several patients reported a preference for a single app rather than two separate apps. We merged the apps into a single app during the study based on this feedback.

Our primary finding of low uptake and engagement with the mHealth app is a limitation as we did not have sufficient power to allow definitive conclusions on the impact of the VTE TOC bundle on clinical outcomes. In addition, because we did not employ a user-centered design process for the app, it is difficult to separate whether the user interface or the characteristics of our population had the largest impact on app uptake and engagement. We also did not assess socioeconomic status, which may be negatively associated with smart phone use. Lastly, we did not assess whether physical or cognitive impairments may have limited app uptake and engagement; these impairments are associated with age and burden of disease and may be significant barriers to mHealth use for hospitalized patients with acute VTE.

Conclusion

Our findings of low uptake and engagement with a patient-facing application facilitated by a patient navigator for patients hospitalized with acute VTE suggest that innovative, user-centered design and implementation methods will be needed for older and more acutely ill populations. The data suggest that there may have been improved outcomes and TOC for the subgroup of patients for whom there was substantial engagement. Further studies are needed to confirm the relationship between age, comorbidity burden, and medical acuity with mHealth uptake and engagement and to identify mHealth features that can be effective in older acutely ill populations. Additionally, future studies may assess whether nontechnological interventions, such as better care navigation, may be of greater utility than mHealth interventions for older populations with high burden of disease.

Informed Consent

The study was approved and monitored by the Institutional Review Board at the Mount Sinai Hospital.

Funding

The authors disclose receipt of the following financial support for the research, authorship, and publication of this article. This work was supported by a grant from Pfizer Independent Grants for Learning & Change and Bristol–Myers Squibb Independent Medical Education. The Joint Commission provided administrative oversight of the program.

Competing Interests

HH reports ownership of 250 shares common stock of Abbott Laboratories, outside of the submitted work. AD is a member of the Advisory Board for ACP’s Center for Quality Initiative, ACP Quality Connect: Atrial Fibrillation, funded by Bristol–Myers Squibb, outside of the submitted work. EG, JP, SE, DS, and BR stated that there are no conflicts of interest to disclose.

Author Contributions

EG, JP, SE, BR, and AD contributed to the conception and design of the study. DS, HH, EG, JP, and SE procured data. HH, EG, JP, SE, DS, AD, BR analyzed data, drafted, and reviewed the manuscript.

Acknowledgments

The authors would like to thank The Mount Sinai AppLab for helping to develop the mobile health smartphone application.

They also thank Anne Myrka of IPRO for sharing the ICD-9 to ICD-10 code crosswalk for identifying bleeding complications.

REFERENCES

1. Chang S, Hamilton M, Lees C, Atreja A. Mobile health in IBD: Enhancing care, one phone at a time. Inflamm Bowel Dis. 2020;26:163–6. 10.1093/ibd/izz262

2. Wood J, Jenkins S, Putrino D, Mulrennan S, Morey S, Cecins N, et al. A smartphone application for reporting symptoms in adults with cystic fibrosis improves the detection of exacerbations: Results of a randomised controlled trial. J Cyst Fibros. 2020;19(2):271–6. 10.1016/j.jcf.2019.09.002

3. Stukus DR, Farooqui N, Strothman K, Ryan K, Zhao S, Stevens JH, et al. Real-world evaluation of a mobile health application in children with asthma. Ann Allergy Asthma Immunol. 2018;120(4):395–400. 10.1016/j.anai.2018.02.006

4. Inouye SK. Delirium in older persons. N Engl J Med. 2006; 354(11):1157–65. 10.1056/NEJMra052321

5. Beckman MG, Hooper CW, Critchley SE, Ortel TL. Venous thromboembolism: A public health concern. Am J Prev Med. 2010;38:S495–501. 10.1016/j.amepre.2009.12.017

6. Heit JA. Epidemiology of venous thromboembolism. Nat Rev Cardiol. 2015;12(8):464–74. 10.1038/nrcardio.2015.83

7. Donze JD, Williams MV, Robinson EJ, Zimlichman E, Aujesky D, Vasilevski EE, et al. International validity of the HOSPITAL score to predict 30-day potentially avoidable hospital readmissions. JAMA Intern Med. 2016;176(4):496–502. 10.1001/jamainternmed.2015.8462

8. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. J Chronic Dis. 1987;40(5):373–83. 10.1016/0021-9681(87)90171-8

9. Premier Inc. Complications methodologies. [cited 2020 Februray]. Available from: https://qualityadvisor.premierinc.com/QualityAdvisorInformation/help/WebHelp/Default.htm.

10. Premier Inc. Quality advisor. 2015. Available from: https://qualityadvisor.premierinc.com

11. SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp. Released 2013.

12. Pugliese M, Ramsay T, Shamloul R, Mallet K, Zakutney L, Corbett D, et al. RecoverNow: A mobile tablet-based therapy platform for early stroke rehabilitation. PLoS One. 2019;14(1): e0210725. 10.1371/journal.pone.0210725

13. Wolf A, Fors A, Ulin K, Thorn J, Swedberg K, Ekman I. An eHealth diary and symptom-tracking tool combined with person-centered care for improving self-efficacy after a diagnosis of acute coronary syndrome: A substudy of a randomized controlled trial. J Med Internet Res. 2016;18(2):e40. 10.2196/jmir.4890

14. Johnson N, Bodegard J, Jerstrom S, Åkesson J, Brorsson H, Alfredsson J, et al. Effects of interactive patient smartphone support app on drug adherence and lifestyle changes in myocardial infarction patients: A randomized study. Am Heart J. 2016;178:85–94. 10.1016/j.ahj.2016.05.005

15. Neumayr C, Voderholzer U, Tregarthen J, Schlegl S. Improving aftercare with technology for anorexia nervosa after intensive inpatient treatment: A pilot randomized controlled trial with a therapist-guided smartphone app. Int J Eat Disord. 2019; 52(10):1191–201. 10.1002/eat.23152

16. Atreja A, Khan S, Rogers J, Otobo E, Patel NP, Ullman T, et al. Impact of the Mobile HealthPROMISE Platform on the quality of care and quality of life in patients with inflammatory bowel disease: A study protocol of a pragmatic randomized controlled trial. JMIR Res Protoc. 2015;4(1):e23. 10.2196/resprot.4042

17. Jibb LA, Stevens BJ, Nathan PC, Seto E, Cafazzo JA, Johnston DL, et al. Implementation and preliminary effectiveness of a real-time pain management smartphone app for adolescents with cancer: A multicenter pilot clinical study. Pediatr Blood Cancer. 2017;64(10):ePublished. 10.1002/pbc.26554

18. Allen JK, Stephens J, Himmelfarb CRD, Stewart KJ, Hauck S. Randomized controlled pilot study testing use of smartphone technology for obesity treatment. J Obes. 2013;151597. 10.1155/2013/151597

19. Davidson TM, Bunell BE, Saunders BE, Hanson RF, Danielson CK, Cook D, et al. Pilot evaluation of a tablet-based application to improve quality of care in child mental health treatment. Behav Ther. 2019;50(2):367–79. 10.1016/j.beth.2018.07.005

20. Weisman O, Schonherz Y, Harel T, Efron M, Elazar M, Gothelf D. Testing the efficacy of a smartphone application in improving medication adherence, among children with ADHD. Isr J Pyschiatry. 2018;55(2):59–63. 10.26226/morressier.5971be87d462b80290b534d4

21. Le Marne FA, Butler S, Beavis E, Gill D, Bye AME. EpApp: Development and evaluation of a smartphone/tablet app for adolescents with epilepsy. J Clin Neurosci. 2018;50: 214–20. 10.1016/j.jocn.2018.01.065

22. Pew Research Center. Device ownership over time. Pew Res Center’s Internet Am Life Proj. 2018. [cited 2019 Nov]. Available from: http://www.pewinternet.org/datatrend/mobile/device-ownership/

Appendix A. List of ICD 9 Codes associated with venous thromboembolism.

ICD 9 Code Diagnosis
451.1 Phlebitis and thrombophlebitis of deep veins of lower extremities
451.2 Phlebitis and thrombophlebitis of lower extremities, unspecified
415.1 Pulmonary embolism and infarction
451.8 Phlebitis and thrombophlebitis of other sites
451.9 Phlebitis and thrombophlebitis of unspecified site
453.2 Other venous embolism and thrombosis of inferior vena cava
453.4 Acute venous embolism and thrombosis of deep vessels of lower extremity
453.8 Acute venous embolism and thrombosis of other specified veins

Appendix B. List of ICD 9 and 10 codes associated with bleeding complications.

ICD 9 Code Diagnosis ICD 10 Code
V582 Blood transfusion without reported diagnosis
2463 Hemorrhage and infarction of thyroid E07.89
2800 Anemia secondary to blood loss
2851 Acute posthemorrhagic anemia D62
2865 Hemorrhagic disorder due to intrinsic circulating anticoagulants D68.32
36281 Retinal hemorrhage H35.6
3636 Choroidal hemorrhage and rupture H31.3
37272 Conjunctival hemorrhage H11.3
37481 Hemorrhage of the eyelid
37632 Orbital hemorrhage H05.23
37742 Hemorrhage to the optic nerve sheaths H47.02
37923 Vitreous hemorrhage H43.1
4230 Hemopericardium I312
430 Subarachnoid hemorrhage I60, I62
431 Intracerebral hemorrhage I61
432 Others and unspecified intracranial hemorrhage I62
53021 Esophageal ulcer with bleeding K22.11
53082 Other specified diseases of esophagus, hemorrhage NOS K22.8
4560 Esophageal varices with bleeding I8501
4590 Hemorrhage unspecified R58
5310 Acute gastric ulcer with hemorrhage K25.0
5312 Acute gastric ulcer with hemorrhage and perforation K25.2
5314 Chronic or unspecified gastric ulcer with hemorrhage K25.4
5316 Chronic or unspecified gastric ulcer with hemorrhage and perforation K25.6
5320 Acute duodenal ulcer with hemorrhage K26.0
5322 Acute duodenal ulcer with hemorrhage and perforation K26.3
5324 Chronic and unspecified duodenal ulcer with hemorrhage K27.4
5326 Chronic and unspecified duodenal ulcer with hemorrhage and perforation K27.6
5330 Acute peptic ulcer with hemorrhage K27.0
5332 Acute peptic ulcer with hemorrhage and perforation K27.2
5334 Chronic or unspecified peptic ulcer with hemorrhage K27.4
5336 Chronic or unspecified peptic ulcer with hemorrhage and perforation K27.6
5340 Acute gastrojejunal ulcer with hemorrhage K28.0
5342 Acute gastrojejunal ulcer with hemorrhage and perforation K28.2
5344 Chronic or unspecified gastrojejunal ulcer with hemorrhage K28.4
5346 Chronic or unspecified gastrojejunal ulcer with hemorrhage and perforation K28.6
53501 Acute gastritis with hemorrhage K29.01
53511 Atrophic gastritis with hemorrhage K29.4
53521 Gastric mucosal hypertrophy with hemorrhage
53531 Alcoholic gastritis with hemorrhage
53551 Unspecified gastritis or gastrojejunitis with hemorrhage K29.5
53541 Other specified gastritis with hemorrhage K29.61
53561 Duodenitis with hemorrhage K29.81, K29.91
53783 Angiodysplasia of stomach and duodenum with hemorrhage K31.81
58784 Dieulafoy lesion (hemorrhagic) of stomach and duodenum
56202 Diverticulosis of small intestine with hemorrhage K57.11, K57.13
56203 Diverticulitis of small intestine with hemorrhage K57.01
56212 Diverticulosis of colon with hemorrhage K57.31, K57.33
56213 Diverticulitis of colon with hemorrhage K57.21
Diverticulitis of both small and large intestine with bleeding K57.41
Diverticulosis of both small and large intestine with bleeding K57.51, K57.53
Diverticulitis unspecified with bleeding K57.81
Diverticulosis unspecified with bleeding K57.91, K57.93
56881 Hemoperitoneum K66.1, K66.9, K68.9
5693 Hemorrhage of rectum and anus K62.5
56985 Angiodysplasia of intestine with hemorrhage K31.811
Hematemesis K92.0
Gastrointestinal hemorrhage NOS K92.2
Melena (blood in stool) K92.1
578 Gastrointestinal hemorrhage
5967 Hemorrhage into bladder wall
59970 Hematuria R310, R31.2, R31.9
59971 Gross hematuria
6262 Excessive or frequent menstruation N92.0, N92.4, N92.6
6268 Other uterine hemorrhage N92.5
6269 Unspecified uterine hemorrhage N93.8
7191 Hemarthrosis M25.0 pg646
7847 Epitaxis R04.0
7848 Hemorrhage of the throat R04.1
7863 Hemoptysis R04.8
Hemorrhage not elsewhere classified R58
E9342 Adverse event related to anticoagulants