User Engagement With a Mobile Health App for People Living With HIV: Observational Study Based on an Engagement Evaluation Framework

Abstract

Background: Mobile health (mHealth) interventions can be effective for people living with HIV, who are sensitive to privacy breach risks. Understanding the perceived experiences of intervention participants can provide comprehensive insights into potential users and predict intervention effectiveness. Thus, it is necessary to plan engagement measurement and consider ways to enhance engagement during the app development phase.

Objective: This study aims to evaluate engagement with a mHealth app using the engagement index and time and to examine differences in engagement according to participant characteristics.

Methods: This observational study, conducted from March 27 to August 31, 2024, was based on an engagement evaluation framework. A total of 261 people living with HIV across 13 medical institutions and 3 related organizations in South Korea engaged in an app-based intervention for 4 weeks. Self-reported surveys were conducted before and after the app usage period, and usage data were collected during the intervention to assess user engagement. User engagement was evaluated using 2 measures: engagement index and engagement time. The engagement index represents a standardized percentage derived from 5 subindices: click depth, loyalty, recency, feedback, and interaction.

Results: The median engagement index and time were 67.6% (IQR 59.7-74.1) and 27.2 minutes (IQR 4.9-81.4), respectively, with a statistically significant positive correlation between the two (ρ=0.589; P<.001). The engagement index was higher among those with self-help group participation (U=6014.0, z=-2.49; P=.01). There were no differences in engagement index and time according to other personal characteristics.

Conclusions: When developing mHealth apps for people living with HIV, it is essential to track both objective indicators such as login data and subjective indicators such as patient experience for comprehensive intervention evaluation. Based on this study's results, high engagement suggests that apps should prioritize user needs through rewards, privacy protection, tailored information, and esthetic features. While the app in this study demonstrated inclusive usability, targeted support strategies may benefit users without self-help group participation.