Increasing Retention in Care for HIV Patients
Partner(s): Chicago Department of Public Health/University of Chicago Medicine
Status: In development
Team: Avishek Kumar, Arthi Ramachandran, Hannes Koenig, Joseph T. Walsh, Adolfo De Unanue, Christina Sung, Jessica Ridgway, Rayid Ghani
Summary
People living with HIV (PLWH) who do not receive regular medical care have decreased quality of life, higher mortality, and are more likely to transmit HIV to others than PLWH who are retained in care. Clinical staff typically identify PLWH after they have missed appointments, or use their expert judgment to try and identify patients at risk for retention in care failure, leaving a lot of room for improvement. This project was focused on creating machine learning model to assess the risk of a patient dropping out of care as well as their risk factors in order to effectively target interventions and intervene with an effective collection of services.
We developed machine models to use in two types of scenarios:
1. A point-of-service machine learning system: during a patient’s medical appointment, this system will use a machine learning model to assess the risk of an HIV+ patient not returning for their next appointment and the associated risk factors to help target personalized interventions.
2. The second model is a prioritization tool a health department can use to identify who is most at risk of dropping-out-of-care at a city-wide level so they may efficiently and effectively deploy their services on a regular frequency instead of at the time of an appointment.
Background
A major cause of the spread of HIV and quality of life reduction due to the virus is poor retention in care. The current state-of-the-art treatment is antiretroviral therapy (ART), which suppresses viral RNA in an infected host. ART requires assiduous adherence to both a daily medical regimen and retention in care. Because virally suppressed individuals are unlikely to transmit HIV, retaining individuals in care is critical to ART adherence and ultimately controlling the spread of the HIV virus. Failure to adhere to the ART schedule can lead to a compromised immune system, drug resistance, and a higher probability of transmitting the virus to others. Existing research on this problem has focused on two aspects: 1) Using retrospective analysis to identify population level, coarse, subgroups at high risk of dropping out of care, such as African-American men who have sex with other men, and 2) understanding root causes and barriers to retention in care, such as mental illness, insufficient means of transportation, lack of insurance, transient life circumstances, disruption of social and sexual networks, criminal justice, unemployment, geography and underutilization of antiretroviral therapy. There are two problems with these approaches: 1) using retrospective analysis to find coarse, at-risk subgroups is useful in describing the at-risk groups, but not useful in proactively targeting resources at the time of service, and 2) targeting interventions using group level risk factors waste scarce resources because a) group-level risk scores do not account for all individual circumstances and behaviors and are hence less accurate than individual risk assessments, and b) risk based on a group presumes that all members have uniform risk, making it impossible to distinguish high risk members from low risk ones. Our hypothesis is that developing a system that is both proactive and predicts risks and associated factors at an individual level would allow health providers to 1) assess an individual’s risk of not returning for their next visit at the time of their current appointment and 2) allocate intervention resources efficiently.
Scope
This project is with two different partners: The Chicago Department of Public Health (CDPH) and University of Chicago HIV clinic (UCM). In partnership with CDPH we have developed a city-wide machine learning model that assesses the risk of an HIV+ person in the city of Chicago dropping out of care as well as personalized risk factors to prioritize who should receive an intervention from CDPH. In partnership with UCM we have developed a point-of-service HIV model to assess a risk of a patient dropping-out-of-care as well as personalized risk factors at the time of their appointment at the University of Chicago HIV clinic so a social worker may launch an intervention before they drop out of care.
Data
University of Chicago Medicine:
The data used in the model includes the entire electronic medical record of all patients that visit the University Chicago HIV clinic, including appointment history with any physician at the University of Chicago; lab tests, viral load, CD4, drug tests; demographic data; and medical provider data. Insurance records and the American Community Survey data are also included in the data.
Chicago Department of Public Health:
The city-wide models uses HIV surveillance data collected by the Chicago Department of Public Health, including demographic data, lab tests, and location data.
Analysis
The data-driven early-warning system (EWS) take all available data and uses machine learning methods to detect patterns that precede patients dropping out-of-care, making the EWS predictive and facilitating the prioritization and informing proactive, individual intervention. Our EWS analyzed thousands of permutations of variables over different time periods to determine which combination of predictive variables and machine learning algorithm is most predictive of finding patients who will drop out-of-care in the future.
Results
University of Chicago Medicine:
In our models, we improve upon an expert baseline model by 5-10 percentage points. While this is a practically significant improvement, the impact on human life is much broader. There are over 20,000 PLWH (Persons Living with HIV) in Chicago which translates to 2000 individuals as the top 10% individuals at-risk . A 5-10 percentage points improvement in identifying the most at-risk patients results in identifying an additional 100-200 individuals for outreach.
Future Plans and Areas for Improvement
Our system also provides insight into which factors are important in predicting whether patient will be retained in HIV care or not. We find that largely intuitive features such as their lab test results and their appointment history are important in predicting whether someone will drop out of care. However, other common factors, substance abuse history and appointments that they have missed are not as predictive though thought to be important by domain experts. Bias analysis of our model shows that while there is still bias against certain protected groups, we show reduction in bias compared to the expert heuristics-based model.
We plan to extend the system to include complex features (such as text features from the clinical notes). A web application is currently in development to record expert feedback and factors for whether an individual will drop out of care.This system is planned to be deployed in the UChicago HIV Clinic, and extended to the entire city of Chicago for use by the Chicago Department of Public Health.
