CRD Researchers Using Deep Learning to Help VA Identify Suicide Risk in Veterans

Through the MVP-CHAMPION partnership, the DOE and VA are working to apply supercomputing, networking, and software development resources at several national laboratories to medical data sets.

Researchers in the Computational Research Division (CRD) at Lawrence Berkeley National Laboratory (Berkeley Lab) are applying deep learning methods and data analytics to electronic health record (EHR) data to help the Veterans Administration (VA) address a host of medical and psychological challenges affecting many of the nation’s 700,000 military veterans.

The project is part of a collaboration between the U.S. Department of Energy (DOE) and the VA that combines the VA’s vast EHR system with DOE’s high performance computing, artificial intelligence, and data analytics resources. Through the Million Veteran Program-Computational Health Analytics for Medical Precision to Improve Outcomes Now (MVP-CHAMPION) partnership, announced in May 2017, the DOE and VA are working to apply supercomputing, networking, and software development resources at several national laboratories to medical data sets collected by the VA from some 700,000 veterans and EHR data from another 22 million veterans. The initial focus of this program is on suicide prevention, prostate cancer, and cardiovascular disease.

“The Energy Department is notably helping our veterans by using the world-class artificial intelligence and supercomputing capabilities at our National Labs to address suicide risks, traumatic brain injury, opioid addiction, and a number of other critical areas,” said U.S. Secretary of Energy Rick Perry. “I am truly pleased that DOE is providing such strong support to President Trump’s 'call to action' to empower veterans and end the national tragedy of veteran suicide.”

Berkeley Lab’s Computing Sciences organization first became involved with the VA’s Million Veteran Program (MVP) Suicide Prevention Exemplar project in 2018 to help address an alarming statistic: Suicide is the 10th leading cause of death in the U.S., and it is significantly higher in the veteran population, with 20-22 deaths per day. The Berkeley Lab team’s goal is to improve identification of patients at risk for suicide through new patient-specific algorithms that can provide tailored and dynamic suicide risk scores — such as whether a person who has been in the hospital for a suicide attempt will attempt it again within 30 days — and make these resources available to VA caregivers and patients.

“The VA has been collecting medical records and genomic data from some 700,000 veterans, and they need help from DOE to interpret all of this information to improve healthcare for these individuals,” said Silvia Crivelli, a computational biologist in CRD who has been spearheading Berkeley Lab’s involvement in the suicide prevention project.

In 2018, five college students who spent their summer at the lab through the Computing Sciences (CS) summer internship program signed on to the project: Rafael Zamora-Resendiz (now a CS staff member), Shirley Wang, Shahzeb Khan, Cheng Ding, and Ryan Kingery, who is a U.S. veteran himself. For three months, working with Crivelli and Xinlian Liu — an associate professor of computer science at Hood College who also spent the summer at Berkeley Lab as part of the same program — the students developed algorithms to do statistical analysis of EHRs to look for key factors related to suicide risks, applying deep learning methods to these large and complex datasets. Working with a publicly available dataset (MIMIC-III) that contains medical record information on about 40,000 patients from one Boston hospital intensive care unit, they searched for patterns that might point to suicide risk. (Liu presented an overview of the group’s work at the recent SIAM-CCSE meeting, and their work was also featured in an article on the March 1, 2019 SIAM news blog.)

But EHR datasets are very noisy and contain both structured data (such as demographics, prescribed medications, lab work, and procedures) and unstructured data (such as handwritten doctors’ notes and discharge notes). As a result, some of the team’s early efforts focused primarily on finding patterns in this diverse and complex information; for example, Zamora-Resendiz initially worked to build a deep learning network that can distinguish and classify patients at high risk for suicide from discharge notes and physicians’ notes in these datasets.

“We first trained the neural network to classify between patients who are at high risk and those who are not to find patterns in the doctors’ language,” said Zamora-Resendiz. “We then applied some techniques on the trained network to find which words contributed most to the final prediction. The real challenge is figuring out a way of tracing how these words are combined internally within the network. This will help provide better insight on common motifs found between suicidal patients.”

This image shows a visualization of word embeddings learned by a deep learning model trained for suicide-related admission classification. The graph shows the top 500 words used by the network to classify between written patient discharge summaries. The researchers used a T-distributed Stochastic Neighbor Embedding (T-SNE) algorithm — a dimensionality reduction machine learning algorithm for visualization — to help explore semantic groupings of words. A subsection of the graph shows words have organized according to common semantic value pertaining to medical conditions.

Although the performance of these neural networks is impressive, they are difficult to interpret. “We are aware that we need to gain the trust of the physicians, and that’s why we are working on developing models that are interpretable,” said Crivelli. Incorporating the physician’s knowledge will improve the accuracy and defensibility of scientific deep learning models. Currently, interpretability is the focus of several efforts, including Berkeley Lab’s Machine Learning for Science strategic initiative, which is also tied to foundational work in learning described in the recent report, “Basic Research Needs Workshop for Scientific Machine Learning Core Technologies for Artificial Intelligence.”

Since last summer, the team has continued to work with the MIMIC dataset to fine-tune their understanding of how natural language processing can be used to sift through the structured and unstructured data collected in an EHR, and Liu is heading back to Berkeley Lab this summer with more students to focus again on this project. According to Crivelli, a few individuals are also about to gain approval to use the MVP dataset — which contains more data than the MIMIC dataset — to further extend this research, especially for suicide risk.

“We believe that, for suicide prevention, the unstructured data will give us another side of the story that is extremely important for predicting risk — things like what the person is feeling, social isolation, homelessness, lack of sleep, pain, and incarceration,” she said. “This kind of data is more complicated and heterogeneous, and we plan to apply what we have learned about deep learning and natural language processing through our MIMIC III research to the MVP data to help VA doctors better decide who is at high risk and who they need to reach out to. They cannot do it alone because they do not have the tools we have at the DOE.”

Data-driven scientific discovery is poised to deliver breakthroughs across many disciplines, and the DOE, through its national laboratories, is well positioned to play a leadership role. Deep learning methods represent a promising approach for analytics in science for discovering subtle patterns in very complex scientific data of all kinds, although more work is needed gain to confidence in life-critical applications such as suicide prevention. The use of these methods for natural language processing are expected to have a significant impact on energy applications that involve human interaction, in addition to veterans’ healthcare.