NYU Langone Health LL.M. Can Predict Hospital Readmissions

Researchers at New York University’s Langone Health Academic Medical Center developed a voluminous language model, now deployed at three of its hospitals, that predicts a patient’s risk of 30-day readmission and other clinical outcomes. does.

why it matters

coincides with its study published in Nature This week, the release of the NYUTron model’s code base to GitHub will enable other healthcare organizations to train their own LLMs and provide doctors with insights that can help them identify which patients to reduce readmissions. may require intervention.

This model has been used to evaluate 50,000 patients discharged from NYU’s healthcare system. NYUTron shares its prediction of readmission risk with physicians by email.

NYU partnered with NVIDIA to develop and run its LLM on several of the company’s artificial intelligence platforms – its stack, libraries and software.

“Not all hospitals have the resources to train large language models from scratch, but they can take a pre-trained model like NYUTron and then fine-tune it with a small sample of local data using GPUs in the cloud. Can,” Dr. Eric Orman, assistant professor of neurosurgery, radiology and data science at NYU Langone Health, said in a blog post on NVIDIA’s website.

“It’s within reach of almost everyone in healthcare,” he said.

“Orman’s team found that after pre-training their LLM, fine-tuning it with data from a specific hospital helped boost accuracy,” NVIDIA said in its announcement.

NYUTron was pre-trained on 10 years of health records from NYU Langone Health, which is more than four billion words of clinical notes representing approximately 400,000 patients. Orman said the team is using medium-sized models trained on highly sophisticated data to perform health-specific tasks.

The team then developed four other algorithms that predict the length of a patient’s hospital stay, the likelihood of in-hospital mortality, and the likelihood of a patient’s insurance claims being denied.

In a webinar last year, the researchers said their approach was to treat reading comprehension prediction as a natural language processing task, with health-system-scale pre-processing of clinical text on high-end multi-node GPU servers. Involved in the creation of trained LL.M.

As NYUTron was developed, he said he aimed to address the following questions:

• How to handle long sequence length?
• How to address label imbalance?
• How to measure the impact of noisy labels on model evaluation?

First, they collected “a massive set of unlabeled clinical notes from the NYU Langone EHR and five task-specific labeled clinical notes,” before training, refining, deploying the model, and testing it in a real-world environment. , the researchers said. Report.

“On smaller samples, Neutron was competitive with a smaller set of clinicians in predicting 30-day readmission,” he said.

Testing a group of six physicians at different levels of seniority against the NYUTron in a head-to-head comparison, they said they established a baseline difficulty predicting all-cause readmission at the time of discharge.

“Median physician performance was worse than NYUTron,” the researchers said.

“For physicians and NYUTron, the average false positive rate was 11.11%, while the average false positive rate for physicians was 50%, compared to 81.82% for NYUTron. Physicians’ average F1 score (a machine learning evaluation metric that allows a model measures accuracy) of 62.8% and a substantial variance of 22.2% compared to NYUTron, which had an average F1 score of 77.8%.

At an advance press briefing on Tuesday, Orman also said that NYU Langone Health is considering licensing its models to organizations that don’t have the resources to build them from scratch.

The next step for Orman’s team is a planned clinical trial to test whether the intervention reduces readmission rates, based on Neutron’s analysis.

big trend

research published in Multidisciplinary Healthcare Journal said last year that about 15% of all hospital patients are readmitted to the hospital within 30 days after initial discharge.

Readmission rates are affected by countless variables—including five common treatments in emergency departments—that not only affect overall patient care, but can also take away beds and resources from patients who need more intensive care. Health care may be required.

“Industry research as well as our own experience suggests that 20% of readmissions are preventable,” said Teresa Radford, clinical program coordinator at University of Virginia Health.

He said Healthcare IT News After discovering in December that UVA Health’s 30-day readmission rate for patients with complex and costly medical conditions was as high as 17% to 18% per year, the healthcare provider created a hospital-at-home program , which reduced hospitalization and readmission. 46%.

A multidisciplinary team of physicians, nurses, and mental health professionals created an intensive care plan for each individual who had at least one chronic illness or behavioral condition that required frequent UVA Health hospital visits and access to hospital-based services. Contributed to higher utilization.

On the record

“While there have been computational models for predicting patient readmission since the 1980s, we are treating this as an (NLP) task for which a healthcare systems-scale corpus of clinical text are needed,” Orman said in the announcement.

“We trained our LLM on unstructured data from electronic health records to see if it can capture insights that people haven’t considered before.”

Andrea Fox is a senior editor for Healthcare IT News.
Email: afox@himss.org
Healthcare IT News is a HIMSS Media publication.