A team from Nationwide Children’s Hospital describes a machine learning technique for timely identification of hospitalized children at risk for deterioration—a worsening clinical condition with higher risk of morbidity and mortality. The technology, which is trained on disease-specific groups, showed promisingly better results in detecting at-risk kids than the current situational awareness program.
Over the years, there has been an increase in the development of predictive algorithms aimed at enhancing clinical treatment, although the majority have not yet been operationalized. According to Laura Rust, MD, an emergency medicine physician and physician informaticist at Nationwide Children’s and the paper’s lead author, Translating the algorithm from the computer to the bedside can be a lengthy process and requires engagement and collaboration from clinicians, data scientists, and clinical informaticists. They are really proud of the project’s effective integration into the safety culture and the impact on patient outcomes, which has taken place over the course of more than five years.
The Watchstander situational awareness program, already in use at Nationwide Children’s, served as the basis for the development of the Deterioration Risk Index (DRI). The team used the same alert response processes for alerts—patient evaluation and a 30-minute huddle with the bedside care team, risk reduction, and creation of an escalation plan—to encourage uptake.
Three distinct predictive models were trained using the three diagnostic categories—structural heart defect (cardiac), oncology (malignancy), and general (neither cardiac nor malignancy)—to create the algorithms used.
According to Tyler Gorham, a data scientist in Nationwide Children’s Hospital’s IT Research & Innovation department and a co-author of the study, Their algorithm simply determines which existing situational awareness criteria are most essential and weighs them accordingly.
Dr. Rust claims that processing a large volume of clinical data from the electronic health record at once, particularly following handoffs or transitions of care, might be challenging. By analyzing these risk criteria automatically in the background, the model aids in reducing this cognitive load. It has the advantage of having all the data from all prior periods in time—not just the present shift—because it is integrated within the electronic medical record (EMR).
The DRI required 2.3 times fewer alarms per detected event and was 2.4 times more sensitive than the current situational awareness program. Notably, the team saw a three-fold increase in sensitivity for malignancy and a four-fold increase in sensitivity for the cardiac group. After implementation, the pilot research discovered that, relative to anticipated event rates in prior years, degradation events fell 77% over the first 18 months.
The developers believe that the transparency of the model is maybe its most significant feature.
This isn’t a mystery box. We demonstrate to clinicians how the algorithm processes the data and sets off alarms, according to Gorham. The ability for the clinical team to understand why an alarm was triggered is one way that the tool supports clinical decision making.
Additionally, the team conducted road shows, visiting the clinical facilities where the tool would be used, responding to inquiries, running simulations with the bedside care teams, and taking comments into account.
The staff at Nationwide Children’s are dedicated to a Zero Hero safety culture, according to Dr. Rust. This served as the framework and unifying goal for their multidisciplinary team to see this through to completion.
The paper contains additional specifics, including information on the algorithm.
According to Gorham, They shared their recipe in the publication. If others are interested, they may retrain the model for their local demographics using the data from their center. They can potentially encourage improved outcomes for all children, including those who are not in their custody, if they can communicate its success to others.
