• Alert Fatigue

    Researchers look to reduce 'noise' of alerts to make care safer

    The move from paper charting to electronic health records, although an enormous improvement in efficiency and safety in patient care, has not been without its challenges.

    One of the features of electronic recordkeeping is computerized provider order entry (CPOE), in which providers enter orders for medications and tests into a computer. CPOE is designed to make ordering easier for doctors and safer for patients. But its assistance is not always welcome.

    When doctors enter an order for medication that the system perceives as a potential error, it sends an alert.

    “We know that our providers are overriding alerts 90 percent of the time,” says Eric Kirkendall, MD. “Nine times out of 10, people are making the judgment that the information they are being presented is not useful or accurate.”

    Kirkendall knows this because he is part of a team working to improve the electronic ordering system so that it becomes a truly useful tool for doctors instead of a nuisance.

    The project is being led by Andy Spooner, MD, a pediatrician and Chief Medical Information Officer in the Division of Biomedical Informatics at Cincinnati Children’s. He, Kirkendall, and Michal Kouril, PhD, a computer scientist in Biomedical Informatics, are tackling what Spooner says is a problem that goes beyond healthcare and pediatrics.

    “The problem is information overload,” Spooner says. “We design systems with information that is supposed to help us, but it becomes too much to process. So when we do things like set up alerts for dose of a drug, people in their overwhelmed state ignore it.”

    Not designed for kids

    The problem is even worse in pediatrics. Dosing decision support systems have largely been designed around adult patients and are sometimes a poor fit for children. Adapting the systems to a pediatric environment, where medication dosing is determined largely by weight, has resulted in an even greater number of pop-up messages.

    The result is “alert fatigue” – a phenomenon in which doctors become so inured to alerts that they ignore them altogether, whether they are delivering helpful information or not.

    When a doctor enters an order for medication into the hospital’s electronic medical record system, it scans its vast collection of rules – its “clinical decision support” (CDS) - for allergies, interactions, and dosing irregularities. If the doctor enters an order at a dosing level the system does not deem appropriate, it sends up an alert challenging the dose. The system is designed so that doctors can override the alerts, which they often do.

    The question for our research is, how much risk are we exposing patients to by ignoring the decision support?” Spooner asks. 

    “Is it truly noise or are we ignoring important things?”

    Defining 'Reasonable'

    To find out, Kirkendall and Kouril have spent the better part of the past year analyzing millions of order entries in our system.

    “We’re trying to measure what happens when people place orders and are presented with alerts, and what reasonable rates of alerts should be,” Kirkendall says. “We think that overriding alerts 90 percent of the time is not appropriate. So how can we tone the alerts down and get people to heed the system?”

    Developing new rules

    One way of toning down the alerts has been the time-consuming process of creating custom dosing rules, says Kouril. Many medications in common use have not been approved by the FDA for use in children, so there are no approved dose ranges built into the system, resulting in high numbers of false alerts.

    As a result, Kouril says, Cincinnati Children’s Information Systems’ team, including pharmacist Tom Minich, RPh, had to create thousands of new dosing rules.

    “These custom rules supersede the rules provided by the system,” Kouril says. “We chose them because they are high-risk medications – the ones most important to get right.”

    The researchers had to comb through the data to determine real alerts from what Kouril terms “false positives.”To do that, they focused on the big overdose alerts - variations of as much as 500 to 10,000 percent or more in excess of what the computer thinks are appropriate dosing amounts.

    Real mistakes – or not? 

    “We are worried about what appear to be big overdoses,” Kouril says. “How many of them actually are errors or were they intentional overrides? And if they were errors, did we catch them, and where?”

    The team narrowed their review to the 20 medications most frequently entered into the system as overdoses. Were they actual mistakes in ordering, or were they simply doctors prescribing doses outside the range limits in the database?

    “We believe most of these so-called ‘overdoses’ are actually intentional overrides by prescribers,” Kirkendall says. “And we should be able to analyze the information to make changes to the system that would prevent them from being [recognized as] overdose orders in the first place.” 

    Once the researchers complete their data gathering and analysis phase, they will publish their findings. The information could help fill the current enormous literature gap in CDS related to pediatric dosing.

    Their next step will be to develop a simulation system based on their findings.

    “We are constructing an analytic framework that will model what a real system does,” Kirkendall says. “Before we put it into practice, we want to model simulations to see how it will work. We have to study it to understand true user behavior and test it to see if what we are developing is as safe as possible.”

  • Andy Spooner, MD.
    Dr. Andy Spooner is leading a project to lighten information overload during order entry.
  • The Gap Between Rules and Practice

    A major reason for the unusually high rate of alerts in pediatric medication ordering is that vendor-supplied electronic rules (“eRules”) are built around prescribing for adults, not children. In a study conducted at Cincinnati Children’s, Drs. Andy Spooner and Eric Kirkendall looked at seven months of medication orders and alerts for 30 medications, across five age ranges and five dosing parameters. Their findings showed that the electronic dosing rules were inaccurate for children more than half the time. They found even greater variability among dosing rules for newborns. The study was published online in June in the Journal of the American Medical Informatics Association (JAMIA). Kirkendall, Spooner and others at Cincinnati Children’s continue their research into the effects of these discrepancies on safe prescribing for children.