NR 500 Week 4 Sepsis Alert Source Evaluation Example

Source Evaluation Write-Up: Four Sources Behind the Claim That Our Sepsis Alert Catches Sepsis Early, and What Each One Actually Is

Student Name

College of Nursing, Chamberlain University

NR 500: Foundational Concepts and Applications

Instructor Name

Month Day, Year

Source Evaluation Write-Up: Four Sources Behind the Claim That Our Sepsis Alert Catches Sepsis Early, and What Each One Actually Is

The Question

My hospital's nursing intranet has a page on the sepsis alert in our electronic health record. It says the alert "identifies patients at risk of sepsis early, so nurses can act before they deteriorate." As an informatics student, I wanted to know what that claim rests on. This write-up evaluates four sources I found while following it, each in the same order: what the source actually is, who produced it and what they had at stake, how far I read into it, and a judgment with a use attached.

Source 1: The Hospital Intranet Page

What it is: an internal education page written by the hospital's clinical informatics team when the alert went live, not a study and not a summary of one. It has no references.

Who produced it and why: the team that implemented the alert, whose goal was to get nurses to respond to it. That is a legitimate purpose, but it means the page was written to encourage use, not to evaluate performance.

How far I read: the whole page, which is short.

Judgment: useful for knowing how the alert is supposed to work on our units and what nurses are asked to do when it fires. Not usable as evidence that the alert works.

Source 2: An External Validation Study

What it is: a peer-reviewed retrospective cohort study in JAMA Internal Medicine evaluating the Epic Sepsis Model, a proprietary prediction model of the kind our alert is built on, at the University of Michigan's academic health system (Wong et al., 2021).

Who produced it and why: researchers at the health system that used the model, with no stated financial interest in the vendor. They wanted to know whether a widely implemented model performed as expected.

How far I read: the full abstract and the results section; I skimmed the methods on how sepsis was defined.

What it found: among 38,455 hospitalizations, the model's area under the receiver operating characteristic curve was 0.63, meaning it was only modestly better than chance at separating patients who developed sepsis from those who did not. It did not identify 67% of the patients who developed sepsis, while generating alerts at its usual threshold for 18% of all hospitalized patients.

Judgment: the strongest source of the four, and one that contradicts the intranet page. It does not prove our alert performs the same way, because our hospital may use a different threshold or version, and because performance can differ between hospitals. But it means the intranet's claim cannot be accepted without local data.

Source 3: The Surviving Sepsis Campaign Guideline

What it is: an international clinical practice guideline, published in Critical Care Medicine, developed by a panel using a formal process for grading evidence (Evans et al., 2021).

Who produced it and why: professional societies in critical care and emergency medicine, to guide sepsis management. The panel is not tied to any vendor.

How far I read: the recommendations on screening and the summary table; not the full document.

Judgment: useful as a standard against which to set our alert. The guideline recommends that hospitals have sepsis screening programs and recommends against using qSOFA, compared with other tools such as SIRS, NEWS or MEWS, as a single screening tool. It does not evaluate any vendor's prediction model, so it cannot tell me whether our alert is good, only what screening is expected to do.

Source 4: The Sepsis-3 Definitions

What it is: the consensus definitions of sepsis and septic shock published in JAMA in 2016 (Singer et al., 2016).

Who produced it and why: an international task force convened by two critical care societies to update the definitions.

How far I read: the definitions and the section on qSOFA.

Judgment: necessary for reading the other sources, because "catching sepsis" depends on how sepsis is defined. The Michigan study used its own composite definition, which may not match how our hospital counts cases. Any local evaluation of our alert will need to state which definition it uses.

A Source Set Aside

A fifth source came up in my search and was set aside. It was a slide deck from a webinar, shared by a colleague, in which a presenter described large reductions in sepsis deaths after an alert was introduced at another hospital. The slides gave a percentage but no numbers of patients, no time period, no definition of sepsis and no description of what else changed at the hospital at the same time. I could not find a published study behind the figure. The deck is not dishonest, but it is not evidence I can weigh: a before-and-after figure without a denominator or a comparison could reflect a new sepsis protocol, extra staff education, a change in how deaths were coded, or chance. I recorded it and noted that I would use it only if the underlying data were published.

Setting a source aside is a judgment too, and it belongs in the write-up. A reader who later meets the same slide deck should know why it was not used, rather than wondering whether it was missed.

Following the Claim Home

The claim on the intranet page traces back to no source at all. The nearest independent evidence, the external validation study, suggests a model of this type can miss most sepsis cases while alerting on nearly one in five patients, which is the opposite of "early" and a recipe for alert fatigue. The guideline and the definitions give the frame for asking the right local question: what is our alert's sensitivity and false alarm rate, measured against a stated definition, in our patients? I have written that question to our informatics manager. Until it is answered, the honest version of the intranet sentence is that the alert is intended to identify patients at risk of sepsis, and that its performance in our hospital has not been reported.

What a Local Answer Would Need

The question I sent to our informatics manager is specific enough to be answered. For a recent period of, say, six months, it asks how many adult inpatients met a stated definition of sepsis, how many of them triggered the alert before the time of diagnosis, how many alerts fired in total, and how nurses responded to them. Those four numbers would give our alert's sensitivity, its false alarm burden and whether it changes nursing action. If the data exist, this is a small project an informatics nurse could lead. If they do not, that is a finding in itself: an alert used across the hospital without local measurement of whether it works.

References

Evans, L., Rhodes, A., Alhazzani, W., Antonelli, M., Coopersmith, C. M., French, C., Machado, F. R., Mcintyre, L., Ostermann, M., Prescott, H. C., Schorr, C., Simpson, S., Wiersinga, W. J., Alshamsi, F., Angus, D. C., Arabi, Y., Azevedo, L., Beale, R., Beilman, G., . . . Levy, M. (2021). Surviving Sepsis Campaign: International guidelines for management of sepsis and septic shock 2021. Critical Care Medicine, 49(11), e1063-e1143. https://doi.org/10.1097/CCM.0000000000005337

Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., Hotchkiss, R. S., Levy, M. M., Marshall, J. C., Martin, G. S., Opal, S. M., Rubenfeld, G. D., van der Poll, T., Vincent, J.-L., & Angus, D. C. (2016). The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA, 315(8), 801-810. https://doi.org/10.1001/jama.2016.0287

Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., Ghous, M., & Singh, K. (2021). External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Internal Medicine, 181(8), 1065-1070. https://doi.org/10.1001/jamainternmed.2021.2626