OHDSI Home | Forums | Wiki | Github

Potential follow-up/study on prediction of pneumonia severity from PLP tutorial?

Researchers
#1

Hi everyone! I’m following up to gauge the community’s interest in turning that jaw-dropping/inspirational live demo of the Patient Level Prediction tool from this year’s Symposium tutorial into an official study. I heard rumors that people are interested in continuing the momentum to further explore this work. So, since I initially tossed out that idea at the tutorial, I thought I’d try to get the conversation started here too!

For those who weren’t at the tutorial or just as a reminder, we attempted to create a model to predict: among adult patients admitted with pneumonia, who will have an ICU stay during their hospitalization?

The on-the-fly model created in 30 min did remarkably well (AUC of 0.88 or so), but we later realized it included all covariates, not just the ones from before Day 0 of admission. But it showed the incredible power and potential of this tool

The larger, more general topic might be predicting pneumonia severity, and we could potentially discuss several outcomes of interest (readmission being another high-value/popular outcome). Some next steps might be a discussion around the clinical question(s) we want to answer and then more clearly defining our cohorts, outcomes, covariates, and the model(s) we want to use.

I’ll start by saying that I was initially inspired by an ED physician who cited the PSI score for predicting severity/adverse outcomes as a reason a patient should be admitted to my team. Looking at the primary literature, I quickly realized the score is from 1997 (20 years ago!). This, along with the CURB-65 score (15 years ago), form the 2 main scores that doctors use to predict how sick someone with pneumonia might get.

I look forward to hearing all your thoughts and ideas! Or just let us know if you’re interested in participating!

#2

Thanks a lot Ray for pushing this! It was great for us to experience so much positive feedback on the PLP tutorial and the PLP pipeline that has been developed.

Happy to help out with your initiative!

I hope this will stimulate others to lead PLP studies in the OHDSI data network, we can do some important work as a community in this area!

Peter

#3

Ray / Peter - our team (Odysseus) would be very interested in joining this effort as well, both from PLP code development and facilitating the federated execution with ARACHNE platform perspectives

#4

Hi Ray - I’m interested in helping out and have bounced the notion this kind of work off of the folks at Stanford.

#5

Glad to hear everyone’s interest and enthusiasm! @Evan_Minty do you know if the Stanford folks would want to join forces and collaborate on this endeavor?

In terms of next steps, I’m hoping to define the research question that we want to study. For now, I’m inclined to go with what we had mentioned above: severity and/or readmissions. So for severity, it would be: for adult patients admitted with diagnosis of pneumonia, who will have an ICU stay or die during their hospitalization. For readmissions, it would be the same cohort, but looking at who will be readmitted within 30 days?

Any other suggestions or thoughts on which ones of these we should focus on first? How do people feel about the quality of mortality or readmission data in their databases?

#6

@rchen

I’m not sure why readmission is a popular outcome. From a medical point of view, severity may be defined by mortality, length of stay in the ICU, and complications of pneumonia such as acute renal failure. On the other hand, if pneumonia is severe it simply won’t last for 30 days: it will either resolve or progress to an abscess, pleural empyema etc.

BTW, PSI is working quite well when it comes to clinical practice.

#7

Thanks Anna, great points. Readmission is often a popular outcome, and specifically 30 day readmission, because it’s tied to reimbursement here in the US. That’s why there’s such an emphasis on studying it (for better or for worse). It’s a publicly reported measure and Medicare (coverage for our elderly) will penalize hospitals that do poorly and pay less for these visits.

And I wasn’t sure how well PSI is doing clinically, I wasn’t aware of any recent major studies on it in the past decade. But I would love to learn more if you have references that you can share. In the places I’ve practiced, I find that more people use CURB65 than PSI but even then, neither is commonly used. More often it seems to be the hand-waving clinical ‘gestalt’/intuition or local practice patterns that seem to dictate admission and triage.

But in my mind, the way to find out is just to run them all and see if we can do better :smile:

#8

I think what we’re arriving on is two things:

  1. Issues in dealing with temporality.
    a)There may be some dimensions of data (concepts derived from billing codes or procedure codes for example) that are applied to an entire encounter and would need to be excluded. I’m actually not sure how the dates are handled. In working through an ETL locally, we’re creating ghost encounters for some billing data as we don’t necessarily know what encounter they belong to if the record is complex.

bottom line is, if a procedure code for intubation or a diagnosis code of respiratory failure can become a predictor in your model, this won’t be doing anything interesting.

b) Consequently, you’d probably only want data sets with temporal and clinical features on the order of the electronic health record. Don’t know what that limits you to in the network.

  1. (1b) would probably be needed to outperform tools like CURB65, most of the criteria there parameterize those ‘gestalt’ features (confusion, end organ dysfunction / inflammatory response etc ) that probably predict an ICU admission.

You could try to predict ICU admission based solely on previous features in the health record, but you’d lose information that correlates to the severity of their current presentation. That will be a tough ask, but you could give it a shot…

t