OHDSI Home | Forums | Wiki | Github

[Patient Level Prediction] Network study to externally validate existing stroke models

Researchers
#1

There is a new network study: https://github.com/OHDSI/StudyProtocolSandbox/tree/master/ExistingStrokeRiskExternalValidation details: http://www.ohdsi.org/web/wiki/doku.php?id=research:external_validation_stroke_models

The aim of this study is to implement a proof of concept showing that the OHDSI standardizations and PatientLevelPrediction package can enable rapid external validation. At present many models are not externally validated within 5 years, but we can develop models using data from one data set mapped to the OMOP CDM and readily share the model across the network, so we should be able to efficiently validate models.

This study has two steps:

  1. Install and check the package is configured correctly + run the target population and outcome cohort creation code to check cohorts are valid across the network.
  2. Run the study - this implements 5 well known stroke prediction models for four different Stroke definitions. Once the result folder is checked, submit the folder (you need to get the key and secret)

We are aiming for a good journal to submit this to and all contributors will be listed as co-authors.

3 Likes
#2

In this issue of the Circulation, the paper predicting CVD risk in HIV patients was published (link).
Like this, there are some subgroups in which traditional risk scoring system is inaccurate or under/overestimate the risk.

Thank you for conducting this remarkable study, @jreps and @Rijnbeek !
I want to join, too

#3

@jreps @Rijnbeek
Most paper using Korean / Taiwanese claim data defined the AF when it was diagnosed once in inpatient setting or twice or more in outpatient setting (Korean paper example, Taiwanese paper example). And we validated this definition in two Korean hospitals.

The traditional risk scores including CHADSVAsc score was developed to predict ā€˜ischemic strokeā€™ in patients without anticoagulations. So we need to censor the patients when they started to use oral anticoagulation or stratify them into several groups. And we need to consider how to handle those with antiplatelet drug.

There are some exemplary studies by using Korean and Taiwanese claim data for this.

#4

Hi Chan,

The current definition should be satisfied if any of these criteria occur:

  • 2 or more diagnoses in outpatient
  • 1 in an inpatient setting
  • 1 in an outpatient setting with an electrocardiogram in the 30 days prior.

The definition currently restricts to first time in history AF diagnosis criteria (which maybe should be removed?) and removes people with prior stroke.

It wasnā€™t currently removing people with prior anticoagulants, but that seems to be a very valid point. How about I remove the first time in history AF restriction (but limit cohort to first AF record satisfying the criteria) and exclude people with prior anticoagulants (concept id 21500803 and descendants)?

#5

Hi - Iā€™m new to the PLP WG and heard about this study on a recent call. I donā€™t have relevant data access, but would be happy to help out in any other ways if needed (interpreting results after submission, writing, etc.) Just let me know.

#6

@jreps , sorry for my misunderstanding the AF cohort definition. The current definition of AF cohort is great.

However, we need to remove people who used anticoaguation not only anytime before the index, but also anytime after the index date and before the stroke (Thatā€™s what the papers you found did).

I donā€™t think removing first time in history AF restriction criteria is necessary.

Could you tell me again the concept_name for ā€˜21500803ā€™? I cannot find this concept_id in Athena.

#7

Friends: This is ETC, a proprietary database. Athena cannot display it, unless you have a license. Use ATC 21600961 ā€œANTITHROMBOTIC AGENTSā€.

#8

Thank you, @Christian_Reich :smile:

#9

Jena,

I studied the protocol and the concept sets are there but pasted as text and somewhat hard to review (or learn from)
.
Also, in the package - there is only SQL cohort code. (not the JSON importable format)

I very much like your more complex definition below

The current definition should be satisfied if any of these criteria occur:

  • 2 or more diagnoses in outpatient
  • 1 in an inpatient setting
  • 1 in an outpatient setting with an electrocardiogram in the 30 days prior.

Would it be possible to post the implementation in Atlas of the definition above. (or better yet all 4 cohort definitions from your package) as either .json files or put them on the public Atlas server?

Similar to a definition like this one: http://www.ohdsi.org/web/atlas/#/cohortdefinition/99322

#10

Sorry, I tried to make these on friday but atlas was running slow.

Here is the revised target pop: http://www.ohdsi.org/web/atlas/#/cohortdefinition/1769063

and the outcomes:

  1. Broad stroke inpatient http://www.ohdsi.org/web/atlas/#/cohortdefinition/1735846
  2. Broad stroke: http://www.ohdsi.org/web/atlas/#/cohortdefinition/1769064
  3. Hemorrhagic stroke http://www.ohdsi.org/web/atlas/#/cohortdefinition/1769065
  4. Ischemic stroke http://www.ohdsi.org/web/atlas/#/cohortdefinition/1769066
1 Like
#11

Great! @jreps.

There are other thin something to consider

  1. Valvular heart disease
    -Originally, the risk score model of AF doesnā€™t target population with valvular heart disease, or valvular AF (ref). Because all patients with valvular AF must take anticoagulant, they would be excluded in this cohort by no-anticoagulant criteria. Still, it would be worth to exclude valvular heart disease (mitral valvular disease, rheumatic heart disease and prosthetic valves).

  2. Antiplatelet
    -Doctors, especially in Asia, tend to prescribe aspirin in low-risk patients to prevent stroke (such as CHADSVasc 1 or 2, female). We need to decide whether to neglect this or not. Since stroke risk reduction from antiplatelet was minimal in AF patients, we can neglect antiplatelet drug.

  3. Anticoagulant after stroke
    -Most papers just excluded patients used anticoagulant. After patients develop stroke, they can start to take anticoagulation. Because any anticoagulation use was excluded in this cohort, even patients started to take anticogulation after stroke would be excluded, too. Sorry, I donā€™t know figure out how to make cohort for this problem by ATLAS. Only solution Iā€™ve found is that making outcome cohort who used anticoagulation. Among target cohort, we can identify patients develop stroke and start anticoagulation. By comparing outcome data, we can identify who use anticoagulation after stroke. I know it is so messy solutionā€¦

The cohort is great. Most studies follows this cohort definition. I mentioned a few points, but I donā€™t think you need to reflect all of these. Again, this study has great clinical implication.

#12

Hi @jreps, what is the function used to check the meaning of the response code (15) that results from the checkplpinstallation run? thanks!

#13

PatientLevelPrediction::interpretInstallCode(15)

It seems to be a connection issue:
Issue with database connection - did not connect
Issue with database connection - did not disconnect

#14

Jenna can we not call this intepretInstallCode function in the checkfunction (looping over all errors).

t