OHDSI Home | Forums | Wiki | Github

How to add Empirical Calibration to PLE skeleton


(Carmen Olga Torre) #1

Dear @jswerdel, @jweave17, all

I would like to add the Empirical Calibration to a PLE study skeleton that comes out of ATLAS.
Could you please advise on how and where to add the code we need to perform it?

I believe it would be of interest to the community if someone could give an explanation of what needs to be done.


(James Weaver) #2

Hi @CarmenOT,

As the ATLAS population-level effect estimation editor is currently configured, you can produce a study package that includes empirical calibration if you select “Yes” to the question “Should we perform positive control synthesis to calibrate confidence intervals?” in the Evaluation Settings tab. But, as the setting says, your study package will include the time-consuming task of generating and inserting synthetic positive control outcomes. If you’d like to empirically calibrate your estimates and confidence intervals on the basis of negative controls only (on the assumption that your study’s residual error doesn’t differ across effect sizes), you’ll have to edit a few lines of code in the R/Export.R file like in this example: https://github.com/ohdsi-studies/Covid19EstimationHydroxychloroquine/blob/master/R/Export.R#L526-L565 (note that the following else statement is excluded: https://github.com/OHDSI/SkeletonComparativeEffectStudy/blob/master/R/Export.R#L554-L560). Happy to discuss is helpful.

Lastly, the estimation editor and skeleton package developers are aware of this issue and in the near future the option to calibrate estimates and CIs on the basis of negative controls only will be available in the editor and you won’t have to make the coding fix.