We are developing a set of Patient-Level-Prediction models.
During this process it was suggested to describe the CI for the AUCPR.
I have not been able to identify a good way to do this and was wondering if someone could help?
I checked out the source code for both plotPrecisionRecall() and the evaluatePLP() (pr.curve), But i was not able to find a good way to add the compution of the CI.
They suggest using either binomial or logit intervals for the CIs and show with simulations it has sufficient coverage. Bootstrapping also works but it approaches sufficient coverage from below with increasing sample size and is also more computationally expensive.
They have R code with those methods on github:
For example the function in line 257 uses the logit method. You only need the auprc estimate which is already calculated in evaluatePLP (auprc) and the number of positive samples and your alpha. The number of positive samples is probably already there as well as length(positive) or something similar.