I’m looking at the Phevaluator (as described in the Book of OHDSI and Swerdel 2019). In both it says that
“We emphasize that these expected cell counts cannot be used to assess the variance of the estimates,only the point estimates.”
To me this means that I can’t calculate confidence intervals for the PPV, Sensitivity, Specificity of different algorithms.
Since there is literature on calculating the variance of these estimates (see for example van Zaane, B., Vergouwe, Y., Donders, A.R.T. et al. Comparison of approaches to estimate confidence intervals of post-test probabilities of diagnostic test results in a nested case-control study. BMC Med Res Methodol 12, 166 (2012). https://doi.org/10.1186/1471-2288-12-166; the other citations in his paper point to some approaches) why can’t the variance be calculated using similar methods in order to derive confidence intervals?
A computer intensive approach, I think, would be to to wrap the Boot package around the Phevaluator package in R, but the computing resources required might be overwhelming.