Concern about the construction of the positive controls in the empirical CI calibration paper

thanks martijn for clarifying that the paper makes an assumption on positive controls: that the injected cases have the same sensitivity as the corresponding negative control. however, the assumption sounds somehow arbitrary to me: why wouldn’t someone else assume that sensitivity is double? of half? or 100%? this assumption propagates directly to the assumption about the ‘truth’ of the RR, which is the critical point of the paper: so knowing that this critical assumption is necessary does not really address my concern. but maybe there is something obvious that i am not seeing?

it would look safer to me to drop this assumption, and rather include an estimate of sensitivity in the computations. i agree with martijn that the estimate could be developed as suggested in this thread. if the estimate is only approximated, the impact on the results of a set of scenarios for sensitivity could be explored.

on the other source of variability of the ‘true’ RR: martijn and i agree that, no matter what k, the difference between n(1-p)/(1-p/k) and n is small if the prevalence p of the outcome is small. i am more cautious in claiming that all the outcomes (beyond presbyopia :joy: ) have low prevalence, though, because what we are interested in here is the prevalence among exposed, not in the general population. in the specific case, there are several negative controls which are related to diabetes, which is highly prevalent among dabigatran users, and p is likely to be much higher here than in the general population. so this is something that could be usefully included in the pipeline for generating the positive outcome: indeed if scenarios on sensitivity are tested, then p can be estimated by kn/N.

as for the fact that non-differential PPV is not of interest: i am not sure i understand why it is not. indeed, it is a classical result that non-differential PPV (which, in itself, does not bias the point estimate of the RR, so, does not induce systematic error) does have an impact on the variance, which is precisely what this paper is addressing. but maybe, also here, there is something obvious that i am not seeing.

i have the feeling this can be replicated in case there are also false positives, that is, PPV is <100%. i will develop the computations and share them here - or, if someone else has already made the computations or seen them in the literature, that would be great. i would also be happy to read any additional input/comment!