OHDSI Home | Forums | Wiki | Github

Community review requested for negative controls

Researchers
#1

I’ve develop a set of negative controls that will be part of an OHDSI benchmark for methods (to see whether the methods produce estimates consistent with the true effect size). To allow evaluation of a large diversity of methods, the set consists of 200 target-comparator-nesting-outcome combinations, stratified by 4 outcomes and 4 exposures.

If anyone would be willing to review (part of) this set that would be greatly appreciated. Here are the questions that need to be answered for every item in the set:

  1. Do you feel confident that the target does not cause or prevent the outcome? (only a causal relationships are of concern. It is ok if the target and outcome are likely associated).
  2. Do you feel confident that the comparator does not cause or prevent the outcome? (only a causal relationships are of concern. It is ok if the comparator and outcome are likely associated).
  3. Do the target and comparator seem like a pair of treatments that could hypothetically be compared in an observational study?
  4. Does the nesting cohort seem like something a nested study (e.g. nested case-control) study might be nested in when evaluating the relationship between the target and outcome?

Here’s the set:
FullSetOfNegativeControls19May2017.xlsx (29.5 KB)

The window for community review closes on June 14.

#2

On http://forums.ohdsi.org/uploads/default/original/1X/7ca369f9058613e8eafe3466f74ba94f83c01857.xlsx FullSetOfNegativeControls19May2017.xlsx, once again, amazing job. I did not look up each drug individually but instead looked at the lists for things I would recognize. I think pancreatitis is the tough one because a lot of things can cause it. I was in particular worried about ketoconazole, but I cannot find an actual case or study. Just a generic mention in a drug-induced pancreatitis paper [Kaufman MB. Drug-induced pancreatitis: A Potentially Serious and Underreported Problem. Pharmacy and Therapeutics. 2013;38(6):349-351.]. So I don’t think that eliminates it.

I see strongyloides causing pancreatitis and treated with ivermectin, but that is the kind of confounding we are looking for.

George

#3

Thanks George!

Wow, the ‘evidence’ for ketoconazole is indeed the weakest imaginable. The paper you mentioned just says:

In addition, the agents used to treat opportunistic infections, such as ketoconazole (Nizoral, Janssen), sulfonamides, pentamidine, metronidazole, isoniazid, and corticosteroids, may also play a role.

with no reference or anything. After reviewing the literature again I found a similar statement in this paper. It does have some references, but I read those (except the Finnish article) and they don’t support the claim.

We could drop it just to be on the safe side. The next viable negative control is:

target: Pilocarpine
comparator: Brinzolamide
nesting: glaucoma
outcome: acute pancreatitis

#4

These look fine:

target: Pilocarpine
comparator: Brinzolamide
nesting: glaucoma
outcome: acute pancreatitis

Ketoconazole is probably fine, too, but these look safer.

George

#5

Hi, @schuemie
Sorry, I didn’t attend the last WG meeting because I was on the subway to go to Korean OHDSI meeting.

I didn’t look up all drug pairs, but I’m a little bit cautious about using sexual hormone treatment here (Because it would be hard to find things not involved with sexual hormone).
There are some evidences that sexual hormones might be related with GI bleeding or IBD. So I’m concerned about 'gonadotropin, or ‘luteinizing hormone’.

http://www.prevention.com/health/hormone-therapy-and-gi-bleeds

https://www.ncbi.nlm.nih.gov/pubmed/25851437

Thanks ^^

#6

@schuemie Sorry if I’m being dense. What target-outcome relationship attributes make a nesting cohort seem wrong? One wouldn’t nest a study of the risk of outcome Y given exposure X within a cohort defined by Z because the relationship between X and Y is…
Or is the outcome irrelevant? Is this a check that the condition defining the nesting cohort is an indication for the target drug?

#7

@schuemie A physician-scientist colleague specializing in behavioral health reviewed all the Target->Outcome and Comparator->Outcome pairs and found no reason to doubt that any of them were apt negative controls.

#8

Thanks everyone for taking a look at the negative controls. Special thanks to @scepeda, who reviewed the literature for each and every one of the controls! Soledad identified quite a few problematic controls, which have subsequently been replaced.

Here’s the new (and final) set of negative controls:
FullSetOfNegativeControls28June2017.xlsx (31.1 KB)

t