One very important thing I realize we haven't discussed: should we evaluate methods that try to quantify risk attributable to an exposure, or methods for comparative effectiveness. In other words, methods tend to answer one of these questions:
- What is the change in risk of outcome X due to exposure to A?
- What is the change in risk of outcome X due to exposure to A compared to exposure to B?
Question 1 can often be answered by reformulating it as question 2 by picking a comparator believed to have no effect on the risk. For example, in our Keppra and angioedema study we picked phenytoin as a comparator because we were certain it did not cause angioedema, allowing us to estimate the effect of Keppra.
I must confess I'm mostly interested in question 1, since comparative effectiveness methods can be viewed as answering question 1 by picking a 'null comparator' as argued above. But we could create two gold standards, one for question 1 methods and one for question 2 methods.