Summary of the discussion Week 1 on peer review?
(all paraphrased)
- In the presence of objective diagnostics with prespecified decision thresholds, we don’t need peer review - but only a check list that a process was followed. However, today we do not have objective diagnostic or a checklist. @Patrick_Ryan
- Peer review is not pitting one persons subjective opinion on another persons work with little-to-no empirical basis to reconcile differences. @Patrick_Ryan
- Peer reviewer role - options for peer reviewed could be a) accept, b) reject, c) revise and resubmit;
- Use of peer review may not have any more advantage over a superficial confidence @Patrick_Ryan
- Future researcher “to determine if the existing phenotype is fit-for-purpose for the new research question or different database.” @Kevin_Haynes
- In the absence of objective diagnostics, "By soliciting another peer scientist to provide their independent perspective - we hope to discover measurement errors that we would not have otherwise seen " @Gowtham_Rao
- Checks if a) clinical idea is described and umabigious, b) cohort definition logic and clinical descriptions are concordant, c) cohort definitions are tested. @Gowtham_Rao
- we apply the OHDSI principles of a systematic approach. If we do undefined “reviews” we add more pain to the current situation. @Christian_Reich
- Job of the peer is to ensure the process to pressure test criteria to correct misclassification - index criterion, sensitivity problem, specificity problem,
- We should develop phenotypes with evaluation in mind - i.e. each step to reduce a source of error. Proposes a diagnostics informed approach for the process @Patrick_Ryan
- Peer reviewers repeating the steps to see if they come to the same conclusion - probably unnecessary. Uses analogy of the publication peer reviewer @Patrick_Ryan
- in phenotyping we don’t need to question intent or relevancy of phenotyping a clinical target @Patrick_Ryan
- It is the clinical description that helps justify design choices in cohort definition @Gowtham_Rao
- Peer reviewer should not pass judgment on whether the described clinical target (as in clinical description) is right or wrong e.g. crohns disease of appendix @Patrick_Ryan
- Estimating differential error maybe a post phenotyping process @hripcsa
- Until objective decision threshold place diagnostics are in place, peer review may be one possible imperfect way to stress test some of the subjectivity in the process and have it questioned by independent person @Azza_Shoaibi
- Phenotype peer review does not need to redo the phenotype development, but should be able to come to a recommendation a) accept, b) reject based on submitted material. @Azza_Shoaibi
- If we parse all ambiguities add add them to clinical description, we would multiply each library entry @Christian_Reich
- The clinical description is 1000 times better than what we have now - wishy washy descriptions in the papers. But still they fall short of what is needed @Christian_Reich
- The question is what is the question. Atrial fibrillation as an adverse event may require a different phenotype from Atrial fibrillation requiring DOAC therapy. Thus necessitating additional clinical descriptions and additional reviews and diagnostic metrics. @Kevin_Haynes