Phenotype Phebruary 2023 - Week 3 - Debate – customized versus standardized approach for Phenotype development and evaluation

[Evan_Minty]

Thanks @Gowtham_Rao – I think this conceptualization really advances the discussion. And thanks to all the others on here for a great thread.

1. A further consideration is whether we presume all research needs are best served by federated network research (and a preference for a generalizable cohort definition). Many organizational use cases may instead be driven by a requirement to maximize performance in their data and their data alone. (e.g. in patient level prediction, proving out generalizability is great, and important, but many models may be driven by local needs with performance proven out in local validation cycles). That notwithstanding, OMOP and OHDSI tools may be a valued part of that engine, and they may drive value from a phenotype library that gets their process started with a phenotype definition best suited to their use case.
2. I think both interests are served by getting a better sense of how the different components (@agolozar 's Lego’s) and combinatorics of a definition (i.e. variations-of-interest in concept sets, and combinations-of-interest between them) perform in a network of databases.

(2) is not straightforward to do. The combinatorics quickly explode, and even with simplifying assumptions, it’s not reasonably tractable (or configurable) without a computational approach.

The tools to do so are likely there in CirceR, Phea, and others, although we’d need a think more about a framework, and a lot of validation related compute…

But what could get interesting there, is then, could a pre-specified error tolerance could inform not a ‘go / no-go’ on the cohort definition / database, but possibly on the combination of components helps configure a definition in a database and permits its entry in a study?

I’d argue this increases our N, our geographic diversity in the network study, while minimizing our measurement error… wouldn’t that lead to more reliable evidence?

A core question becomes,

@Patrick_Ryan

In absence of this approach, if we don’t standardize our algorithms in a network study and instead encourage different questions and sources to all use different phenotype algorithms, we are just forcing unnecessary human-generated variance into our analysis pipeline.

What if it’s machine-generated variance instead?

(I’m sure week 4 still has some interesting discussion to come, on that theme )