OHDSI Home | Forums | Wiki | Github

Phenotype Phebruary 2023 - Announcements

The Phenotype Development and Evaluation workgroup is having a planning discussion on Phenotype Phebruary tomorrow Thursday 1/19/2023 at 9:30AM EST to 10:30AM EST

MS Teams Meeting invite

The following was sent out by the workgroup.

Do you want to play a role in Phenotype Phebruary? Then please join this planning meeting

What happened last year:

  • community through voting selected clinical ideas to phenotype.
  • Individuals signed up to lead each clinical idea.
  • Leads followed a process: clinical idea, literature review, cohort definitions, evaluation summary
  • Others in community engaged in the discussion
  • Achievement: about 12 completed
  • Problems: not all completed, not everyone can evaluate, individuals were more interested in one component of the process vs entire process, an individual may not be able to go through the entire process. It was too much content

Planning related discussion for 2023 so far:

  • Less is more, last year was overwhelming
  • One person cannot complete all steps, we might need to stagger.
  • We are not trying to innovate new ways for phenotyping - make current ideas clearer, efficient, more trustable
  • Focus on core questions/themes and have discussions to drive incremental knowledge on these topics.
  • Address debates in community related to phenotyping
    • Terminology
    • Analytic use case vs Target clinical idea
    • Can phentoypes be developed independent of data
    • what is completion/evaluation, Peer review)
  • We want more participatory engagement - not just few people doing most of the work.
  • Define completion: we did not complete most last year.

Some weird reason - a cancellation notice was sent out by MS teams. Please ignore that.

The original meeting invite is still active and we are having the meeting in about half hour

MS Teams folder
Phenotype Phebruary 2023

Recording of the first planning session meeting
Phenotype Phebrurary 2023 - Planning session 1-20230119_093235-Meeting Recording.mp4

Recording of the second planning session meeting
Meeting in General-20230127_090308-Meeting Recording.mp4

Thanks for the meeting recording @Gowtham_Rao !
I was not able to join today’s meeting but would love to be more involved.
Are there f/u meetings i can attend?

Looking forward to the work!

1 Like

Phenotype Phebruary 2023

In 2022, we started many discussions related to phenotyping.

  • Many perspectives on phenotyping were discussed. Some we agreed on, others are still debated.
  • We started wanting a structured process for phenotype development and evaluation and so we experimented on many ways.
  • Collaborators started to systematically use our OHDSI tools (ATLAS, CapR, PHOEBE, APHRODITE, CohortDiagnostics, PheValuator,…)
  • Over time a pattern evolved that started looking like good practices.
  • We built out the OHDSI Phenotype Library. We outlined a peer review process for content to get into the library, and specified what and how cohort definitions could get in, and how we retrieve and use them in our research.

Phenotype Phebruary 2022 gave us so many new ideas on good practices that matured over the year. It is time to test if those good practices are really best practices – can we harden them further in 2023? If we all focus the community effort to collaboratively review, evaluate and build a number of phenotypes together using the insights we learnt last year, we debate the ideas behind these practices – we may move us closer towards having a resource that can support all of our community analysis activities.


So, we have Phenotpye Phebrurary 2023:

For week 1:
Inputs: You will be given

  • A target Phenotype
  • A clinical description
  • Prior work (literature review)
  • One or More cohort definitions
  • Output from OHDSI tools (CohortDiagnostics/PheValuator) on the definition executed on one or more datasource
  • A written summary and interpretation of findings by an OHDSI collaborator

Deliverables: complete a peer review to these inputs and provide recommendation to

  • accept the phenotype definition
  • Modify the phenotype definition
  • Reject the phenotype definition

Target: 4 phenotypes will be posted. One every day in the first week of February.

Debate topic (to take place at the OHDSI forums): What is peer review? What is the value of peer review? What should be in peer review? Who should do the peer review?

Looking for volunteers: Do you want to be part of Week 1? We are looking for collaborators to perform peer review existing phenotype definitions and evaluation summaries. A successful collaborator will think critically about what it entails to review another collaborator cohort definition and provide peer review judgments on the work. Collaborators will receive 1:1 assistance from organizers on their activities. No prior knowledge or skills required. Clinical understanding of the phenotype is preferred.


For week 2:

Inputs: You will be given

  • A target Phenotype
  • A clinical description
  • Prior work (literature review)
  • One or More cohort definitions
  • Output from OHDSI tools (CohortDiagnostics/PheValuator) on the definition executed on one or more datasource
  • A written summary and interpretation of findings by an OHDSI collaborator

Deliverable 1: a written evaluation report that summarizes your findings
Deliverable 2: complete a peer review to these inputs and provide recommendation to

  • accept the phenotype definition
  • Modify the phenotype definition
  • Reject the phenotype definition

Target: 3 phenotypes will be posted, spread across the second week of February. Activities may be started earlier.

Debate topic (to take place at the OHDSI forums): What is evaluation and why should we empirically evaluate cohort definitions? What to look for and perform an evaluation? What are the operating characteristics of a cohort definition? How helpful are the tools (Cohort Diagnostics, PheValuator, Cohort Explorer)? What is a gold standard and do we truly need it?

Looking for volunteers: Do you want to be part of Week 2? We are looking for collaborators to perform the two deliverables. For Deliverable 1: You will be a) Interpret cohort diagnostics and/or phevaluator output, b) Modify phenotype definition when required, c) Author a summary of the evaluation. For Deliverable 2: Collaborators are needed to complete the peer review.

For week 3: You will be given

  • A target Phenotype
  • A clinical description
  • Prior work (literature review)
  • One or More cohort definitions
  • Output from OHDSI tools (CohortDiagnostics/PheValuator) on the definition executed on one or more datasource
  • A written summary and interpretation of findings by an OHDSI collaborator

Deliverable 1: execute OHDSI tools (CohortDiagnostics/PheValuator) on the definition on the OHDSI network and get output. Make results publicly available e.g., shiny on data.ohdsi.org
Deliverable 2: a written evaluation report that summarizes your findings
Deliverable 3: complete a peer review to these inputs and provide recommendation to

  • accept the phenotype definition
  • Modify the phenotype definition
  • Reject the phenotype definition

Target: 2 phenotypes will be posted, spread across the third week of February. Activities may be started earlier.

Debate topic (to take place at the OHDSI forums): What makes cohort definitions reusable? Do we customize phenotype development to data and/or analytical use case (study) or, is phenotyping an independent activity? What terminology should we use when we describe operating characteristics of a cohort definitions (e.g., sensitive cohort, specific cohort). What is the value of OHDSI PL and what should be in it?

Looking for volunteers: Do you want to be part of Week 3? We are looking for collaborators to perform the three deliverables. For Deliverable 1: Data partners with access to data on OMOP CDM and willing to run Cohort Diagnostics and submit results. For Deliverable 2: You will be a) Interpret cohort diagnostics and/or phevaluator output, b) Modify phenotype definition when required, c) Author a summary of the evaluation. For Deliverable 3: Collaborators are needed to complete the peer review.

For week 4: You will be given

  • A target Phenotype
  • A clinical description
  • Prior work (literature review)
  • One or More cohort definitions
  • Output from OHDSI tools (CohortDiagnostics/PheValuator) on the definition executed on one or more datasource
  • A written summary and interpretation of findings by an OHDSI collaborator

Deliverable 1: collaborative develop one or more cohort definitions for the target. Seek clarification of the clinical idea and make changes as needed. Use material from literature review. Build cohort definitions that may be run on OMOP CDM that may be either rule based on probabilistic but the output should be cohort.
Deliverable 2: execute OHDSI tools (CohortDiagnostics/PheValuator) on the definition on the OHDSI network and get output. Make results publicly available e.g., shiny on data.ohdsi.org
Deliverable 3: a written evaluation report that summarizes your findings
Deliverable 4: complete a peer review to these inputs and provide recommendation to

  • accept the phenotype definition
  • Modify the phenotype definition
  • Reject the phenotype definition

Target: 1 phenotype will be posted, in the fourth week of February. Activities may be started earlier.

Debate topic (to take place at the OHDSI forums): The role of probabilistic modelling in phenotype development and evaluation. Can we sustain this level of effort for rule-based phenotyping, does it scale?

Week 5: Synthesis
Debate topic (to take place at the OHDSI forums): Is phenotyping a scientific work that is worthy of scientific dissemination via publication? How was comparing to Gold standard useful, feasible, limitations

Next steps

Volunteering Collaborators:

  • You are not alone! Collaborators will be supported by an experience OHDSI leader. If this is your first time in OHDSI, you are most welcome. Join the Journey.
  • One collaborator may sign up for one task or multiple tasks required for each week.
  • You will have support person (phenotype pal) to help complete the task.
  • You can sign up by contacting me, @Patrick_Ryan @AzzaShoaibi

Phenotype selections:

I will have a follow-up post with proposed phenotypes.

1 Like

Week 1: Peer review only

  1. P1 Acute Pancreatitis - Evan Minty MD @Evan_Minty (Peer reviewer )
  2. P2 Anaphylaxis - Andrea Noel MD @Andrea_Noel (Peer reviewer )
  3. P3 Appendicitis - Azza Shoaibi PhD BPharm @Azza_Shoaibi (Peer reviewer )
  4. P4 Neutropenia - Anna Ostropolets MD PhD @aostropolets (Peer reviewer )

Week 2: Evaluation and Peer review

  1. P5 Systemic Lupus Erythematous - (Evaluator) @jswerdel , (Peer reviewer ) @Daniel_Prieto
  2. P6 Acute Hepatic Failure - unassigned
  3. P7 - unassigned

Week 3:

  1. Parkinsons Disease - Allan Wu (need peer reviewer)
  2. Acute Myocardial Infarction - unassigned

Week 4:

  1. Hypoxic Ischemic Encephalopathy - Khyzer Aziz MD
1 Like

We are looking for volunteers to take on P6, P7 and P9.

Looking for volunteers for P6

All 10 phenotypes are now taken.

Week 1: Peer review only

  1. P1 Acute Pancreatitis - Evan Minty MD @Evan_Minty (Peer reviewer )
  2. P2 Anaphylaxis - Andrea Noel MD @Andrea_Noel (Peer reviewer )
  3. P3 Appendicitis - Azza Shoaibi PhD BPharm @Azza_Shoaibi (Peer reviewer )
  4. P4 Neutropenia - Anna Ostropolets MD PhD @aostropolets (Peer reviewer )

Week 2: Evaluation and Peer review

  1. P5 Systemic Lupus Erythematous - (Evaluator) @jswerdel , (Peer reviewer ) @Daniel_Prieto
  2. P6 Acute Hepatic Failure - (Evaluator) @Christian_Reich , (Peer reviewer) @Patrick_Ryan
  3. P7 Idiopathic Inflammatory Myopathies (Evaluator) @Christopher_Mecoli (Peer reviewer) pending

Week 3:

  1. P8 Parkinsons Disease - Need to run network study, (Evaluator) @allanwu, (peer reviewer) pending
  2. P9 Myocardial Infarction (Evaluator) @mirza_khan, (Peer reviewer) pending

Week 4:

  1. Hypoxic Ischemic Encephalopathy - @Khyzer_Aziz

Week 1: Peer review only

  1. P1 Acute Pancreatitis - Evan Minty MD @Evan_Minty (Peer reviewer )
  2. P2 Anaphylaxis - Andrea Noel MD @Andrea_Noel (Peer reviewer )
  3. P3 Appendicitis - Azza Shoaibi PhD BPharm @Azza_Shoaibi (Peer reviewer )
  4. P4 Neutropenia - Anna Ostropolets MD PhD @aostropolets (Peer reviewer )

Week 2: Evaluation and Peer review

  1. P5 Systemic Lupus Erythematous - (Evaluator) @jswerdel , (Peer reviewer ) @Daniel_Prieto
  2. P6 Acute Hepatic Failure - (Evaluator) @Christian_Reich , (Peer reviewer) @Patrick_Ryan
  3. P7 Idiopathic Inflammatory Myopathies (Evaluator) @Christopher_Mecoli (Peer reviewer) @agolozar

Week 3:

  1. P8 Parkinsons Disease - Need to run network study, (Evaluator) @allanwu, (peer reviewer) pending
  2. P9 Myocardial Infarction (Evaluator) @mirza_khan, @atifadam

Week 4:

  1. Hypoxic Ischemic Encephalopathy - @Khyzer_Aziz

Thanks @Gowtham_Rao and @Azza_Shoaibi for organizing this year’s 2023 Phenotype Phebruary, and thanks to all the volunteers who have taken on leading some part of the fun (@Evan_Minty @Andrea_Noel @Azza_Shoaibi @aostropolets @jswerdel @Daniel_Prieto @Christian_Reich @Christopher_Mecoli @agolozar @allanwu @mirza_khan @atifadam @Khyzer_Aziz )

There’s a lot going on during Phenotype Phebruary this year, and I’ve received a few folks asking: "how can I get involved?’

The answer is simple: Join the Journey! Read and post on the discussions here on the Forums, participate in a Phenotype WG meeting each Friday or listen to the weekly updates on our OHDSI community calls on Tuesday. Share your thoughts, ask your questions, solicit collaboration for developing or evaluating a phenotype. Everyone is welcome and encouraged to engage in this fun community collaboration opportunity, so we can all share, learn, and advance together.

‘Phenotype Phebruary’ is just an excuse to have our community put focused attention for a few weeks on an important topic that touches every study that everyone of us conducts every single day when trying to generate reliable evidence from observational data. The science of phenotyping is still very much unsettled (I would even contend that its still in its infancy, and more of an art than a real science at this point). There are still lots of disease targets that haven’t been phenotyped and lots of other diseases that have candidate algorithms but little-to-no proper validation. So there’s a lot of opportunities for us to work together to move this field forward. Let’s do it!

1 Like

We are looking for volunteers to help with phenotype development activities for week 3 and 4. ie if you are familiar with atlas and would like to help with development of Cohort definitions for phenotype phebruary please message me.

This will require 1 to 2 hour session with either @Christopher_Mecoli @allanwu @mirza_khan or @Khyzer_Aziz to develop, review or improve the cohort definitions.

Today February 8th 2022, we are planning to have three interactive work sessions.

9am EST to 10 am EST - Inflammatory Myopathies - Dermatomyositis

  • This cohort definition was previously developed by @Christopher_Mecoli at JHU.
  • Purpose: assist running CohortDiagnostics, perform evaluation of the developed cohort definitions.
  • Meeting invite

10am EST to 11am EST - ST Elevated Myocardial Infarction

11am EST to 12pm EST - Parkinson’s disease

Please join if you are avaiable.

t