Are there cases that standard concepts become non-standard?
If there are, what are reasons that this happen?
What column values change in the concept record?
Usually if the holder of the vocabulary deprecates a concept. Invalid Concepts cannot be Standard. Sometimes if we find a duplicate set and one of the two has to step down. In the first case, we rely on the vendor to give us an upgrade (successor), which happens, but not always. In the latter case we will provide a successor (“Concept replaced by” as well as “Maps to” relationships).
Oh great! I see that “Concept replaced by” is a good place to see how to find a old relationship can map to the new standard concept. I assume these are the records marked invalid_reason = ‘U’.
Although the examples in which “…the holder of the vocabulary deprecates a concept” and the cases where vendors do not give a successor standard concept may cause a consistency problem on analysis. I would arguably say in these situations that these concepts stay standard although they’re deprecated, just invalid (standard_concept = ‘S’ and invalid_reason = ‘D’).
The problematic situation I can think of would be data I continually receive and transform into OMOP. In the past, I map a non-standard concept to a “standard” concept. Analyses are built around this concept (dashboards, reports, etc.). I continue to receive new data from the same source and today I run transformation to OMOP with updated vocabularies. The concept now maps to the same concept, but it’s now “not standard” without any indication of a successor.
It seems I have 2 choices:
- In adhering to the OMOP standard, I would find a new standard concept to map to. And fix all processes, analyses, reports, etc. to use a new standard concept. Annoying to do…
- But to keeping analysis consistent, I would break the OMOP standard and just continue to map the non-standard concept as the concept to do analysis on. And everyone doesn’t need to know that this concept is now “not standard”.
The 1st choice is adhering to the standard which is good, but is the 2nd choice okay?
I may be nitpicky here and thinking in a more technical sense as opposed to understanding better how vocabularies evolve over time which I admit I’m not an expert at medical vocabularies. I’m also thinking OMOP isn’t perfect, but it’s a data model that has offered harmonization of disparate data. Is it okay to make certain choices in my organizations processes that may break rules in OMOP?
It’s unfortunately 1). 2) is a bad idea. Because deprecation of a concept usually happens for a strong reason, or because of a duplicate. In the first case you don’t want to continue basing your analysis on it, in the second one updating would not cause a problem.
In order to do 1) we need to build a Cohort Upgrader, which would automatically find all Concepts that are no longer valid and replace them with valid upgrades (for invalid_reason=‘U’). The ones that do not come with a upgrade it would at least list out, so you could do damage control manually, or with the help of the Vocab Team.
That’s what I tried to sell to you in the other post, but you elegantly started a new one - this one. 
Thanks @Christian_Reich
I guess I didn’t understand what you meant by Cohort Upgrader the first time. I do now. And it’s seemingly trivial given a “Concept replaced by” relationship and the application of the upgrade.