OHDSI Home | Forums | Wiki | Github

Proposal to keep outdated standard concepts active and standard

Vocabulary Users
#12

@mgkahn:

You are correct, and I read on. But it is not that simple.

Yes, it will be removed. But there are two cases really here in play:

  1. Source codes get mapped to a different standard concept. This is the case for Drug and Condition, since (except Charlie) nobody codes in RxNorm or SNOMED. So, if a Standard Concept dies, all the sources will be remapped to new Standards, and we are all set.

  2. Source codes are Standard Concepts. Unless we have the case of an upgrade (there is a new Standard Concept), we have a problem, because the now dead Concept has nowhere to go and gets mapped to 0. We lose data, even though the code and Concept were fine till very recently. That’s the use case we need to tackle. The hierarchy follows that just fine.

#13

Not sure we need to. Our valid_end_date has nothing to do with the fact that drugs are off the market, NDCs are no longer used or CPT-4 codes are not accepted for billing anymore. Our valid_end_date means that the semantic meaning of a concept is no longer upheld. So, from that perspective we should keep as is.

Right now the discussion in THEMIS is to create lists outside the standard Vocab tables for those ultra-rare cases. So, I wouldn’t abuse the valid_end_date any longer for this situation.

#14

@Christian_Reich:

  1. “source codes get mapped to a different standard concept.” True for those who do a complete ETL/code mapping with every new vocabulary release. Not true for those of us who are using OMOP as their core data warehouse model that do daily incremental loads. In this setting, the old standard concepts remain.

1a. ETL processing could be altered to remap every source code to the most-recent standard concept each time the vocabulary is refreshed. This would be a pretty massive remapping job at every vocabulary update.

1b. Still breaks those historical (often repeated) queries that reference terminal standard concept_id’s directly or indirectly. By indirectly, I’m think about queries that reference a Classification concept as an entry point into the concept_ancestor table and the Classification concept is retired. All of these would break.

  1. Consider adding standard_concept value of “H” (beyond just “S” and “C”) for historical and keep these concept_ids, mappings and ancestors always in place but marked as H. Maybe this is just another way of stating what others are proposing. The bottom line would be to NEVER get rid of any code or MAPS TO, or ancestor map that existed previously. I can only begin to imagine what this would do to the current terminology creation process…
#15

@Christian_Reich, @mgkahn,
Ok. at least one rule defined exactly:
our valid_end_date value and invalid_reason =‘D’ means that concept is wrong. Wrong parsing of drug dosage and then wrong RxNorm Extension concept, or wrong SNOMED entity (duplicate or non-sense).
In this case it’s totally OK to remove these concepts from Ancestor and Not to map TO these concepts anymore (as it is now).

But second: What to do with Outdated concepts (for example, ICD10CM changed their classification and now this disorder belongs to different sub-chapter, but, say, 2 years ago, doctors used this code to encode the disease)?
To be technically correct we need to create another “end_date” column, kind of “out_of_use_date”. And keep the standard_concept_id = ‘S’, (nobody will rewrite existing queries changing ‘S’ to (‘S’, ‘H’) - anyway you proceed Historical and Active concepts in the same way).
I think, people are also used to search the concepts using “where… invalid_reason is null”.
So less painful is not to touch invalid_reason logic, but add another field “active_status”.
So what I propose: Add 2 new columns: “active_status” and “out_of_use_date”.

@TBanokina, what can you say from ETL perspective about this approach?

#16

@Christian_Reich,
So what?:slight_smile:

#17

Oh. I see. We need to figure out something. Otherwise you will deviate increasingly from the standard, which will put your ability at risk to do network studies. This is a big deal.

That’s what folks do who don’t update continuously. A full job on all the data. Are you going to run into infrastructure problems, or why would you not want to do that?

Well, “break” also means “being fixed”. Goes both ways.

But yes, we need to think about a mechanism to update a query. I think that can be automated.

The problem is we don’t control that. Most of our content comes from the sources. If SNOMED updates concepts or links we cannot just ignore that.

Exactly. So, the solution is that we need to string the CDM data along and update the queries. No way around it.

What’s the use case? Why would we need to know it’s no longer in use? The only case I can think of is for purposes of debugging cohort definitions trying to identify the reasons for change over time. Is that it?

I also a lot of “where sysdate between start_date and end_date” notations. We will have to do a THEMIS convention.

#18

I can see me voting Yes for the proposal

So what I propose: Add 2 new columns: “active_status” and “out_of_use_date”.

So that CPT4 “76645 | Ultrasound of breasts” that is correct but outdated has active=0 and out_of_use_date=‘2017-07-01’ and has ‘S’ standard status

2 Likes
#19

As for me the final idea is:

valid_end_date – date of source outdating or date when we removed it from the vocabulary because it was added mistakenly
Add new value of invalid_reason:
M – concept deprecated as it was added mistakenly
D –outdated by the source
U - updated concept
Standard_concept = ‘S’ for concepts with invalid_reason = null or invalid_reason =‘D’,
Standard_concept = ‘C’ respectively for classification concepts

#20

@Dymshyts:

so, the real difference is that D no longer invalidates a Concept and turns it from a Standard to a non-standard. Correct? While M does it the same way D does it today.

We could just not turn a concept to D if the source wants to deprecated it. Just keep it going. Invalid_reason=null, but valid_end_date something before 2099. Wouldn’t that do the trick?

1 Like
#21

I like that. No coding change and then D would only mean one thing, there
is something wrong with this concept and it should not be used.
Assuming in addition, if invalid reason set to D then Standard Concept set
to NULL and ‘maps to’ relationship is removed, while if only the
valid_end_date is changed, the concept remains Standard and the ‘maps to’
relationship continues to exist.

Expired Concepts, No Successors, Standard and No invalid Reason
#22

correct

I also like that. Seems to be the simplest solution

#23

We had a nice discussion 6 of March work-group and finalized the decision:

For a HCPCS and CPT4 concepts deprecated because of the source withdrawal, we make the Standard_concept = ‘S’ and invalid_reason = null, but valid_end_date reflects the withdrawal date.
This makes possible to use these concepts as standard and fill up the CDM, and also CDM builders can easily check the consistency of the data ensuring that event date is between concept’s valid_stard_date and valid_end_date.

Once we have resources available, we’ll implement these changes

Note, we make this for HCPCS and CPT4 only. In the future we’ll analyze the other vocabularies, understanding why the source deprecates the concepts and then we’ll make a decision whether we need to deprecate them or just change the valid_end_date but not affect Standard_concept and invalid_reason values.

1 Like
Anastomosis, choledochal cyst, without excision
#24

That’s great news! This will help our ETL since there were codes we were loosing b/c they were deprecated.

#25

If the source code is deprecated in the source vocabulary (was valid before, if not valid now). Are the following correct?

  1. We will not make the source code invalid. We will set the valid_to_date to the date the code was deprecated.
  2. Mapping to standard will exist in the concept_relationship table
  3. If a code from standard vocabulary like snomed is deprecated, it will still exist as invalid is null, but would be mapped to shoulder equivalent snomed that is not deprecated.
#26

Yes, but only for HCPCS and CPT4 as the source withdrawal means change of bulling system classification or something like that - when the semantic meaning is correct but concept is not in use anymore.

Yes, again for HCPCS and CPT4,
as they will not be deprecated they remain standard, so they’ll map to themselves or to SNOMED or RxNorm equivalent (if exist)

SNOMED and RxNorm mostly deprecate the concepts because of the semantic non-sense: Drugs with impossible dosage, Conditions breaking the rules of SNOMED hierarchy or having duplicates and so on.
Thus if SNOMED source deprecated we also deprecate this concept.
So the changes discussed here will be applied only to HCPCS and CPT4.

In the future will analyse the behavior of different sources and define the corresponding concept deprecation scenario for them.

#27

Reviving this problem since we still need to deal with the source removing codes.

2020 release of ICD10PCS invalidates ~2000 codes. It does not seem that codes were erroneous per se. At a glance it looks like that ICD10PCS sometimes changes logic for definition of some procedures: for instance, hemorrhage control procedures are no longer coded for a specific organ and rather for anatomical region.

Since old codes are not wrong (duplicates or calamities) and will not be reused by new ones (since ICD10PCS does not allow to reuse codes), I suggest keeping historical codes Standard for ICD10PCS (at the moment they are deprecated).

1 Like
#28

Is it the exact case when we can map it?
A specific organ procedure to an anatomical region procedure?

#29

What’s the decision here, guys?

#30

No decision yet. We want to hear from the community first.

#31

Ok, no objection here.

We’ll make them standard, add them to the SNOMED hierarachy like we did for the other ICD10PCS concepts. Thus they will appear in concept sets defined through parent concepts and also will keep the original meaning.

t