OHDSI Home | Forums | Wiki | Github

How do I handle condition observation records properly?

CDM Builders
,
#1

Hello all,

My name is Chao Pang and I am a data engineer at Columbia. I have a question regarding the observation table.

We have a visit_diagnosis table that contains all the conditions encoded by ICD9/10. Based on the concept driven approach, we redirect some of the visit_diagnosis records to the observation table based on the domain_id of the standard concepts.

For example, we have a visit_diagnosis record coded by ‘ICD9:V67.59’,
CONCEPT_ID : 44827397
CONCEPT_NAME : Other follow-up examination
DOMAIN_ID : Observation
VOCABULARY_ID : ICD9CM
CONCEPT_CODE: V67.59

We mapped the concept to the standard concept 4307024 through the concept_relationship table
CONCEPT_ID : 4307024
CONCEPT_NAME : Follow-up encounter
DOMAIN_ID : Observation
VOCABULARY_ID : SNOMED
CONCEPT_CODE: 390906007

We then put the mapped standard concept in observation_concept_id and left value_as_number, value_as_string and value_as_concept_id empty. However the Achilles Heel reports complained about such records in the observation table, the error message says “814-Number of observation records with no value (numeric, string, or concept); count (n=58,217,588) should not be > 0”. I am wondering what we should do to resolve this issue? Should we model such condition observation records differently?

Many thanks in advance, I am looking forward to your replies!

Cheers,
Chao

#2

I feel like that rule should be removed…I think it makes sense for that type of error message when checking the Measurement table, but but I think it’s reasonable to have Observations that state an event occurred without mention of some sort of associated value.

#3

I agree with @Chris_Knoll here. More often than not in claims data, source codes that are mapped to observations do not have values associated and this error is thrown. For us at Janssen we typically export the Achilles Heel report each time we build a CDM and add comments next to each error or notification explaining why they are occurring. We then save these reports to refer back to in an effort to make sure we aren’t see any new errors with each run. Our large databases typically have about 30 messages in Achilles Heel that we can explain based on how the ETL is written.

#4

Is there any decision made on this? The error still pops out and I can hardly explain why it exists: we have totally legitimate Observations with no value required. V67.59 Other follow-up examination is a good example.

#5

I agree. There are some ETL guidance for “history of” codes, which is challenging to code in an ETL. However, there are other codes that don’t fit that model like a “vehicle accident” where the date is enough. @Vojtech_Huser should this be removed from Achilles?

#6

That is nice example where the community drives better DQ rules.
To try to capture the group consensus:

  1. the rule is too broad and should be modified or replaced with new rule(s)
  2. some observations expect a value and a list of those can be made (a knowledge base for Achilles and way to incorporate such KBs (probably not as a list in SQL IN (xxxxxx) statement
  3. some observations are high quality with just a date (no value) (and a list of those can be made (or driven by a consensus derived from actual network data) (perhaps in a ThemisConcept study subpart)
t