OHDSI Home | Forums | Wiki | Github

With new CDM releases, should SynPuf ETL be updated too?

General
#1

Hi all,

I was wondering if perhaps as part of the CDM specification lifecycle, we could add a step of updating the SynPuf ETL to meet the new specification. For example, now that we have CDM v6, a deliverable that should also be made available is the SynPuf ETL for CDM v6.

This could be useful to developers to have a CDM database in the newest CDM version to pressure test the application against the new standard. This can help ensure the applications are certified to work. For Achilles, I’ve been working on a CDM v6 branch, but I have no CDM in v6 to test against. While we will eventually move our CDMs to v6, that may not be something that happens right away.

@Chris_Knoll and @anthonysena – would this help you guys too?

Thanks,
Ajit

Tagging @ericaVoss and @clairblacketer

2 Likes
#2

I think this is an interesting idea and actually is fitting for Synpuf given that is the whole point of the data set to begin with - to be a test bed for development. Also, gives a way to force data into the new structure and might help us catch issues before a release is made.

My concern is we’d have to build out the builder in a way that is easy to update and that it has the support system to update it in a timely way.

#3

Yeah, that’s the question: can the builder be flexible enough? And, can we formalize this process so that when new CDM specs are being ratified, the ETL-CMS repo is updated proactively?

@Christophe_Lambert, curious about your thoughts.

#4

I think this is a great idea. If we do this, we need to bear in mind that there are two moving targets for keeping track of versions: the changing vocabulary and the changing common data model.

#5

In the light of a new SynPuf ETL; would it be possible to include values in the observation and measurement (value_as_number and/or value_as_concept_id)? From the SynPuf documentation I get the feeling that these values are not available. Maybe we could generate some numeric values per measurement concept?
So far few tools within Atlas use these values, but for future development it might be useful to have some demo date for these fields.

#6

Thanks @Ajit_Londhe for raising this topic. I’d say from an Atlas/WebAPI perspective, having access to a SynPuf CDM v6 (and v5 for that matter) has utility for testing the application. Let me know if there is anything I can do to assist. Thanks!

#7

@ajit_londhe, yes this would be helpful for tool development.

t