CDM format and extension strategy - recovered topic

[Christian_Reich]

Hello all,

I’m investigating OMOP CDM for my team to build a better data standard of our data from different sources (including digital health, clinical trials, … etc). I’m new to this community and having some basic questions:

1. I wondered if it is a violation to store OMOP CDM in csv files? I saw example code from most of the tools such as DQD, Altas, … etc all connect to a relational database at the beginning. Also, I downloaded the ETL example and find CPRD builder_output are in txt file. Does it mean OMOP CDM can be realized by the tools if they are in txt or csv format?
2. We are handling a lot of streaming data, digital data, … etc. Different devices might have different attributes/data contents. What is the ideal approach if we wanted to have additional field/domain. I browsed through some related topic in the forum and please let me know my understanding below is correct:2a) Ask in the forum to see if there is a possible way to put our data in current CDM.
   2b) May need to create private fields/domains for my use case if not available in current CDM schema. The downside is that we can not use tools on these private fields/domains.
   2c) If we find a fields/domains which is very common to be used
   for analysis in specific area, we may initiate discussion and propose on github issue to add it to CDM.
3. Where can I check guidance to build CDM, regarding best practices or violation to the standard?

Thank you very much for your time!
Christi

Welcome to the family, @11122. (Who started this habit of using numeric names? )

@11122

store OMOP CDM in csv files

The OMOP stack is built on the premise of having clinical data stored in relational databases. All the tools expect the ability to send SQL queries to them. Txt files are only interim output to be loaded into the database. Of course, you could use Hadoop-type technology to keep your files, and put a SQL interface on top. Why do you want txt? If you need a free database system use Postgres.

@11122

2b) and 2c)

Exactly. If you have a use case, and you need fields/domains, please submit to the issue list and potentially also lobby for it here on the Forum. You might be surprised to have your proposals added to the standard.

@11122

guidance

Guidance on what? The CDM is defined in the Wiki. Is that what you had in mind?

[CRoeder (Chris Roeder)]

I understand the appreciation for CSV files, especially in the context of R etc. Plain text is simple and gets many jobs done well, clearly and simply. Relational databases solve a few problems that arise when you have lots (but not really Big Data lots) of data and more than a few different tables. There is some overhead in using relational databases, but it’s a net gain.

Regarding #3, I found the Book of OHDSI to be helpful. I even sprung for a paper copy!

[Ajit_Londhe]

Another database type to consider that is free and lightweight it is SQLite.

Never mind!

[Christian_Reich]

You are going to receive the wrath of @schuemie for just glibly putting out such ideas, @Ajit_Londhe. I’d check in first.

[Ajit_Londhe]

Did something happen with SQLite? Isn’t it still a supported dialect in SqlRender?

[krfeeney (Kristin Kostka, MPH)]

@Ajit_Londhe

Isn’t it still a supported dialect in SqlRender?

This is a VERY political question! There will soon be some workshops via @gregk @mvanzandt to iron out ideas going forward.

[schuemie (Martijn Schuemie)]

Actually, SQLite is supported (somewhat), as it is used by Eunomia.

You can run Achilles, CohortMethod, PLP on SQLite. But there seems to be a problem running the era logic embedded in all Circe cohort definitions.

SQLite is mostly used for educational purposes and unit testing.