OHDSI Home | Forums | Wiki | Github

ERA scripts generate HUGE temporary tables

Implementers
#1

Hello!

We have currently finished our ETL process to the OMOP CDM. The only thing we are missing is filling the ERA tables (drug_era and condition_era). We are using the scripts privided by OHDSI (sqlScripts.knit) but they generate huge temporary tables (≈400GB), making the server to run out of memory and stopping automatically the query.

Is it normal? Did you experience a similar issue? How can we solve it?

Thanks in advance!

#2

Hi Carlos - I am not presenting a solution for you (sorry!) but wanted to say we have had the same issue when running this script on a DB with particularly deep data.

The script ran for 10 days and then ran out of log space on the server.

I wondered if anybody had modified the script to perform in batches or a similar solution?

t