OHDSI Home | Forums | Wiki | Github

Issue while running condition_era query


(Micheal ) #1

Hello, OHDSI all
First of all, I’m sorry to repeatedly mention that I’ve had issues while running query.
I almost inserted data for synpuf 2.33M patients, but condition_era data has been still running since 8 days ago. It means that I’ve still run it for 8 days so far. I got it from ETL_CMS on Github.
For OHDSI managers,
I hope to check if ETL_CMS github scripts have problem or not.
Also give me any ideas that I can solve.

Thank you!


(Christian Reich) #2

@MichealJeong:

I think you are doing the right thing. You run into problems, ask the community, it doesn’t have an answer, you solve it. Only thing left: Give back. Add your experience to the documentation, or improve the code.


(Micheal ) #3

Thank you for your answer @Christian_Reich
In my opinion, this synpuf data set doesn’t seem large size data.
I successfully ran drug_exposure data into postgres for only 11 hours.
However, the thing is I don’t get it why only condition_era data has been running for 8 days.


(Rae Woong Park) #4

xx_era table needs complex join and heavy calculations.
You may need indexes for the source tables and high-speed SSD RAID arrays for the tables and enough RAM and CPU clocks.


(Micheal ) #5

Yes you’re right @rwpark
There are some sub_queries in xx_era table.
I already created indexes for the tables and have enough 64GB RAM and 8_core processor(16 CPUs).


(Rae Woong Park) #6

Usually enough is not enough. Sometimes our server with 768GB of RAM and 32 core of 3.6GHz CPU with 24 SAS SSD RAID faces not enough performance.


(Micheal ) #7

:frowning: That sounds really bad to me.
I was confident with my server spec tho, now I should change my mind.
First of all, I have to check if it’s been working like data_process.
Thank you for your answer @rwpark. :slight_smile:


t