| OHDSI Home | Forums | Wiki | Github |
Hello, OHDSI all
First of all, I’m sorry to repeatedly mention that I’ve had issues while running query.
I almost inserted data for synpuf 2.33M patients, but condition_era data has been still running since 8 days ago. It means that I’ve still run it for 8 days so far. I got it from ETL_CMS on Github.
For OHDSI managers,
I hope to check if ETL_CMS github scripts have problem or not.
Also give me any ideas that I can solve.
Thank you!
I think you are doing the right thing. You run into problems, ask the community, it doesn’t have an answer, you solve it. Only thing left: Give back. Add your experience to the documentation, or improve the code.
Thank you for your answer @Christian_Reich
In my opinion, this synpuf data set doesn’t seem large size data.
I successfully ran drug_exposure data into postgres for only 11 hours.
However, the thing is I don’t get it why only condition_era data has been running for 8 days.
xx_era table needs complex join and heavy calculations.
You may need indexes for the source tables and high-speed SSD RAID arrays for the tables and enough RAM and CPU clocks.
Yes you’re right @rwpark
There are some sub_queries in xx_era table.
I already created indexes for the tables and have enough 64GB RAM and 8_core processor(16 CPUs).
Usually enough is not enough. Sometimes our server with 768GB of RAM and 32 core of 3.6GHz CPU with 24 SAS SSD RAID faces not enough performance.
That sounds really bad to me.
I was confident with my server spec tho, now I should change my mind.
First of all, I have to check if it’s been working like data_process.
Thank you for your answer @rwpark. 