OHDSI Home | Forums | Wiki | Github

Achilles v1.5: Multithreading of analyses

Developers
#1

Hi all,

I’ve been working on a branch of Achilles that is intended to speed up performance of Achilles in MPP database platforms (Redshift and PDW) by utilizing multiple SQL sessions to run analyses that have no dependencies: https://github.com/OHDSI/Achilles/tree/multithreaded. Thus far, I’ve observed Achilles execution (without cost) that can take 1 or 2 hours on Redshift or PDW, compared to multiple hours using the older version.

I’m looking for help on testing with multiple dbms platforms, particularly those with Oracle, MySQL, Impala, or BigQuery expertise and access.

This version also brings us up to speed with CDM v5.0.1, meaning cost analyses are now possible.

Thanks,
Ajit

1 Like
Data outside of OBSERVATION_PERIOD
#2

@Ajit_Londhe this sounds like a great enhancement.

I can help by testing on Redshift and Impala. Is it ready for testing now?

2 Likes
#3

Hi Lee (@admin),

Yep, it’s ready to test. The cost analyses are still a big bottleneck, but at least can work with the new COST table in CDM v5.0.1+. Also, from my testing, the Redshift dense storage clusters are significantly slower than the Redshift dense compute clusters when executing this.

Thanks,
Ajit

#4

Sounds great. We can help testing on BigQuery.

1 Like
#5

Hi @admin and @t_abdul_basser, just checking in. Any issues found so far?

#6

@Ajit_Londhe Sorry, I have not gotten around to testing this yet.

#7

Hi ALL,

Where can I find the PostgreSQL script to create the data base schema in CDMv5?

Thanks,

#8
#9

Hi all,

I am working on a clinical trial data and now i wanted to connect my postgresql tables with R.

I am using these scripts in the same sequence:
install.packages(“devtools”)
library(devtools)
install_github(“ohdsi/SqlRender”)
install_github(“ohdsi/DatabaseConnector”)
install_github(“ohdsi/Achilles”)
install.packages(“fastmatch”)
library(Achilles)

Every thing is ok till here but when I am trying to conncet:

connectionDetails ← createConnectionDetails(dbms=“postgresql”, server=“localhost/ohdsi”, port=“5432”, user=“postgres”, password="******", schema=“public”)

Nothing happens…

image.png1366×768 79.5 KB

Please guide me how to connect with Psql and how to visualize the results.

#10

How do I know my R and Postgresql got connected and how wd I proceed furthr…

Need your inputs!

Thanks

#11

@nitishkjha:

This is a community forum. Maybe somebody will walk you through step by step, but that is not likely going to happen. Perhaps, it makes more sense to first take the tutorials about the Data Model and the technology stack. And since you guys are a commercial entity, please reach out to the commercial consultants on OMOP/OHDSI.

#12

That just creates the connectionDetails object. You then need to connect like this:

connection <- DatabaseConnector::connect(connectionDetails = connectionDetails)

#13

Dependency of measures (and construct of derived measures) is an interesting problem in Achilles. How are you handling the analyses (=measures) that depend on each other?
Do you envision a knowledge base of what depends on what? (or hard code it somehow?)

#14

For the Achilles analyses, they’re all independent of each other, so they can run in parallel. In Heel analyses, there’s a batch of independent scripts that run in parallel, and then are unioned together into the final tables. There’s then a batch that runs in serial because they have dependencies.

Currently, there’s not a good knowledge base of this breakdown, aside from the folder structures indicating “dependent” or “independent.” But I think this is a good idea to help make the package more transparent.

Did you get a chance to try this branch in your environment?

t