| OHDSI Home | Forums | Wiki | Github |
Hi Jun,
Cyclops uses cyclic coordinate descent for optimizing the likelihood function. Also, Cyclops makes extensive use of the fact that the data is sparse, both for storage (no need to store zeroes) and for computation of the likelihood (no need multiplying betas with zeroes).
It also helps that Marc implemented the algorithm in C++ in very efficient way.
A more detailed description can be found in this article.
Hi all,
We have just released version 1.1.0 of CohortMethod. Here is the description:
This version adds support for running large scale studies including many target-comparator combinations, outcomes, and analysis. This functionality is documented in a new vignette.
Also included are many bug fixes, as well as significant performance improvements in data fetching, matching, and covariate balance computation. (Recent changes in Cyclops have already improved performance in propensity model fitting and outcome model fitting). This new version also supports the Common Data Model version 5.
Current users of CohortMethod should be aware of these changes in the user interface:
I’m getting unexpected results from running CohortMethod and was wondering if you could point me in the right direction.
Here is what I have done:
I created three cohorts using Circe. Here they are on ohdsi.org if helpful:
Target - http://www.ohdsi.org/web/circe/#/814
Comparator - http://www.ohdsi.org/web/circe/#/815
Outcome - http://www.ohdsi.org/web/circe/#/816
I confirmed that all three cohorts generated patients from my local CDM. I then ran some manual SQL joins to ensure that there is overlap in patients between my target / comparator cohorts and outcome cohort.
I moved these three cohorts to a dedicated table and set up the CM configuration as described in the SingleStudy vignette, using the appropriate cohort_definition_ids for my targetId, comparatorId, and outcomeId.
Things run swimmingly for a while, then I get what appears to be empty result set (i.e., no outcomes found).
Constructing treatment and comparator cohorts
Executing multiple queries. This could take a while
|==================================================================================================================================| 100%
Analysis took 0.822 secs
Fetching data from server
Loading took 2.56 secs
Constructing default covariates
|==================================================================================================================================| 100%
Analysis took 9.72 hours
Done
Fetching data from server
Loading took 4.11 mins
Removing redundant covariates
Normalizing covariates
Constructing outcomes
Executing multiple queries. This could take a while
|==================================================================================================================================| 100%
Analysis took 6.27 secs
Done
Fetching data from server
Loading took 0.306 secs
Error in UseMethod("open") :
no applicable method for 'open' applied to an object of class "data.frame"
In addition: Warning message:
In lowLevelQuerySql.ffdf(connection, sql) :
Data has zero rows, returning an empty data frame
What puzzles me is why the result set would be empty when I have confirmed patients do indeed appear in the outcome group. I assume I have done something incorrectly in my configuration. Has any one else had this problem?
I am happy to post my configuration but didn’t want to bomb people’s inboxes.
Thanks
Jon
Hi Jon,
I think this thread was actually just fine in the issue tracker (keeps the issue close to the offending package). I’ll reopen it there.
Hi all,
Please be aware that a major new version of CohortMethod has been released. See here for details.
In the statistical tutorial - the speakers referenced several statistical textbooks that are useful. At the time of the tutorial, I did not take note of those.
To dig deeper into statistics (related to CohortMethod), which [text]books would you recommend as good reference textbook?
I think one mentioned book was this by Rosenbaum (but it is from 2010 which seems not that recent)
We just released CohortMethod v3.0.0!
The changes, as already discussed in the population-level estimation workgroup, are the following:
Adding inverse probability weighting: see inversePtWeighting argument in the fitOutcomeModel function. This is also described in the single studies vignette.
Adding interaction terms: see interactionCovariateIds argument in the fitOutcomeModel function. This is also described in the single studies vignette.
Adding summary statistics to plots: see show... arguments in the plotPs and plotCovariateBalanceScatterPlot functions.
Enforcing consistency in language: for example, createDrugComparatorOutcome has become createTargetComparatorOutcome. There are many small changes, and unfortunately it probably means your old code won’t work anymore.
Relative instead of absolute paths: the result of runCmAnalyses used to have absolute paths to the various files. Now these are relative to whatever was the outputFolder. Note that the summarizeAnalyses function now requires the outputFolder as argument.
Compression of intermediate results: see compress argument in the saveCohortMethodData function and the compressCohortMethodData argument in the runCmAnalyses function.
In addition, there are many improvements in speed that will likely only matter if you run millions of analyses (which few people seem to do).
If you still require the old version of CohortMethod, please run:
devtools::install_github("ohdsi/CohortMethod", ref = "v2.6.2")
I’ve also updated the population-level estimation study package skeleton to be compatible with CohortMethod v3.0.0. Another new feature of this skeleton is that it exports to a well-defined data model.
Thank you for great work, @schuemie I’ll make the Cohort Study with the new skeleton package. Thank you again!
Just a heads up that a late minute change introduced an annoying bug in v3.0.0 (stratified population files were not being generated.). Please use v3.0.1 instead.
I feel like I’m the only one having trouble of installing v3.0.1.I’ve tried many different ways, but none of those worked out. Am I supposed to get rid of the old version first? I’m trying to installing it in the server that’s not connected to internet. Could it be the reason of the error?
the error message says:
cannot open compressed file ‘CohortMethod/DESCRIPTION’, probable reason ‘No such file or directory’
Error in install.packages : cannot open the connection.
by the way, I used the code
install.packages() with the option repos=NULL, type=“binary”
Any help or comment would be appreciated. Thank you.
The correct way to install the package is:
install.packages("devtools")
devtools::install_github("ohdsi/CohortMethod")
Could your try that?
Thank you very much for your reply.
I tried those codes and it had another issue.
tools::install_github(“ohdsi/CohortMethod”)
Error in curl::curl_fetch_memory(url, handle = h) :
Failed to connect to api.github.com port 80: Timed out
That looks like a network problem when connecting to Github. Could you try again? Maybe your institute’s firewall is blocking the connection?
Maybe it’s trying to get to port 80 (unsecured) when github (I think) only uses SSL port 443?
If that is the case, is there any solution for this problem? I haven’t solved it yet!
@JUNGEUN_PARK
What about downloading the whole github repository of CohortMethod, and then installing the library by using local github repo?
git clone https://github.com/OHDSI/CohortMethod.git
Thank you for your reply! Actually I’ve tried that, but had another issue. ’
Below is the code I used to install CohortMethod.
filename<-"CohortMethod-master.zip"
install.packages(filename,type="binary",repos=NULL)
and I could see “CohortMethod-master” folder(not .zip) newly created,
but when I tried to load package by running library(CohortMethod), (I’ve also tried “CohortMethod-master”)
it didn’t really work.
there is no package called ‘CohortMethod'
I think I’m missing something, but have no idea how I could fix this.
Thank you very much.
Could you try to clone the git repo to local repo of yours (using ‘git clone’ in command) and then launch the .Rproj object in that repo? Then, click the ‘build and reinstall’ button in the RStudio. I think this would be the easiest way.
I think @SCYou’s approach would indeed be easiest. Alternatively, you can download the source package here.
@SCYou
Thank you very much for your help!!