Heracles just uses the general getCohortDefinition, which I thought was the one that Circe uses, so unless the ‘new’ version has a different path, it should consume whatever changes Chris makes.
Heracles uses this.
Alex, give Marcel’s diverging schemas for Cohort and Results:
Results Database Name / Schema: public
Cohort Database Name / Schema: cdm
I would have intuitively expected flyway to create the Cohort tables in the ‘cdm’ schema rather than the ‘public’ schema. Is this mistaken?
That dir is what Circe uses, so I would say no, they are not obsolete. However, there may be certain things that are such as the SQL above. I would defer to @Chris_Knoll on that though.
@jon_duke, I think it would use public in your example, because I’m not sure that cohort schema is actually used anywhere.
Flyway will only manage the schema you tell it to and by default (in Olympus) it is the ohdsi/results schema, in this case public. I believe it has always been the intent to mange the ohdsi/results schema. I’m a bit unclear what the database.cohort.schema property is used for. @cahilton or @Chris_Knoll, can you summarize?
@mdewilde, can you clarify on these questions:
Does Circe show cohorts and Heracles not? That would be a Heracles problem.
If you can’t see them in either application, then that’s a WebAPI config probem.
Do you get any results back entering this in your browser once Olympus is running?
@jon_duke , I don’t think he’ll get any results back using the WebAPI endpoint, because that should look at public.cohort_definition, which will be empty until Marcel creates a cohort definition.
@Chris_Knoll or @cahilton , does Circe create cohort definitions via WebAPI and the cohort_definition table created by WebAPI/Flyway in V1.0.0.3?
@Chris_Knoll , does Circe still use the following file for anything? Seems like this file should change to reflect the correct table definition (created by WebAPI):
Anyway, the troubleshooting here is mainly cohort logistics and what apps use, so I’m not sure I’m providing much help here. I defer to @Chris_Knoll & @cahilton
H everyone. This is a lot of communication and I haven’t read everything in detail so forgive me if I’m restating anything that’s been said:
Firstly, Marcel: You have everything set up correctly. It was expected that after you did your re-build, you’d have CDM tables in your cdm schema, and the WebAPI (ohdsi) tables would be in public. That’s what you have, and it is completely correct that you have a cohort_definitoin table and cohort table in BOTh the public and cdm schema. The cohort_definition table is not identical between the public and cdm schemas, but the cohort table is.
Next: The webAPI reads available cohort definitions from the webAPI’s cohort_definition schema, in your case ‘public’. If you already have existing cohorts in your CDM, you must move the data from your CDM into the WebAPI schema (‘public’) using a manual process or I think charity did implement something to do this operation because Jon’s enviornment has this exact need: He generates cohorts from an NLP process which he wants to analyze in HERACLES, so he performs an operation that coppies the data into the WebAPI cohort schema.
The cohort_schema property: Per @alfranke comment, there was a question about what this property does. It’s no longer used. This is how the CohortGenerationTasklet (the cohort processor for CIRCE) configures the job:
JobParametersBuilder builder = new JobParametersBuilder();
builder.addString("cdm_database_schema", this.getCdmSchema());
builder.addString("target_database_schema", this.getOhdsiSchema());
builder.addString("target_dialect", this.getDialect());
builder.addString("target_table", "cohort");
builder.addString("cohort_definition_id", ("" + id));
It will us the CDM tables from the cdmSchema, and write to the “cohort” table in the ohdsiSchema (‘public’). At one point there was a field for cohort schema, but then we realized the case where you may have created other cohorts in your CDM cohort table that would have a conflicting cohort_definition_id, so we made it so that the CIRCE cohort generation ALWAYS writes to the ohdsiSchema.cohort table using the COHORT_DEFINITION_ID from ohdsiSchema.cohort_definition.
This is this reason that in order to get your custom cohorts you built in your CDM into HERACLES you must insert records into the ohdsiSchema to get the corect cohort_definition_id, and then insert (copy) the cohort rows from the cdm.cohort table into the ohdsiSchema. Once that’s done, you will see your cohort appear in HERACLES, and you will be able to do analysis on it.
Ok, I think that’s all the information I can give for now, I hope that clears up some of the confusion above. Please let me know if you have any additional questions.
-Chris
@alfranke,
That must have come from an old checkin becuase the cohort_definition table in the OHDSI schema has the ‘cohort_definition_syntax’ called out to a different table entirely ‘cohort_definition_details’. This was to deal with cases where large character blob support might be different in the different database platforms. In addition, the ohdsi.schema has a last modified date, who modified it (for when we have a security layer implemented) other features as well.
If HERACLES uses the cohortdefinition/ webapi to get the cohort definitions, then it’s using JPA and the tables created by flyway to query the database table. JPA is used becuase this table is read/write, manages relationships across tables, and does the string encoding to prevent SQL injection attacks.
The only purpose for the cdmSchema’s cohort_definition table is to maintain cohort definitions that are not managed by CIRCE.
To answer your other question:
CIRCE uses the tables defined in migraiton 1.0.0.3 to save/load cohort definitions. This migration script was specifically to support CIRCE.
-Chris
Hi Jon,
Circe: if I start Circe from the Main windows I just get an empty screen, only the back ohdsi header is there. Also after waiting for a couple of minutes…. Still empty.
Typing the line in the browser gives me a nice picture of almost naked people standing on top of a small waterfall. Unfortunately his message is: “Oops! Something bad happened. Go home.”. Hahaha, this is a good one. Given the local time (18:23) this is a good idea. 
Oh wait a minute after clicking go home I see that the WebAPI is not running. Sorry about this. Went to configuration and pressed launch (do we have to do this every time we start Olympus or is this an indication of an error???).
New attempt: circle gives same empty window.
Line in browser tells me just:
[]
I assume this is an empty list. Hope this helps you figuring out what’s going on.
For today I will follow the “Go home” advice from a couple of minutes ago.
Thanks again,
Marcel
Re: CIRCE:
I made a checkin 3 days ago where we’d fallback to a local copy of the javascript libraries if we could not load them from the CDN. Has Olympus packaged up this updated code?
What should happen at this point for you, Marcel, is that when you go into CIRCE, you should get nothing found because you haven’t created a CIRCE cohort definition. This is normal, and to just try some things out, you should click new definition and play around with building a cohort using CIRCE. Note: I understand you are getting a blank screen, I’m guessing htis is because you have an outdated CIRCE and it’s trying to load some things from the internet. Latest build accomodates local enviornments only.
If you can start with a fresh CIRCE cohort expression, look for pepole with a specific condition, for example, and then generate it, you should see it in the HERACLES runner, and be able to generate reports on it. Let’s start with getting you to that point so that we know CIRCE/HERACLES works.
Later, we’ll work on a way to import your cohort definitions from your CDM schema into CIRCE so that HERACLES will operate on it.
-Chris
You can auto-start WebAPI with the following property to the java process
-Dolympus.webapi.launch.enabled=true
I’m working on building a new snapshot, which will include Chris’ change to Circe. @jon_duke will have to make this available to you Marcel.
Hi Chris and Alex,
First making sure Heracles works before touching my own cohort makes sense. I understand. I also have a backup of the database just before starting Olympus just as a fallback. So we can also take risks that might be messing up the database.
The version I run now is 1.1.0 downloaded from:
http://www.ohdsi.org/analytic-tools/olympus-a-unified-platform-for-ohdsi-web-applications/
On GitHub i find a release v1.0.0 and thats older:
You describe modifications only 3 days old. I dont see a release that recent on both websites. Is this correct? Is there a release including these changes I can download? Whats the primary location for releases? Is there a third location I’m not aware of?
What about the following scenario? Desclaimer: I’m one of this rare specimens on earth that develops everything in Delphi (I have my reasons…) and doesn’t know a thing about Java (again I have my reasons…). So maybe this is completely nonsens. Is it an option to install Olympus on my laptop first and configure it on a empty database? This laptop has an internet connection so everything that can be downloaded at startup if needed. Is it then possible to copy the content of the Olympus directory from AppData to my server and run it again there? Does it contain all updates by then? Or does this Delphi developer have no clue what he’s talking about?
Other sugestion I want to check with you guys before messing up the database. What about if I try to insert a record in the public.cohort_definition table? Should this popup in the Heracles tool? In that case I might not need Circle at this time.
Thanks again and again and again,
Marcel
Guess I was the one with the most to learn!
First, we are going to start a regular monthly release of OLYMPUS with all
the newest versions of our apps. Alex can provide a sense of when we can
get the modified CIRCE incorporated to allow running without the internet.
Second, maybe for Chris, how is the cohort schema configuration intended to
be used? Is it used by the WebAPI at present? Seems like we need to send
it to the right place from Olympus.
Thanks Alex I will add this one to my startup cmd tomorrow.
Jon,
The original idea around the “cohort schema” started with the idea that the CDM schema may not be writable and therefore we could not write into cdmSchema.cohort. We established a cohort table in the ohdsiSchema that could be set via ‘cohort schema’ to write to. However, we found that if a CDM already had cohorts identified in it (let’s say cohorts 1,2,3,4,5) that is completely unknown to WebAPI, then installing WebAPI to point to that CDM would start writing it’s OWN cohort (1,2,3,4,5) to the cdm schema, thus blowing away any existing CDM cohorts that you already had.
Conclusion: forget about "cohort schema’, and always write to ohdsiSchema.cohort table within WebAPI processes. The cohort_definition_id will be the cohort_definition_id comming from the ohdsiSchema.cohort_definition table, and if you ever wanted to do analysis on any pre-existing, pre-webAPI cohorts, you have to import them into the ohdsiSchema by creating a cohort_definition in ohdsiSchema.cohortDefinition, and use THAT cohort_definition_id when importing the data from cdmSchema.cohort table.
To be honest, I though you were already doing this at your site where you define cohorts through an NLP process and then import the results in to the OHDSI cohort table/cohort definition table so that you can run HERACLES analysis on it.
If you are not, how are you doing that?
-Chris
Marcel,
Installing Olympus on your laptop pointing it to an empty database will just cause the OHDSI tables to be created that blank database. Any configuration files that get produced during this time can be copied over to any other location that hosts the java application, it’s true.
The piece I think you are missing is that the web applications (the HTML and javascript that is run inside the browser) refer to external resources via script tags to core libraries so that we don’t have to copy these libraries over and over in our repo. But for you, to have an offline mode, we’d have to take special steps, something that I think the Olympus installation can manage, but I don’t know enough about it. But it’s NOT the case that when you install Olympus it does an “install-time download form the internet so that future executions can run with cached code locally”. It’s just not how that works. It may be possible to package up Olympus as a completely self contained application where all 3rd party internet hosted assets are put locally. But Jon would have to comment on if that’s possible.
-Chris
Ok, thank you @alfranke, looks like the ‘cdn fallback’ support was just merged into Olympus, so if you pull it down you should get CIRCE working:
My suggestion is once you have the latest build, you open up circe, create a simple cohort (maybe look for pepole with a common condition in your CDM), and GENERATE it. Once generated, you should be able to move over to Heracles, search for it in the dropdown, and generate reports for it.
-Chris