Some cohorts require significant time to be generated. Of course, it’s hard to judge from “has many rules” and “measurements and drugs as usual are huge”, but 45 minutes sounds quite realistic in this case.
You can try adjusting the indexes (if it’s a classical DBMS) or playing with data distribution/partitioning (if you’re using something like Redshift), this may help improving the generation time significantly.
@rookie_crewkie - Thanks for the response. We use classic RDBMS and it’s been running now for more than 3 hours but still no output. When I did ctrl+shift + i, I see the below error
Less has finished and no sheets were loaded.
Application.js:210 Initializing service information
Application.js:238 Init sources from server:8899/atlas/undefinedlookup/identifiers:1 Failed to load resource: the server responded with a status of 404 ()
autoGrowInput.js:46 newWidth: 30
autoGrowInput.js:46 newWidth: 17
autoGrowInput.js:46 newWidth: 23
autoGrowInput.js:46 newWidth: 30
autoGrowInput.js:46 newWidth: 17
Clicking the URL in error message, shows the error
No, sorry, haven’t seen this before. Seems unrelated though, because in case Atlas couldn’t fetch the list of sources completely, you wouldn’t be able to see and use them at all, which is probably not the case since you’re already running cohort generation. (Judging from the code logic here)
You could try:
running the SQL code for the cohort generated by Atlas manually, to see if there’s a specific query that gets stuck;
asking someone who has more insight into Atlas tricks - unfortunately, I’m just an Atlas user.
Generally, the “heaviest” part of cohort generation is when the CDM event table(s) are filtered by concept_id(s) that you’ve included in the concept sets. Therefore, building an index on the columns containing these concept_ids should theoretically help to speed up this part.
Thanks @rookie_crewkie. But as concept table is huge, it is indexed. I checked that as well. Even my previous version (without concept addition) working cohorts suddenly throw this error under console tab. But it was working fine last week (took only less than 30 mins), meaning I was able to see the cohort person count after cohort generation. But now I see it is just running but no output (even after 4 hours)
But I see another cohort which is generated successfully also has this error message (under console tab) when I press “ctrl+shift+i”. The cohort is successfully generated and I see the number of people in cohort. There is no red color message in “network” or “sources” tab
So what could be the reason as to why cohorts which were successful earlier suddenly doesn’t produce the people count? Whereas few other cohort produce the people count. Kindly request you to guide me to fix this?
@Chris_Knoll - Can you share us some insights on this issue?
The url that starts with something like xxx/atlas/undefined is wrong, which indicates some problem with your vocabulary source key (the key that’s used to query the vocabulary).
I don’t know what caused that problem, but for that particular error, the URL is being constructed wrong.
When you insert a source into the webAPI schema table: source, you specify the source_key that is used to indicate which database to access.
For some reason, the Atlas configuration is coming up with ‘undefined’ when performing the vocabulary queries. I’m not sure the reason for this, but I would double check your configuration.
