Cohort generation - results not updated with new data in OMOP

Implementers
1

In our environment we are adding new data to OMOP on a daily base. We are using the WebAPI to run the cohort generation on existing cohorts. Unfortunately the number of patients gets not updated as expected.
From what I understood this is behaviour is expected as the cohort generation results get cached in table cohort_cache in the results schema. When I delete the content of this table, the cohort generation works as expected and new patients (subjects) get included.
We are using OMOP in a docker setup. To ensure this gets updated, when starting the docker environment we added the pom.xml details to our docker-compose command.
I checked the cache configuration options in the pom.xml and changed it as follows:
CDM_RESULT_CACHE_WARMING_ENABLE: “true”
CACHE_GENERATION_INVALIDAFTERDAYS: 1
CACHE_GENERATION_CLEANUPINTERVAL: 90000
CACHE_GENERATION_USEASYNC: “true”
CACHE_INSPECTION_PERIOD: 90000

Unfortunately the WebAPI cache config documentation is limited. I could not find any more details on how to configure the WebAPI to make this working.

Is there a solution, work around, config to let this work for us?

Thanks a lot for your help. Cheers Ines

2

Hi @ines.reinecke, we have actually had the same issue in my organization, but have not found a way to update the WebAPI. However, every time we update the CDM, we are cleaning out the cache tables that are produced by WebAPI/ATLAS cohort generation. Then the cohort_ids and design hashes are regenerated and produce the correct count. It would be great to hear back on whether you have found a better solution for this issue in the meantime.

KR,
Adamantia

1 Like
3

Yes, that’s correct: the cohort cache doesn’t know about underlying CDM data changes. If you update your cdm data, truncate the cohort_cache table in your results schema, and cohorts should re-generate instead of using cached results.

1 Like
4

Hi Chris,
Thank you for redirecting me to this discussion.