OHDSI Home | Forums | Wiki | Github

ATLAS-Search-RC and DRC are populated as 0


(Chetan) #1

Hello @Chris_Knoll,

We are using the ‘Search’ tab in ATLAS and trying to look for ‘hypertension’ related concepts.
We get around 1300+ results for hypertension, but the RC and DRC counts are 0 (zero) for all of them.

We checked CONDITION_OCCURRENCE table in our CDM for few of them (the ones with Domain=‘Condition’ and Vocabulary=‘ICD9CM’) and got row count greater than 0 (infact in the range of 100k rows).
We have been trying to find the root cause/source of this.
We tried to check the activity on the database to identify the query being hit (PostgreSQL) to retrieve this result but don’t see any query.

Can you please help us in identifying the source of the values that get populated under RC and DRC columns?

Thanks in Advance.


(Chris Knoll) #2

@AnthonyMolinaro may have better insight to those columns, since the behavior has changed since the initial introduction of those columns in the UI. My current understanding is that those statistics come form the Achilles results But Anthony may have more up-to-date information on that.

As far as finding ICD9 concepts in the CONDITION_OCCURRENCE table, which column are you looking at: SOURCE_CONCEPT or CONDITION_CONCEPT? If you’re finding them in the CONDITION_CONCEPT_ID column, then I think you have an ETL problem because those ICD9’s should have been mapped over to the standard concepts (found in SNOMED).


(Chetan) #3

Hello @Chris_Knoll,

Thank you for the prompt reply. :slight_smile:

I am looking for the concept-ids in CONDITION_SOURCE_CONCEPT_ID column of CONDITION_OCCURRENCE. Along with this, I am also checking on CONDITION_SOURCE_VALUE for the concept codes.

Thanks.


(Chris Knoll) #4

Ok, thanks for confirming. Initially, we were only showing record counts for standard concepts in the output, however, my understanding is that recent changes have looked at both the standard concept field (condition_concept_id) and the non-standard concept field (condition_source_concept_id), although I’m not exactly certain where those changes were made (if it’s a new version of Achilles, or if it is some additional script that does that).

@AnthonyMolinaro: do you have this information, or @Ajit_Londhe?


(Ajit Londhe) #5

Achilles does have analyses that characterize source_concept_id fields. For example, analysis id 425, counts condition_source_concept_id values.


(Chetan) #6

Hello @Ajit_Londhe,

We are using Atlas version 2.7.4 with WebAPI version 2.7.4.

We indeed had run Achilles analysis. I checked for analysis-id 425 and for condition_source_concept_id=‘44834715’ in achilles_results table in results schema and found a non-zero value. Below is the screenshot -

I checked for the SQL script (from here)that is used for analysis-id 425 and executed against my database.Below is the screenshot-

I am still unsure why the values are still 0(zero) when being displayed in Atlas
Please help.

Thanks in advance.


(Ajit Londhe) #7

Hmm. Do you have the concept_hierarchy table?


(Chetan) #8

Hello @Ajit_Londhe,

Yes, it’s there under results schema.

image


(Chetan) #9

Hello @Ajit_Londhe, @Chris_Knoll,

We did some more analysis and executed few more queries on the database to ensure that we have proper data in the tables.
We executed the query of getConceptRecordCount on our database and it returned a non-zero count.

We also tried refreshing cache for the data source from the Configuration tab/page of Atlas and executed the loadConceptRecordCountCache query to ensure that non-zero counts are present. But still the count on Atlas is being displayed as 0 (zero).

While refreshing the Atlas->Search page we found that the call through WebAPI to URL https://servername/WebAPI/cdmresults/omop_cdm/conceptRecordCount doesn’t return any response.

Can you please help us in identifying the cause of this issue?

Thanks.


(Ajit Londhe) #10

Has the WebAPI server been restarted since you finished running Achilles? Perhaps the vocab cache in WebAPI hasn’t been refreshed.


(Chetan) #11

Hello @Ajit_Londhe,
We had restarted Tomcat service but not the physical server.
Thanks.


(Ajit Londhe) #12

Oh that’s fine then. I was just referring to Tomcat.


(Ajit Londhe) #13

@Chris_Knoll – I’m at a loss here. From an Achilles perspective, nothing has changed recently with any of the codebase. I’ve recently run it and added a CDM to Atlas and all the record counts come through.

@CSC – have you set up your source correctly in WebAPI? That is, does the source daimon for the results have a priority of 1?


(Chetan) #14

Hello @Ajit_Londhe, @Chris_Knoll
The priority was set to 2 for results in webapi’s source_daimon table.
I changed it to 1, logged-in to Atlas again and performed the search for ‘hypertension’ but still zero count under RC and DRC.
Should I 'Refresh Cache’ for the data source from the Configuration page in Atlas?

Thanks.


(Ajit Londhe) #15

I would restart Tomcat, then in Atlas, refresh the cache in config.


(Chris Knoll) #16

I’m not sure either. The getConceptRecordCount query is the query that is executed at the conceptRecordCount endpoint. However, that endpiont is a POST, so you can’t just go to the URL and look for a result (it should give you something like ‘not allowed’).

When I search on my local system, and look at the console, I see the request, and the payload:

The response is shwoing non-zero values:

My only hypothesis is that the source configuration is looking at the wrong schema and the wrong schema just has empty tables in it. I can’t understand why you’d get results out of a raw sql query to the DB, but the server’s doing the same thing (supposedly) and getting nothing…


(Chetan) #17

Hello @Ajit_Londhe,

As suggested, I restarted Tomcat and then performed the ‘Refresh cache’ in Atlas from Configuration tab.

I am still getting zero counts.

Thanks.


(Ajit Londhe) #18

Can you post your source and source_daimon tables — but abstracted of all login/server info?


(Chetan) #19

Hello @Ajit_Londhe,

Here are the screenshots of source and source_daimon tables -
source

source_daimon

image

Thanks.


(Ajit Londhe) #20

Can you change priority to 0 for source_daimon_id 2 and 4? And then restart Tomcat, clear Atlas cache.


t