We are using the ‘Search’ tab in ATLAS and trying to look for ‘hypertension’ related concepts.
We get around 1300+ results for hypertension, but the RC and DRC counts are 0 (zero) for all of them.
We checked CONDITION_OCCURRENCE table in our CDM for few of them (the ones with Domain=‘Condition’ and Vocabulary=‘ICD9CM’) and got row count greater than 0 (infact in the range of 100k rows).
We have been trying to find the root cause/source of this.
We tried to check the activity on the database to identify the query being hit (PostgreSQL) to retrieve this result but don’t see any query.
Can you please help us in identifying the source of the values that get populated under RC and DRC columns?
@AnthonyMolinaro may have better insight to those columns, since the behavior has changed since the initial introduction of those columns in the UI. My current understanding is that those statistics come form the Achilles results But Anthony may have more up-to-date information on that.
As far as finding ICD9 concepts in the CONDITION_OCCURRENCE table, which column are you looking at: SOURCE_CONCEPT or CONDITION_CONCEPT? If you’re finding them in the CONDITION_CONCEPT_ID column, then I think you have an ETL problem because those ICD9’s should have been mapped over to the standard concepts (found in SNOMED).
I am looking for the concept-ids in CONDITION_SOURCE_CONCEPT_ID column of CONDITION_OCCURRENCE. Along with this, I am also checking on CONDITION_SOURCE_VALUE for the concept codes.
Ok, thanks for confirming. Initially, we were only showing record counts for standard concepts in the output, however, my understanding is that recent changes have looked at both the standard concept field (condition_concept_id) and the non-standard concept field (condition_source_concept_id), although I’m not exactly certain where those changes were made (if it’s a new version of Achilles, or if it is some additional script that does that).
We are using Atlas version 2.7.4 with WebAPI version 2.7.4.
We indeed had run Achilles analysis. I checked for analysis-id 425 and for condition_source_concept_id=‘44834715’ in achilles_results table in results schema and found a non-zero value. Below is the screenshot -
We did some more analysis and executed few more queries on the database to ensure that we have proper data in the tables.
We executed the query of getConceptRecordCount on our database and it returned a non-zero count.
We also tried refreshing cache for the data source from the Configuration tab/page of Atlas and executed the loadConceptRecordCountCache query to ensure that non-zero counts are present. But still the count on Atlas is being displayed as 0 (zero).
@Chris_Knoll – I’m at a loss here. From an Achilles perspective, nothing has changed recently with any of the codebase. I’ve recently run it and added a CDM to Atlas and all the record counts come through.
@CSC – have you set up your source correctly in WebAPI? That is, does the source daimon for the results have a priority of 1?
Hello @Ajit_Londhe, @Chris_Knoll
The priority was set to 2 for results in webapi’s source_daimon table.
I changed it to 1, logged-in to Atlas again and performed the search for ‘hypertension’ but still zero count under RC and DRC.
Should I 'Refresh Cache’ for the data source from the Configuration page in Atlas?
I’m not sure either. The getConceptRecordCount query is the query that is executed at the conceptRecordCount endpoint. However, that endpiont is a POST, so you can’t just go to the URL and look for a result (it should give you something like ‘not allowed’).
When I search on my local system, and look at the console, I see the request, and the payload:
My only hypothesis is that the source configuration is looking at the wrong schema and the wrong schema just has empty tables in it. I can’t understand why you’d get results out of a raw sql query to the DB, but the server’s doing the same thing (supposedly) and getting nothing…
