Atlas trying to read vocabulary-schema tables in the cdm-schema

Implementers
1

A company is trying set up Atlas in our enviroment.

The system is up and working, but Atlas triggers multiple errors in when calling the webAPI.
For example:

Caused by: com.simba.googlebigquery.support.exceptions.GeneralException: [Simba][BigQueryJDBCDriver](100032) Error executing query job. Message: 404 Not Found
{
  "code" : 404,
  "errors" : [ {
    "domain" : "global",
    "message" : "Not found: Table fg-dev3-library:finngen_omop_cdm.concept was not found in location europe-north1",
    "reason" : "notFound"
  } ],
  "message" : "Not found: Table fg-dev3-library:finngen_omop_cdm.concept was not found in location europe-north1",
  "status" : "NOT_FOUND"
}
        ... 113 more

It seems Atlas tryies to access the table concept in the cdm_schema. However, the table concept should be in the cdm_schema.

They follow the instructions in https://github.com/OHDSI/WebAPI/wiki/CDM-Configuration and configure 4 schemas in BigQuiery.

Is there any configuration we are missing by which in BigQuiery the tables should be defien diferently ?? Or any other clue ??

(from what i’v seen on post @mgkahn seems to have experience in installing CDM on BigQuiery)

Thanks.

2

In case someone run on the same problem

we solved this by combining the schemas of vocab and cdm and reconfiguring source daimons for Atlas to recognize them from the same dataset .

3

hi @Javier - the vocabulary tables are a part of CDM and should be always siting in the same schema together with data that these vocabs were used to convert.

4

thanks @gregk,

I speak on behalf of the thechnical person installing this:
I think that what was confusing for him is that
webAPI configuration documentation ( here ) uses one schema name for each giving the feeling that these should be in separated schemas.

1 Like
5

@Javier,
You are correct that the webAPI documentation does describe multiple schemas when setting up a source:

schema permissions
cdm read-only
vocabulary read-only
results insert/delete/select/update
temp full control (create/remove tables & data)

Depending on your version of WebAPI and Atlas, the ability to use an external vocabulary schema for a WebAPI process may not be completely supported. A stand-alone vocabulary schema makes sense if you want to set up a data source only for vocabulary browsing (so only the vocabulary tables of the CDM schema would be populated).

But, @gregk is correct that the CDM schema encapsulates the vocabulary tables. So, for simplicity, when you are setting up a CDM schema, you should populate it with the vocabulary tables as well.

6

Hi Chris,

Are you saying that this separation will not be supported in upcoming WebApi releases? I ask because I’ve found several places in the WebApi code where the CDM schema is used instead of the Vocabulary schema, for example, with the concept and concept_ancestor tables. Ideally, you would want these separated in your vocabulary schema so they can be used with different data sources. Why would you want to duplicate data?

Thank you.
Alla

7

The solution to duplication of data is to create views in each CDM schema from a ‘shared’ vocabulary. No duplication when you create a view.

There seems to be a misunderstanding about the vocabulary schema: there isn’t actually a separate vocabulary schema in the CDM specification. There are vocabulary tables that are part of the CDM schema, but there’s not an actual specification of a ‘vocabulary schema’. The reason for the confusion is that the WebAPI group accepted a request to define vocabulary schema independently from the CDM schema for some function, but I don’t think any (or very few) of the other HADES packages that access CDMs allow specifying a ‘vocabulary schema’ as a execution setting.

What I meant when I posted above, is that older versions of WebAPI/Atlas did not support the vocabulary schema setting, but newer one’s did. However, due to the confusion this causes (and that a simple solution is to just create views) it may make sense to remove this functionality in a future release, but we haven’t made those plans yet.

8

Thank you very much.