OHDSI Home | Forums | Wiki | Github

Confused about Atlas and pointing to CDM

Developers
#1

Hello again!

So I have installed Atlas, WebAPI, and Achilles. I am a bit confused how Achilles works in this. I have Atlas and WebAPI running however, I don’t believe it’s pointing to my CDM correctly.

The CDM tables exist in a schema called ‘cdm’. I created these tables using these files: CommonDataModel/inst/ddl/5.4/postgresql at v5.4.0 · OHDSI/CommonDataModel · GitHub

I then populated some of these tables from the download from Athena. I followed the instructions as far as I could for setting up Atlas and this link:
http://localhost:8080/WebAPI/source/sources

shows this data:

// 20230628120726
// http://localhost:8080/WebAPI/source/refresh

[
  {
    "sourceId": 2,
    "sourceName": "My Cdm",
    "sourceDialect": "postgresql",
    "sourceKey": "MY_CDM",
    "daimons": [
      {
        "sourceDaimonId": 2,
        "daimonType": "CDM",
        "tableQualifier": "cdm",
        "priority": 0
      },
      {
        "sourceDaimonId": 3,
        "daimonType": "Vocabulary",
        "tableQualifier": "vocabulary",
        "priority": 1
      },
      {
        "sourceDaimonId": 5,
        "daimonType": "Results",
        "tableQualifier": "results",
        "priority": 1
      },
      {
        "sourceDaimonId": 6,
        "daimonType": "Temp",
        "tableQualifier": "temp",
        "priority": 0
      }
    ]
  }
]

However, I’m not sure it is configured correctly as Atlas shows nothing when I go to try to Search something. The vocabulary table I have was populated into my ‘cdm’ schema in the vocabulary table. However, I believe Achilles created a vocabulary schema, but I have no tables for it. What am I missing?

Another additional question, is there any way to generate/insert mock data for my CDM? I am tasked with trying to create our own OMOP CDM to be used, but having an idea of how it looks in the end-product and in the tables themselves I think will be a great help in developing our app.

Thanks everyone!

#2

Matt,

I was recently in the same situation as you and found that there are a lot of different pieces and parts that all have to work together. One suggestion for sample data is to use the Synthea dataset. To use this, you have to download the Synthea data generator and run it which can save the output as a series of CSV files. Then there are R scripts available that can load the CSV data into the common data model. You can read more about it at A Builder for Converting the Synthea Data to the OMOP CDM • ETLSyntheaBuilder

Feel free to email me directly at bkissing@networkhealth.com if you have specific questions.

Bruce

#3

If you’re loading your vocabulary data into the tables in your CDM schema, then change your sourceDaimonId:3 to tableQualifier=CDM. the reason you use a Vocabulary daimon is 1) you want to define a source that is only used for vocab searching, or 2) you want to have a dedicated schema to host your vocabulary tables. If you don’t want either of these, then specify your daimonType:Vocabulary to be your CDM schema.

Achilles won’t create schemas, only tables inside a results schema. If you want the data to be seen by Atlas, then you will tell Achilles to write it’s results into the same schema that you’ve configured your ‘Results’ daimonType.

I think that is what @BruceKissinger is referring to.

#4

Thanks. I believe I did actually create the schemas according to some of the Achilles docs setup.

However, when looking at the source_daimon table. Since you mentioned changing my sourceDaimonId:3 to tableQualifier=CDM. Is the table_qualifier column actually meaning the schema? I was a bit confused about how it knew which schema to find the tables in.

Edit: Ah, I think I see how it gets the schema from the jdbc connection string in the source table.

Edit2: I got it! The issue was my jdbc connection string. I was getting database and schemas mixed up. By changing the string to my database which is named ‘postgres’ (probably default for most). It does appear that the table_qualifier in source_daimon is actually the schema name. Using the catalina.out log I could see errors happening and that helped guide me to the issues.

#5

Yes, the reason why we picked ‘table_qualifier’ instead of schema is that on some platforms (cough, oracle, cough) there is sometimes confusion between a database, schema, user, and other concepts. We opted for the ‘none of the above’ option of ‘tableQualifier’ to indicate that you could put a schema in there, a user (oracle again), you can even put database.schema for MSSQL. The challenges of supporting multiple databases (we used to do postgres, mssql, oracle).

Now that we just support Postgresql in WebAPI db, we can use postgres-specific terminology (we’d put schema as the column name) but that would break backwards compatibility. When we move to a 3.0 version of WebAPI, we will be free to break those things.

1 Like
#6

Thanks for the insight. That makes sense. I assumed it would be something from supporting multiple dbs.

If possible, I would make this clarification on this page as it wasn’t clear to me when I was following it. I also have a bad habit of skim reading so if I missed something obvious then I apologize.

#7

Sure, we can indicate that the schemas should be specified in the table_qualifier column.

1 Like
t