We have more-or-less successfully composed Broadsea on CentOS 7 (the Hades container not withstanding). We are having problems loading additional vocabularies to the system.
It starts out looking good:
[drando2@toolbox Broadsea]$ docker compose --profile omop-vocab-pg-load up -d
[+] Running 2/2
Container omop-vocab-load Started 0.0s
Container traefik Running 0.0s
But then the container does not materialize. I expect it to spawn a container that will load the vocabularies stored locally:
[drando2@toolbox files]$ ls -l ~/repo/Broadsea/omop_vocab/files
total 5736216
-rw-r--r--. 1 drando2 domain users 1747031305 Dec 5 14:48 CONCEPT_ANCESTOR.csv
-rw-r--r--. 1 drando2 domain users 17697 Dec 5 14:48 CONCEPT_CLASS.csv
…
But nothing happens. Has anyone else run into this?
Ah, the broadsea-atlasdb container wasn’t running because a local PostgreSQL instance was squatting on the default port. So everything seems to be in order on my Mac.
My original problem is on Linux, and it appears to be related to a networking problem that prevents the Postgres client from being installed:
[drando2@toolbox Broadsea]$ docker logs omop-vocab-load
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/main/x86_64/APKINDEX.tar.gz
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.17/main: network error (check Internet connection and firewall)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/main: No such file or directory
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/x86_64/APKINDEX.tar.gz
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.17/community: network error (check Internet connection and firewall)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/community: No such file or directory
ERROR: unable to select packages:
postgresql-client (no such package):
required by: world[postgresql-client]
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/x86_64/APKINDEX.tar.gz
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.17/main: network error (check Internet connection and firewall)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/main: No such file or directory
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.17/community: network error (check Internet connection and firewall)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/community: No such file or directory
ERROR: unable to select packages:
postgresql-client (no such package):
required by: world[postgresql-client]
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/x86_64/APKINDEX.tar.gz
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.17/main: network error (check Internet connection and firewall)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/main: No such file or directory
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.17/community: network error (check Internet connection and firewall)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/community: No such file or directory
ERROR: unable to select packages:
postgresql-client (no such package):
required by: world[postgresql-client]
On the Mac, I was surprised not to see the updated vocabularies in ATLAS. This is because by default omop_vocab_load populates a separate schema called omop_vocab, which is not accessed by ATLAS. ATLAS keeps looking at the demo_cdm schema, which has its own set of vocabulary tables. You need to change the setting in Section 9 of the .env file:
VOCAB_PG_SCHEMA="demo_cdm"
and then rerun the
docker compose --profile omop-vocab-pg-load up -d
command. This makes the new vocabularies visible to ATLAS.
Another key point is you can access the broadsea-atlasdb PostgreSQL database running in the container through localhost:5432 on a Postres client. I am using DbVisualizer. The userid is postgres, and password is mypass.
It seems that ATLAS has access to parts of the WebAPI that Joe User is not. Is there a way around this? I am trying to compile a list of source codes that are included in a concept set, and I would like to do this through the WebAPI.
The Linux install of PG Client failed, but on Mac, it worked fine.
In develop, we’ve changed to Alpine 3.18.2, but it looks like pg client should be available in 3.17.
I’ve added a call to apk update to hopefully resolve this.
While the omop-vocab-pg-load profile will load whatever batch of Athena downloaded OMOP Vocab files into a PG schema of your choosing, in Atlas, you still need to configure the app to use this vocab. The default source/source daimon setup that “ships” with Broadsea will indeed use an outdated vocab from the Atlasdb container.
Indeed the atlasdb PG instance is accessible on the host, we’ll look to add a section to the Readme on this.
Not all API calls are open, especially if you have security enabled, you would need to authenticate and have a user role that can access it.
To get all source codes in a concept set, I’d highly recommend using the ROhdsiWebApi package, as that nicely wraps all the API calls into functions, and even supports authentication.
