Can Arachne Execution Engine be connected in Atlas + WebAPI?

Developers
, , ,
1

We are currently operating our CDM research portal using Atlas + WebAPI.

Our Atlas and WebAPI versions are 2.13.0 (not the latest).

So far, we have been downloading research packages for Estimation and Prediction and running them locally by executing the R project files. Since this is cumbersome, we would like to automate R-based research.

  1. Are there any examples of running and connecting WebAPI + Arachne Execution Engine + Hades together?

  2. We are trying to modify package versions inside Hades2023q3 and run it with Docker. However, when we run execution-environments/Hades2023q3 locally,

    R-installs.R (line 15 - Download Modules Specification files)

    fails to execute beyond that point.

    Is it possible to run Hades locally?

  3. Then, is there a specific version of WebAPI and Atlas that is compatible with the currently released odysseusinc/execution_engine:latest? If so, could you let me know?

Steps to reproduce

We found that WebAPI provides APIs to request Estimation and Prediction execution.

For example:

runGeneration() → [POST] /WebAPI/estimation/{id}/generation/{sourceKey}

Therefore, we tried to connect WebAPI with the Arachne Execution Engine running in Docker.

I downloaded the following Docker images and configured callbackURL and related settings in WebAPI for the connection:

  • docker pull odysseusinc/execution_engine
  • docker pull odysseusinc/r-hades:latest

Then, I started the executions container on the VM with the following command:

docker run --rm \
  -p 8888:8888 \
  --add-host=host.docker.internal:host-gateway \
  -e RUNTIMESERVICE_DIST_ARCHIVE=/runtimes/r_base_focal_amd64.tar.gz \
  -v /home/megabridge/execution_engine/runtimes:/runtimes \
  --privileged \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /home/megabridge/execution_engine/executions:/etc/executions \
  -e DOCKER_IMAGE_DEFAULT=odysseusinc/r-hades:latest \
  -e DOCKER_IMAGE_FILTER="odysseusinc/r-hades(.+)" \
  -e ANALYSIS_MOUNT=/home/megabridge/execution_engine/executions \
  odysseusinc/execution_engine:latest

When testing a request, I encountered the following error:

2025-07-03T07:38:07.551Z DEBUG 19 --- [eam--1388767997] c.o.a.e.execution.r.DockerOverseer       : Execution [61] STDOUT: Error in (function (databaseDetails = createDatabaseDetails(), modelDesignList = list(createModelDesign(targetId = 1,  :
2025-07-03T07:38:07.551Z DEBUG 19 --- [eam--1388767997] c.o.a.e.execution.r.DockerOverseer       : Execution [61] STDOUT: unused argument (splitSettings = list(0.25, 0.75, 10711, 3))

From this, I found that Atlas passes Training Settings into the analysis JSON, but with the ExecutionEngine + r-hades:latest combination, the splitSettings version does not match, causing PLP execution to fail.

To resolve this, I am trying to build a renv.lock using the R package versions I used locally. I attempted to build a Docker image with execution-environments/Hades2023q3, but I keep running into failures.

Since the analysisSpecificationsUrl could not find the raw GitHub URL at that location, I tried replacing it with a similar file, but it still did not work:

Strategus - cdmModulesAnalysisSpecifications.json → i change it to rawgithub~~

This file also failed to run.

144.8 1: In download.file(analysisSpecificationsUrl, destinationFile) :
144.8   URL https://raw.githubusercontent.com/OHDSI/Strategus/develop/inst/testdata/cdmModulesAnalysisSpecifications.json: cannot open destfile '/strategus/modulesSpecification.json', reason 'No such file or directory'
144.8 2: In download.file(analysisSpecificationsUrl, destinationFile) :
144.8   download had nonzero exit status
144.8 Error in ParallelLogger::loadSettingsFromJson(destinationFile) :
144.8   File '/strategus/modulesSpecification.json' not found
144.8 In addition: Warning message:
144.8 In normalizePath(fileName) :
144.8   path[1]="/strategus/modulesSpecification.json": No such file or directory
144.8 Execution halted
------
Dockerfile:75
--------------------
  74 |     COPY R-installs.R /tmp/R-installs.R
  75 | >>> RUN Rscript /tmp/R-installs.R
  76 |
  77 |     RUN mkdir /results
--------------------
ERROR: failed to solve: process "/bin/sh -c Rscript /tmp/R-installs.R" did not complete successfully: exit code: 1

Any guidance or examples would be greatly appreciated