OHDSI Home | Forums | Wiki | Github

Using WhiteRabbit with Spark, Databricks (Delta Live tables) or Snowflake

Hi everyone,

Hope you are well.

We are looking at building a pipeline where the data volume/size is of the order of terabits. We want to ensure both the source data as well as the CDM data is compatible with analytical tools for big data as well as the OHDSI analytical tool kit.

Since the scanReport from WhiteRabbit tool is integral to the mapping process, wanted to check if WhiteRabbit supports the use of any of the following:

  • Spark
  • Databricks (Delta Live tables)
  • Snowflake

Many thanks,
Solmaz

See following GitHub issue for further details - Using WhiteRabbit with Spark, Databricks (Delta Live tables) or Snowflake · Issue #394 · OHDSI/WhiteRabbit · GitHub

Tagging @Brad_Rechkemmer – Brad, I think we had a local copy of WhiteRabbit working with Spark?

t