OHDSI Home | Forums | Wiki | Github

Managing Version Dependencies

Developers
, ,
#1

A topic that was discussed on the Architecture call today was how to manage version dependencies between the various components of the architecture. The purpose of this post is to introduce the discussion we have been having and allow for comment from the community.

We currently have evolving versions of practically every aspect of our architecture including our R packages, CDM specification, WebAPI, ATLAS, etc… As these platforms evolve we will need to clearly document what dependencies exist for running a particular part of the platform.

For example, today any version of ATLAS beyond 2.0 requires at least version 2.0 of the WebAPI. In this particular case (ATLAS and the WebAPI) we are keeping the version numbers in sync. If you want to run 2.2 or ATLAS you should run 2.2 of the WebAPI. It is not the case that all version numbers are currently in sync. For example the CDM specification has currently released version 5.3. While no one is running their CDM in version 5.3 today, they soon will be. Here is where our conversation began.

Changes in the CDM V5.3 specification will change tables and will require changes to the WebAPI. It will also require changes in ATLAS to support a designer for the new cohort definitions that will be possible. Here are a few of the issues that were discussed:

  • The WebAPI has no property in data source configuration that tracks the version of the CDM specification for that particular data source.
  • ATLAS does not know the version of the underlying WebAPI
  • WebAPI does not have a list of “supported features” that it can provide to help control what the UI presents to the user.
  • In the case where a user were to create a cohort definition that requires the table updates found in CDM v5.3, there is no way to know that the cohort definition requires that CDM version, and if that cohort definition were shared to an environment where their CDM is still on versions prior to v5.3 how should that be handled?

These issues will certainly require our attention. I invite others from the call to comment on other topics that I did not capture here and invite others from the community to provide comment as we consider possible solutions.

1 Like
#2

@Frank:

Couple points:

  1. If ATLAS and Web-API are in sync, and the latter is only used as the former, why are we having them in two separate modules? Or are you anticipating other clients of the Web-API? If so, I would not put them in sync at all.
  2. I think all these modules should know their version, being able to communicate them, and know which versions of other modules they can interact with. Reason is this: The field is going to spread significantly over time: Folks will be slow catching up, will invest in tools and CDM versions at different speeds, and it will be a mess.
#3

I completely agree with Christian—at least about the “it will be a mess” thing.

As the Architecture Working Group is beginning to formulate specifications for data structures and APIs to deal with versioning and sharing of concept sets and cohort definitions, vocabulary and CDM dependencies, etc., there are going to be more and more entities with versions and dependencies: like the concept set spec itself as well as individual concept sets. Some discussion arose yesterday about whether versioning or version dependencies should be specifiable for particular ATLAS or WebAPI features in addition to whole versions.

I don’t know what the solution is, but I’m hoping that this is a problem that’s been solved somewhere else at some point. I think if we start implementing piecemeal version dependency rules and utilities based on current needs rather than trying to address an uncertain future, the mess could get bigger. The general CDM strategy of excluding anything without an immediate, important use case doesn’t quite carry over here: 1) we are able to get away with that strategy because the CDM is actually well-designed for flexibility so excluding wish list items now doesn’t severely complicate adding them later; 2) unnecessary CDM complexity would burden the whole community, while an overambitious dependency mechanism would only be a hassle for programmers; and 3) any version dependency mechanism we put place based only on immediate needs is likely become an obsolete legacy headache very quickly since we are already working on features that will place greater demands on it.

1 Like
t