OHDSI Home | Forums | Wiki | Github

Keeping track of OHDSI network studies

(Martijn Schuemie) #1

There are a lot of OHDSI studies going on, but keeping track of them is not that easy. The OHDSI website has this page, which unfortunately is a bit outdated (I think literally all of the listed research projects are no longer active). There’s this Wiki page which is also lags severely behind.

As discussed here, ever since we moved to the ohdsi-studies GitHub for our study protocols and code, we have also enforced a bit of standardization of the first part of the study repo README file. This has allowed scraping these files to create a near-real-time view of OHDSI studies, like in this Shiny app I created.

My question: do we want to start using this Shiny app as our main view of ongoing (and past) OHDSI studies? We could replace the tab on ohdsi.org with this page.

Tagging @CraigSachson, @MauraBeaton, @krfeeney

(George Hripcsak) #2

This is spectacular. Can we add a column with the GitHub subdirectory, or even the whole URL (e.g., https://github.com/ohdsi-studies/Covid19PredictionStudies for “Development and validation OHDSI network studies for the covid19 prediction topic”)? To be honest, just searching on the already-supplied title+“GitHub” does get you there pretty quickly.

(Emma Vos) #3

Hi Martijn,

At The Hyve we are actually currently in the process of creating a website for the COVID-19 study-a-thon results, in light of the EHDEN project: https://covid19.ohdsi.app/.
The website is still in progress, but will be populated with more studies, databases and authors soon. The goals of this website are:

  • to research and propose an interoperable standard for metadata for OHDSI studies
  • to pilot static website generation as a framework to quickly stand up a website that contains this data readable for both humans and machines
  • to use the COVID-19 studyathon as a test case for the feasibility of this approach

The idea behind this website is to publish OHDSI studies in a FAIR (Findable, Accessible, Interoperable and Reusable) way. On the community call on the 5th of May we gave a presentation on this: OHDSI Community Call 5May2020. You can find the slide deck here: https://www.ohdsi.org/wp-content/uploads/2020/05/FAIRification-of-OHDSI-assets-OHDSI-Community-Call-May-5th.pdf

The data is now populated manually, but we do intent to supply other information automatically. The standardization of the GitHub repositories that is currently in place, as well as the GitHub scraper scripts are of course very useful for this.
We are still at a development stages, and are looking how to best integrate this with existing tooling. Would you be available for a call somewhere in the coming weeks?

(Martijn Schuemie) #4

@hripcsa: probably not clear from the current UI, but if you click on one of the studies, you get additional details, including the URL to the GitHub repo. Is that what you mean?

(George Hripcsak) #5

Oh my goodness. So sorry. My browser window had cut it off. Yes of course.

(Martijn Schuemie) #6

Hi @EmmaVos! I’m sorry, I missed your talk. It seems like you though about this problem a lot more than me. I love the machine-readable interface!

Yes, happy to talk. I’ll send you an e-mail.

(Patrick Ryan) #7

Thanks @schuemie, this is great to see, and I fully support this replacing our legacy ‘studies’ list on the OHDSI website, since we never established an appropriate process for keeping it up-to-date. I like the idea of a fully automated approach. Well done.

(Andrew Williams) #8

This tool built by Dave Eichmann at Iowa for use in CD2H might be helpful for automating the process: https://github.com/eichmann/gitforager

(Filipa Duarte-Ramos) #9

Thank you!
This is a very useful resource and I think it would be a great idea.
How can we identify those projects that are allowing participants entry?

Best regards

(Craig Sachson) #10

This makes a lot of sense @schuemie, and will make it much easier to track the breadth of ongoing research in the community. Thanks for sharing. I’ll change the link on OHDSI.org.


Great question Filipa! I was wondering about that too.

(Vojtech Huser) #12

@schuemie, can you please point me to where are instructions on how a study lead can add a study to the https://github.com/ohdsi-studies

Is there a mechanism to request membership in that github repo/organization. It seems it only works one way (to be invited).

Do I first publish my study and have it evaluated by some gatekeeper?

(Martijn Schuemie) #13

@FDuarteRamos, I agree this is a good question. The answer is that currently you can’t easily identify those. In my experience, every study goes about this differently, with some just starting with a fixed group of collaborators and just moving forward with that group, while others may welcome data partners at some point, or others still may need collaborators with very specific skills, etc.

I didn’t feel a simple ‘help wanted’ flag would convey this complexity, so I deliberately didn’t add it. I think it would be good to have a better process in place for how collaborations are formed around a study, but that looks like a very large separate endeavor, and I therefore left that out of the scope of this app.

At least the app shows you what studies are going on, and who leads them. You can always contact the study leads if you’re interested in joining.

(Martijn Schuemie) #14

Hi @Vojtech_Huser! Another great question.

The current study on-boarding process is very informal: contact one of the ohdsi-studies admins, and they will set you up.with your repo. Admins are: @krfeeney, @SCYou, @jreps, @schuemie, @Patrick_Ryan, and @msuchard.

@lee_evans at one point proposed to create a Google Forms (I think he actually did create it) where people could request SFTP accounts for their study, and we briefly discussed this form could be extended to request an ohdsi-studies repo as well. But then COVID-19 happened, and we forgot about it. Perhaps we could pick that up again?

I don’t think we need a strong gatekeeper function. At a high level, I think the requirement should be that a study is aligned with the OHDSI mission, vision, and values. To make the Shiny app work, additional requirements are:

  1. Adhere to the README file template.

  2. Have one repo per study. I did some policing here, because people originally asked for one repo per R package, and many studies actually have more than one package (e.g. a cohort diagnostics, characterization, and estimation package). Those can just be sub-folders in the study repo. As a rule-of-thumb for what is the unit of a study, I ask whether results would be disseminated in the same paper. If so, it probably is one study and should be in one repo.