I’m new to the group, am trying to get up to speed with your efforts, though I think it’s great this group is working on adding radiology to OMOP. I’d like to understand the specific use cases you have in mind for querying image data in connection with this effort. As to standardizing radiology procedures, the RadLex Playbook/LOINC effort has done that by creating a post-coordinated terminology built using RadLex atoms. See https://www.rsna.org/RadLex_Playbook.aspx.
I also want to point out that all of the key information related to images is not only in the DICOM header; the Annotation and Image Markup project (AIM) created a data model for image data, and this was recently incorporated into DICOM, though this has not yet been incorporated into commercial PACS workstations.
Thank you for your interest, @dlrubin
I need to learn more about RadLex and other efforts for standardization in radiology. It is really helpful that you told me the RadLex playbook website and AIM project. I’ll dig into RadLex and the AIM project.
I have two use cases in my mind.
-
Building artificial intelligence model by using brain CT in patients with suspicious stroke.
Recently, the paradigm for ischemic stroke is shifting from intravenous thrombolysis with tPA into endovascular procedure (mechanical thrombectomy. The ASPECTS (Alberta Stroke Program Early CT Score) is a quantitate score that measures the extent of early ischemic changes. The ASPECTS relies on subtle CT findings and thus requires an experienced radiologist. In Korea, many hospitals usually don’t have experienced radiologist or neurologist in the night. So I want to make help clinicians to calculate ASPECT score by deep learning.
For this purpose, I need to extract only ‘non-contrast’ phase from whole brain CT. By using R-CDM, the same phase of brain CT can be pulled from target cohort (patients with suspicious symptoms for stroke).
And then deep learning model can be developed by using these images (For standardizing the image size, I need three-dimensional resolution, which will be in R-CDM, too). We’ve converted brain CT of 300 patients with suspicious symptom of stroke in our hospital and started to build the model now. -
Building predictive model by using clinical and radiology data within PatientLevelPrediction ecosystem.
Recently, I’m trying to develop the model predicting in-hospital mortality by using the deep learning module of PLP packages. On top of the clinical variables, I want add features from ‘chest x-ray’. Like other features generated by FeatureExtraction package, the some features from x-ray can be converted into features by using unsupervised deep learning. And then, features from image can be used for building predictive model with clinical features in PLP ecosystem.
Those two are my future plan. I think I can show the pilot result of first use-case.
I really appreciate your helpful comment, again.
Thanks for clarifying. Those are useful use cases. I look forward to learning more about the R-CDM model, is it formalized yet or is there a working draft somewhere? I’d suggest you review the AIM model (which is in XML schema; AIM instances are in XML)
https://wiki.nci.nih.gov/display/AIM/Annotation+and+Image+Markup+-+AIM#AnnotationandImageMarkup-AIM-AIMModel
AIM captures image ROI info, quantitative image features, and qualitative info about the image in an XML format and AIM XML can be converted to DICOM SR, http://dicom.nema.org/medical/dicom/current/output/html/part21.html#chapter_A
However, until imaging equipment vendors support AIM or the DICOM SR TID 1550 object, this won’t help your use cases, and if R-CDM development is based only on what you can glean from the DICOM header, then there are limits to the info you can capture. I think it will be helpful to access and link info that can be extracted (using NLP) from radiology reports associated with the imaging study to fulfill use case such as yours involving radiology. I have some comments about your two specific use cases:
-
In order to train a model to predict ASPECTS score, you need a representation of the score (and possibly the imaging features described in the radiology report that led to selection of that score). Is this modeled in R-CDM? Also, in order to recognize DICOM images that are from the “non-contrast phase” series of a brain CT, I presume R-CDM has a model for the value of the SeriesDescription DICOM element (0008,103E) which I believe is where you would determine this? The challenge is that this is a free text field and I’m not aware of any standard terminology for naming the imaging series (the Radlex Playbook/LOINC project so far has only tackled naming imaging study names). I’d be interested to learn more detail about what specifically you did when you say you converted 300 brain CT studies. Finally, in the USA, MRI is generally performed in acute stroke and it might be useful in your deep learning model to work with MRI as well as CT (or separate models for each) if you’re looking for other sites to potentially contribute here.
-
In terms of adding features from chest x-ray images, are you creating something in R-CDM to represent these features? There’s work going on to standardize description of image features (Image biomarker standardisation initiative), but image features learned from DL models would be challenging to describe/represent in an explicit paradigm. I’m not sure I see how one would describe image features learned from a DL model would be described in R-CDM so they can be linked to clinical features in the PLP ecosystem. Perhaps I’m not understanding the details of this ecosystem.
Sorry for late reply, @dlrubin
It’s still in its infancy, but I released the working draft of sample data of R-CDM here
( Oncology radiology imaging integration into CDM ).
[Annotation / Report in Radiology]
I know annotation data and ROI are very important, and I need to learn from AIM project. But as you said, I do not have the data containing information about this, now. I suggested the extension model of ‘NOTE’ table for storing radiology report ( https://github.com/OHDSI/CommonDataModel/issues/172 ) , and I hope this proposal to be adopted in OMOP CDM v6.0.
[Question1: imaging for stroke and standardization of DICOM headers]
We leverage information fro DICOM headers, but we cannot totally dependent on it. These informations are not standardized as you said. The solution for identifying the specific phases in radiology images should be developed one by one in each institution for each radiology images. We found out which rule we should use to identify phases in brain CT by browsing DICOM headers.
Again, R-CDM project is in its infancy.
I am really pleased if you can join this research. I can make CNN model for MRI (actually it is much easier to use brain MRI rather than CT, because there is much more open-source code analyzing brain MRI). But I think it would be much more difficult to standardize MRI procedure (The phases are so complicated in brain MRI. But we can start with only T1, T2 and FLAIR for the pilot study.)
[Question 2]
PLP ecosystem requires features in the format of n-dimensional vectors. Actually, I need to figure out how to make feature vectors… It can be done through supervised learning or unsupervised learning. But I do not have concrete idea for this for now.
[R-CDM] Thanks for the link to your draft of this, I see two files but I don’t understand their contents. Is there any documentation describing this?
[Annotation/Report in Radiology]
Since AIM already has a data model for describing the key semantic information in radiology images, I’d encourage that R-CDM describe similar information entities that AIM does, so that interoperability between AIM instances and R-CDM/OMOP will be facilitated. In particular, including AnatomicEntity, AnatomicEntityCharacteristic, ImagingObservation, GeometricShape, ImagingObservationCharacteristic, and Calculation entities would be helpful. Then images that have been annotated using AIM (and soon the AIM/DICOM-SR) compatible tools can interoperate.
[Imaging for stroke and standardization of DICOM headers]
If you are primarily pursuing the stroke use case, it would be good to focus on brain MRI for now. As to standardizing MRI procedure, I agree it’s fine to start with T1, T2 and FLAIR for the pilot study, and put those into the R-CDM model. Then each site would need to map their ad-hoc series names to the equivalent entities in R-CDM.
[Question 2]
For this particular use case, I think you’ll get a lot of signal for prediction by using the radiology report, since that has already been synthesized/summarized by a human. Looking directly at the image data and doing feature extraction from that could be helpful in different use cases, like generating the report (very hard problem), so you might make the problem easier on yourself by using the report text and integrating that with other clinical EMR data as a first pass at this problem.
Sorry for late reply @dlrubin
[Annotation/Report in Radiology]
Yes, we’ll review the AIM and try to make the model compatible with this. Thank you for helpful advice, again.
[Imaging for stroke]
Our job for brain CT is almost done,now. We’ll show the actual use case of this in OHDSI symposium.
Definitely one of the next steps would be standardization of brain MR. But currently, I’m planning to integrate chest x-ray first.
[Question 2]
Thank you for the precious idea. Yes, generating the report is actually one of my dream.
First of all, I’ll predict prognosis of stroke patients with their initial brain CT or predict overall mortality in hospitalized patients with their chest x-ray
Thank to @NEONKID 's effort, we released the ETL module converting DICOM file into Radiology CDM, which can extract required meta-data from DICOM for Radiology CDM.
Thanks for the reply @SCYou. Is there any documentation on R-CDM or how can I get up to speed to understand this model as I think about possibly building on it or pursuing related work?
@dlrubin Thank you for your interest, again.
Sorry that I didn’t prepare the detailed documentation for Radiology-CDM. You can see the abstract about Radiology-CDM for OHDSI symposium 2018, here.
And I sent my e-mail address to you through PM.
how we can standardize the radiology procedures
It depends on source data. E.g., if from billing data - we don’t have a good strategy what is the best target terminology. CPT itself is standard concept.
If we have data from more advanced sources (e.g., EHR system) - we may have to think about how these will co-exist with the billing input.
e.g.,
CT or MRI of the brain performed within 24 hours of arrival to the hospital (STR) concept_id 2106371
concept code 3111F
http://athena.ohdsi.org/search-terms/terms/2106371
It implies that both CT and MRI were done (given current mapping)
another is
Radiologic examination, chest, two views, frontal and lateral
2211361
71020
http://athena.ohdsi.org/search-terms/terms/2211361
Mapping here is nicer and goes to plain chest x-ray
I like the connection to device_exposure in your model. I am not sure if the vocab-driven ETL would touch the device table at all.
paste for others:
you model has no terminology binding.
The key part is ‘device_concept_id’
Can you please reply with what concepts you use for that.
In dplyr it would be:
radiology_occurrence %>%dplyr::count(device_concept_id)@Vojtech_Huser
Thank you for the comment. Yes, the adding device_id in R-CDM is definitely one of my goals. I hesitated to add it because ‘device_exposure’ is actually ‘exposure’ table for device, not device itself. So I concerned whether we should add sort of ‘device’ table for ‘device’ itself. Device information is essential for PGHD information (e.g., which wearable device) genetic extension model (e.g., which sequencing device),and radiology information.
Linking with the device_exposure is actually good solution. For brain CT, I want to use ‘45762714’(Full-body CT system) for device_concept_id (http://athena.ohdsi.org/search-terms/terms/45762714), though I’m not sure this is the best term…
@dlrubin, Here’s my model how to use unsupervised deep learning for medical image combined with structured clinical data to predict clinical outcome.
I’m trying to convert medical image into small-size vector representing the the original image, and combine it with other clinical features.
I uploaded the code to [github][1] for building encoder for medical image, and make feature vector from medical image based on radiology CDM. This is totally compatible with other OHDSI tool ecosystem, such as FeatureExtraction package and PatientLevelPrediction package.
I know i need more explanation to show entire process. I can show it in the poster at the OHDSI symposium.
[1]: StudyProtocolSandbox/RadiologyCovariateExtraction.R at master · OHDSI/StudyProtocolSandbox · GitHub
The details of model architecture and the package for extracting and converting metadata from DICOM to R-CDM file are available at: https://github.com/NEONKID/RCDM-ETL
This would be useful for work we are doing and have planned.
What a wonderful amount of work you’ve done! I would love to find out more about what you are doing with it. I couldn’t get to your poster at the symposium and the posters aren’t available on the OHDSI website yet.
I need to bring this up again. @christian_reich @dymshyts
Radiology imaging is classified into Procedure in OMOP-CDM, to my understanding.
I argue that the domain of LOINC concepts imported from RadLex should be classified into ‘Procedure’ rather than ‘Measurement’. You can find the list of LOINC codes in LOINC/RSNA code book here.
I think this (adoption of LOINC/RadLex as standard vocabulary for radiologic imaging procedure) may facilitate further standardization of radiology images in OMOP-CDM and collaboration between clinical data scientists and radiologists in OHDSI.
I agree with @SCYou.
@Polina_Talapova @zhuk @Alexdavv
are you agree with that as well?
Can you take a look at the technical aspect of how to implement this change?
@Dymshyts I agree with that. From a technical perspective, we are able to this. So, if there is no objection, let’s do it.
Just reviewed the list.
My understanding was that the most of these concepts should be used with the result what isn’t possible in the Procedure Domain.
But now I see most of them are complex procedures and cannot be described by the categorical result.