OHDSI Home | Forums | Wiki | Github

Data Quality Dashboard tutorial - slides


(Ajit Londhe) #1

For those attending the Data Quality tutorial, here’s my slides on the DataQualityDashboard tool:


(Conor McGrath) #2

Hello Ajit, when is the tutorial on? today? Will the presentation be recorded? Regards Conor.


(Ajit Londhe) #3

It is right now :slight_smile:

Yes, it will be recorded and made available


(Taha Abdul-Basser) #4

Great work @Ajit_Londhe!


(Don Torok) #5

Great tool, up and working already. Question about the test: For the combination of CONCEPT_ID 3024128 (Bilirubin.total [Mass/volume] in Serum or Plasma) and UNIT_CONCEPT_ID 8840 (milligram per deciliter), the number and percent of records that have a value less than 1.00e+00. (Threshold=1%). However, looking at the description for LOINC 1975-2, it gives the range as mg/dL[0.3,1.0] which I take to say between 0.3 and 1.0 milligram per deciliter. Given that range, seems the DQ test for values less than 1.0 is incorrect.


(Evan Sholle) #6

@Ajit_Londhe

Great tutorial. As we discussed, I’m still getting an error with the following traceback:

 > DataQualityDashboard::executeDqChecks(connectionDetails = connectionDetails, 
+                                       cdmDatabaseSchema = cdmDatabaseSchema, 
+                                       resultsDatabaseSchema = resultsDatabaseSchema,
+                                       cdmSourceName = cdmSourceName, 
+                                       numThreads = numThreads,
+                                       sqlOnly = sqlOnly, 
+                                       outputFolder = outputFolder, 
+                                       verboseMode = verboseMode,
+                                       #writeToTable = writeToTable,
+                                       checkLevels = checkLevels,
+                                       checkNames = checkNames)
Connecting using SQL Server driver
Processing check description: plausibleGender
Connecting using SQL Server driver
Error in parse(text = thresholdFilter) : 
  object 'thresholdFilter' not found
> traceback()
5: parse(text = thresholdFilter)
4: eval(parse(text = thresholdFilter))
3: .evaluateThresholds(checkResults = checkResults, tableChecks = tableChecks, 
       fieldChecks = fieldChecks, conceptChecks = conceptChecks)
2: .summarizeResults(connectionDetails = connectionDetails, cdmDatabaseSchema = cdmDatabaseSchema, 
       checkResults = checkResults, cdmSourceName = cdmSourceName, 
       outputFolder = outputFolder, startTime = startTime, tableChecks = tableChecks, 
       fieldChecks = fieldChecks, conceptChecks = conceptChecks)
1: DataQualityDashboard::executeDqChecks(connectionDetails = connectionDetails, 
       cdmDatabaseSchema = cdmDatabaseSchema, resultsDatabaseSchema = resultsDatabaseSchema, 
       cdmSourceName = cdmSourceName, numThreads = numThreads, sqlOnly = sqlOnly, 
       outputFolder = outputFolder, verboseMode = verboseMode, checkLevels = checkLevels, 
       checkNames = checkNames)

It seems like the issue is in the .evaluteThresholds() function, which is setting thresholdField at each level. I can’t figure out what’s going on, though. Anyone have any thoughts?


(Ajit Londhe) #7

Hi @esholle – can you re-install and try again? I pushed a fix for a faulty if statement that was causing the error when running only plausibleGender


(Evan Sholle) #8

@Ajit_Londhe it worked!! You are the best. A million thanks.


(Michael Gurley) #9

This is probably something stupid that I am missing but I am having problems installing the DataQualityDashboard on Windows. When I run the following command in Rstudio:

devtools::install_github(“OHDSI/DataQualityDashboard”)

I get the following error:

Error: Failed to install ‘DataQualityDashboard’ from GitHub:
(converted from warning) installation of package ‘C:/Users/Nubic/AppData/Local/Temp/RtmpMtPHxs/file58cbe650c5/DataQualityDashboard_0.0.1.tar.gz’ had non-zero exit status

I am getting this on Windows 7 Enterprise and my colleague is also getting the same error on the latest Windows.


(Ajit Londhe) #10

Can you post more of the console log that appears after running the install command?


(Dave Barman) #11

Hi Ajit, thanks for posting the slides. We are testing the DQDashboard on a larger CDM and I have a question if there is a simple way to check that the install is OK and the connection works? Maybe a lower resource test that would be best to start with?


(Andrew Williams) #12

@Ajit_Londhe I’m having similar troubles to @mgurley. Since he’s having trouble I feel less embarassed about admitting mine. :disappointed_relieved: Maybe it’s a default download option I need to change? Others seeing the same error message for other packages have concluded as much but I haven’t had luck with that yet. Here’s what I get:

devtools::install_github(“OHDSI/DataQualityDashboard”)
Downloading GitHub repo OHDSI/DataQualityDashboard@master
/Rtools/bin/tar: Child returned status 127
/Rtools/bin/tar: Error is not recoverable: exiting now
External tar failed with --force-local, trying without
/Rtools/bin/tar: Child returned status 127
/Rtools/bin/tar: Error is not recoverable: exiting now
External tar failed with --force-local, trying without
Error: Failed to install ‘DataQualityDashboard’ from GitHub:
Does not appear to be an R package (no DESCRIPTION)
In addition: Warning messages:
1: In utils::untar(tarfile, …) :
‘tar.exe -zxf “C:\Users\AWILLI~1\AppData\Local\Temp\RtmpEdU9k7\file44e816a67f0e.tar.gz” -C “C:/Users/AWILLI~1/AppData/Local/Temp/RtmpEdU9k7/remotes44e8467e7433”’ returned error code 2
2: In system(cmd, intern = TRUE) :


(Evan Sholle) #13

@Dave.Barman Try setting checkNames to just PlausibleGender. That’s what I did to test out the whole soup-to-nuts process. It ran in 23 seconds on a SQL Server-hosted instance with ~3 million PERSONs.


(Michael Gurley) #14

For posterity, @Ajit_Londhe answered me offline in email with the following:

"Looks like there’s an issue with rJava. Most likely it is due to having both 32 and 64 bit versions of R.

I would re-install R and Java using the steps outlined here : [https://ohdsi.github.io/TheBookOfOhdsi/OhdsiAnalyticsTools.html#installR]"

This fixed the issue for me.


(Ajit Londhe) #15

And let me add this is an immediate benefit of the Book of OHDSI, so thank you @schuemie and @Frank for a well written set of instructions for R installation!


(Dave Barman) #16

Oh this could be useful for me immediately, but I can’t find the instruction on the online version. Which chapter and heading are the R installation instructions on?


(Evan Sholle) #17

8.4.5


t