DQD- Exclude select columns from verification and validation

Implementers
1

Hello Everyone,

I would like to turn off/exclude checks based on plausibleValueLow and plausibleValueHigh for few columns.

For ex: I would like to exclude scenario a) from being checked but apply the check for scenario b)

scenario a - Our dates are shifted into the future. So this is expected and not an error. So, would like to turn off this rule check. But couldn’t find any info on documentation on how can I trun off a column being checked? I know we can turn off the check itself or exclude the table itself as a whole. Can’t columns be excluded from check?

The number and percent of records with a value in the OBSERVATION_PERIOD_START_DATE field of the OBSERVATION_PERIOD table greater than DATEADD(DD,1, GETDATE()).

scenario b - Our lab values should be checked for threshold limit (which uses similar DQ Rule type as Scenario 1)

For the combination of CONCEPT_ID 3018910 (ALKALINE PHOSPHATASE.BONE [MASS/VOLUME] IN SERUM OR PLASMA) and UNIT_CONCEPT_ID 8748 (MICROGRAM PER LITER), the number and percent of records that have a value less than 10.000.(Threshold=5%)

While I can exclude tables but how can I exclude that specific check for _date columns only?

If i have to exclude for all columns, I understand it’s straight forward but how to retain the check for select columns and not apply for few columns (in all tables)

2

Turn off the check for events outside of the Observation_period_start_date. Look at the csv file ending in ‘_Check_Descriptions.csv’. In the column ‘checkName’ there is an entry ‘inObservationPeriod’ and for that row there is the entry ‘inObservationPeriod==‘Yes’’ in the column labeled ‘evaluationFilter’. Now you have two options:

  1. Disable the test in ‘_Check_Descriptions.csv’ by changing the evaluationFilter to something like inObservationPeriod==‘XXX’, which will ignore the test for all files.

  2. Open the csv file ‘_Field_Level.csv’ and find the column ‘inObservationPeriod’ then blank out the values ‘Yes’ for the tables that you do not want to run the check on.

The ‘_Check_Descriptions.csv’ defines the tests, the SQL file used to execute the test, the file to look at to determine if the test should be run, and the column heading and value that is set to activate the test.

1 Like