DataQualityDashboard: lexical error: invalid char in JSON text

[crhightower (Caleb Hightower)]

Hi all,

I am not sure if this is the right place, but I am trying to run the example code provided for the DQD package on some synthetic CMS data. The executeDqChecks seems to run fine. There are a few errors like column cdmtable.admitting_source_concept_id does not exist but the check finished and output a JSON. However, when I try to open the JSON with viewDqDashboard, I get the error lexical error: invalid char in JSON text and the shiny opens a default file with a gray filter. Is there an issue with the JSON that was created? It seems okay at first glance. Has anyone else had encountered an issue like this? Thanks!

[katy-sadowski]

Hello! Generally we see this issue when you’ve passed in a nonexistent/invalid filepath to viewDqDashboard. Are you sure that your output file lives in “output/results.json”?

See JSON file not generated after running DataQualityDashboard::executeDqChecks - #5 by katy-sadowski

[crhightower (Caleb Hightower)]

Hi Katy, thanks for the response. Yes, I am sure the output file lives in “output/results.json”. I can verify this by running jsonlite::fromJSON(jsonFilePath) which returns an R object with the JSON contents. However, when I run jsonlite::validate(jsonFilePath) the validation fails.

[1] FALSE attr(,"offset") [1] 1 attr(,"err") [1] "lexical error: invalid char in json text.\n output/results.json\n (right here) ------^\n"

So maybe there was an issue with the creation of the JSON or jsonlite? I just can’t figure out what.

[katy-sadowski]

It looks like fromJSON will accept a filepath as input, but validate only accepts json strings. So when you try to validate the filepath it’s treating it like a json string (hence the error).

That said, it’s odd that fromJSON works but viewDqDashboard doesn’t because the Shiny app uses fromJSON to read in the json from the file… Are you sure you’re in the right directory when you’re running viewDqDashboard?

Also - what version of DataQualityDashboard are you using?

[crhightower (Caleb Hightower)]

Ah, good point. My mistake. That’s really interesting and might give me some insight as to why it’s failing.

I think I’m in the right directory. I’ve even changed the directory over to “output” and even copied the file over to my base directory, and both ended in the same error.

I am using version 2.4.0.

[katy-sadowski]

Odd. Could you please share the full series of commands you’re running that leads up to the error? (including executeDqChecks and the line where you set jsonFilePath)

[crhightower (Caleb Hightower)]

I figured it out! You were right about it being a problem with the directory/path. The function obviously needed the full path and doesn’t rely on the current working directory. So instead of “output/results.json”, I needed to supply “home/…/output/results.json”. This is made a little extra complicated because I’m working in a virtually hosted R session. Thanks for your help!!

[katy-sadowski]

No problem! This issue has come up multiple times so I will look into how we can clarify our docs and perhaps even how these functions/parameters work to prevent the hassle for others