Is there a way for Usagi to map source codes to the standard codes by maintaining the data in database tables rather than csv files which might in turn help maintaining and managing the data when new codes are added/updated etc.
I’m afraid not. Usagi needs to build a full-text search index for ‘fuzzy’ string matching, which is not supported by third-party databases.
I am aware of the problem posed by changes in the vocabulary (and the source coding system). For now, there’s the ‘Apply previous mapping’ option in Usagi that tries to apply a mapping previously created in Usagi if the source codes and target concepts both exist in the new context.
We’re just getting starting with Usagi and I thought of the same question as @nmandadi.
I understand @schuemie’s point about Usagi needing its own underlying BerkeleyDB and Apache Lucene files.
But @nmandadi’s question was about the .csv files. Manual usage of Usagi still requires a human to click the “Open” or “Import codes…” menu option to start the process, and to click the “Save” or “Export” menu option when done.
Rather than muck about with importing and exporting .csv files, wouldn’t it be great to have menu options for “Import from database…” and then “Export to database…”? In other words, read and write the code records to and from an ODBC connection.
I agree these database tables should not be the OMOP tables, but rather other tables that exist in an ETL “staging” schema somewhere…
I can understand why they want to use .csv files. They have absolute control over reading and writing the data. Databases can have so many different datatypes that have to be delimited in different ways.
On the other hand, source_to_concept_map should be standard across OMOP instances. It would be nice if USAGI could pull existing mappings from source_to_concept_map and write mappings directly to it.
(edited for readability)(geesh, who wrote this? -R)
Hi @quinnt, I like your suggestion. This would add a lot more dependencies to Usagi (e.g. all the different jdbc’s to connect to the different database sytems), but I can see that it adds value. Could you open an issue describing the enhancement on Github so we can put it on the roadmap? I would especially like to see a toy example of how this should work. e.g. what data is read in (and should Usagi sum frequencies itself)? In what format should it be exported?
@MaximMoinat, Thanks so much for your willingness to consider such functionality.
My OMOP project team is currently thinking about how this might work. As you know, such “interoperability” data exchange between 2 systems – Usagi and a CDM database – is not a trivial design problem, for the reason that @roger.carlson mentioned and other reasons.
I’m happy to create a GitHub issue when I have a well-formulated proposal. More to come!