I wanted to share a paper our group just published in the Journal of Biomedical Informatics: Redirecting
The practical problem was a familiar one — mapping local radiology procedure names to the LOINC/RSNA Radiology Playbook. Anyone who has done ETL for imaging data knows how messy these names are: abbreviations, mixed languages (in our case Korean + English), missing modifiers, institution-specific shorthand. Usagi and similar lexical tools work fine when local terms are fully spelled out, but that’s rarely the case in practice.
What we tried is a framework we call ARKE (Automated Radiology Knowledge Encoding). Instead of treating mapping as text-to-text matching, we decompose each local name into Playbook attributes (modality, region, laterality, contrast, view, etc.) using an LLM, then retrieve candidate codes with a plain rule-based weighted similarity over a RadLex knowledge graph — no LLM involved at this step. The LLM only comes back at the end, to pick one code among the top 10 ontology-validated candidates. So the model never gets a chance to invent a code that doesn’t exist; the ontology defines the search space, and the LLM resolves the subtle distinctions (view combinations, contrast routes, MRI vs MRA and so on) that similarity scores can’t.
We evaluated this on 3,326 unique procedure names from two tertiary hospitals in Korea, against radiologist-curated references. Final recall@1 was 0.70 on the internal test set and 0.81 on the external one, compared to about 0.48-0.50 for direct code-name embedding — a >20 point gap (p<.001).
My honest motivation for posting here, beyond the radiology use case: I keep seeing arguments that LLMs make controlled vocabularies obsolete.Our experience was the opposite. The ontology was what made the LLM usable — it constrained the reasoning enough to make outputs reproducible and auditable. I suspect the same expand-then-map pattern would work for other compositional domains in our vocabulary ecosystem, and it fits naturally into ETL for the imaging CDM extension (image_occurrence).
Codes really are tokens, and folks will claim that LLMs can use tokens from dirty texts just as much as standardized vocabulary codes.
Folks will continue spreading the suspicion that us insisting on our codes is merely because we don’t want to become obsolete.
At the end of the day it will depend on who will build the models that beat the large foundational models in understanding of dieases, treatments and outcomes.
I completely agree that it will ultimately come down to which ecosystem performs best in practice. But as we look ahead to that future, a couple of thoughts stand out from our experience:
1. The Illusion of “Generic” Ontology AI in Medicine
Many tech companies and AI experts talk about applying “ontology AI” to healthcare, drawing parallels to how platforms like Palantir handle ontologies in other industries. But medical ontology is a completely different beast. A massive part of medical education is simply learning the exact definitions and boundaries of clinical terms. I struggle to think of any other field where such a vast, intricate vocabulary exists with globally unified meanings. While there is no shortage of ontology AI experts out there, very few truly understand the clinical nuances of medical ontologies.
2. Medallion Architecture for medical AI"
We also frequently see data engineers pushing the Medallion Architecture (Bronze → Silver → Gold) as the ultimate data strategy. From where I stand, we don’t need to reinvent the wheel—the OMOP CDM is already the Gold Layer. It provides the curated, semantic truth that foundational models can’t just guess their way into.
If an AI model is built on “dirty tokens” without this structured foundational layer, it might look impressive on the surface, but it lacks the reproducible, auditable grounding required for high-stakes clinical research. My gut feeling is that as people start moving legacy DWs into Silver or Gold layers to build agentic AI, they will finally begin to realize the true, indispensable value of the OMOP CDM and standard vocabularies. Ultimately, the models that “beat” generic foundational giants in understanding diseases and outcomes will likely be the ones that embrace, rather than dismiss, this deep medical infrastructure.
To your point about this ongoing debate, our group at Yonsei recently launched a global community survey led by Jiwon Um (PhD candidate) to systematically capture where the OHDSI community stands: “Does data standardization still matter in the era of foundation models, or is it becoming a legacy cost?”
Please visit this thread and participate in the survey