After executing the cpt4.jar utility to update CONCEPTS, I observe the following output:
[INFO ] 2022-12-07 09:53:27.411 [main] ConceptService - Writing updated data to CONCEPT.csv
Updated CPT4 records: 16632/16632
[INFO ] 2022-12-07 10:27:15.867 [main] ConceptService - Not processed cpt4 concepts: 12487.
See logs/not-processed-concepts-12-07-2022-09-53-25.out, file.
You can find more information about errors in the logs/logfile.log file
I have run the cpt4.jar utility numerous times, including on different machines, with the same result. A colleague also ran the utility this morning, with the same result.
Interestingly when my colleague ran the process he received the following error after the process had completed. On my runs I didn’t see any exceptions (including in the .log, .out files):
Exception in thread “main” org.odhsi.utils.cpt.Cpt4Exception: Unexpected code Response{protocol=h2, code=401, message=, url=https://utslogin.nlm.nih.gov//cas/v1/tickets/TGT-93158-1emw14dvlbf4e4lf-cas}
at org.odhsi.utils.cpt.restclient.RestTicketClient.checkSuccessful(RestTicketClient.java:76)
at org.odhsi.utils.cpt.restclient.RestTicketClient.logout(RestTicketClient.java:113)
at org.odhsi.utils.cpt.restclient.RestTicketClient.close(RestTicketClient.java:125)
at org.odhsi.utils.cpt.Application.main(Application.java:49)
I’m having the same problem. For me, there are always significant # not processed, even when run cpt.sh multiple times in the same session (or starting from scratch). The first time, it was 12488 (and reduced that to 3625 not processed when ran it a second time. Then, when I started from scratch, it was 12450 not processed even when ran multiple times.
However, when I review the logs/not-processed-concepts.out file, they all appear to be valid CPT codes, and I get get accurate labels when searching for them via athena.ohdsi.org
I’m having the same problem. I tried to walk around this issue by using CPT4 concept that I previously downloaded in Sep 2022 but it failed.
SQL Error [23503]: ERROR: insert or update on table “concept_relationship” violates foreign key constraint “fpk_concept_relationship_c_1”
Detail: Key (concept_id_1)=(777313) is not present in table “concept”.
Looking at the Athena last update for CPT4, it shows “01-May-2022”, but Athena has Valid start “12-Oct-2022”. so we have new CPT4 version in Athena? @mik@Dymshyts Could you fix the issue like you did last year? Thanks.
Our engineers have run into the same problem. I can’t tell from the threads if there’s a fix available or whether we should use old tables instead. @Christian_Reich?
Hi @david_seligman , do you reference to " violates foreign key "? If that is the case, you are trying to mix vocabulary versions in your tables. When loading vocabularies clear your vocabulary tables first and load them fresh…
If you are experiencing issues with operating the cpt4.jar, we have just noticed that we need to finetune that again for the latest release. Bear with us a little.
@mik Thank you. This is the error we got “2022-12-06 00:18:10.517 [main] ConceptService - Not processed cpt4 concepts: 12714. See logs/not-processed-concepts-12-05-2022-22-13-43.out, file.” We can post the log file if it would be helpful.
okay, @david_seligman , sounds like you need the fixed cpt4.jar. We are currently testing it and I will post here, once it is available through ATHENA. You will have to re-initiate the vocabulary download (go to the download page, select your vocabs and request the download). My assumption is that it should be available tomorrow (stay tuned).
Hi @solmaz.eradat ,
I checked my csv files and found the concept id 45889325 and 45890303 in both, the CONCEPT_CPT4 and the CONCEPT.csv file after reconstitution. As tiresome it is, can you re-run the download and reconstitution exercise if you cannot find those id’s in your csv files? If you do find them, there must be something happening during the import.
cheers - mik
I am facing the same issue with the latest cpt4.jar that comes with the vocabulary download where a large no of concept update is failing. Tried looking for the source code but couldn’t find it. By any chance, is the cpt4.jar source code available on Ohdsi git?
