Hi All,
I thought it would be good to put this on the forum for awareness.
For those who are interested find here the link to the evaluation of Achilles as performed in the European Medical Information Framework (EMIF) project. In total 26 users with different backgrounds filled in a questionnaire when having access to an instance of Achilles.
Questions were asked about each of the reports but also specific questions for the Data Custodians related to sharing Achilles Reports.
In general the tool was wel accepted but also many suggestions for improvements were made by the users.
Most important is the clear need for better documentation that contains explanations of each of the generated graphs. This also definitely applies to Achilles Heel. In a recent face-to-face meeting with data custodians it was also clear that there are concerns their data is miss-interpreted, for example, by suggesting prevalence of conditions instead of code frequencies but also because it is not possible to annotate the graphs. A security layer on top of Achilles which allows for authorization per report is also a requested functionality (is already worked on but not released yet).
There has been a meeting with the Achilles developers leads to go through the results of the evaluation. The suggested improvements will be added in the Github placeholder as created recently by Vojtech.
Thanks to all persons involved in this especially Myriam Alexander and Glen James from GSK for creating the questionnaire and the evaluation report!
If there are questions about this evaluation from the OHDSI community let me know.
Peter