Error in installing R-Packages

hui_xing_tan · September 5, 2018, 4:22pm

Hi there,

I was try to install Achilles using the install_github command in R Studio but encountered errors in installation. Currently using a Mac OSX and if i type java -version on the terminal, it is java version “1.8.0_91”

Below is the error message in Rstudio.

> install_github("ohdsi/SqlRender")
Downloading GitHub repo ohdsi/SqlRender@master
from URL https://api.github.com/repos/ohdsi/SqlRender/zipball/master
Installing SqlRender
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore  \
  --quiet CMD INSTALL  \
  '/private/var/folders/h0/_ywzt1zx7nn93k337156lprr0000gn/T/RtmpPHsitR/devtools40941abfa33/OHDSI-SqlRender-b87d04f'  \
  --library='/Library/Frameworks/R.framework/Versions/3.4/Resources/library' --install-tests 

During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C" 
2: Setting LC_TIME failed, using "C" 
3: Setting LC_MESSAGES failed, using "C" 
4: Setting LC_MONETARY failed, using "C" 
* installing *source* package 'SqlRender' ...
** R
** inst
** tests
** preparing package for lazy loading
Error : .onLoad failed in loadNamespace() for 'rJava', details:
  call: dyn.load(file, DLLpath = DLLpath, ...)
  error: unable to load shared object '/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so, 6): Library not loaded: /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/lib/server/libjvm.dylib
  Referenced from: /Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so
  Reason: image not found
ERROR: lazy loading failed for package 'SqlRender'
* removing '/Library/Frameworks/R.framework/Versions/3.4/Resources/library/SqlRender'
* restoring previous '/Library/Frameworks/R.framework/Versions/3.4/Resources/library/SqlRender'
Installation failed: Command failed (1)

Appreciate any help or advice here. Thank you in advance

Regards,
Hui Xing

hui_xing_tan · September 5, 2018, 4:16pm

Oh yes, this also applied to 2 other lines:

install_github(“ohdsi/DatabaseConnector”)
install_github(“ohdsi/Achilles”)

rtmill · September 5, 2018, 8:41pm

Looks like an issue with your java path.

Relevant stackoverflow:

t_abdul_basser · September 5, 2018, 10:54pm

Thanks @rtmill! You saved us from having to post the same.

We have users here at who have run into the same issue. Other users running Rstudio on OSX should bookmark your answer since it brings the relevant links to the workarounds together in one place.

hui_xing_tan · September 5, 2018, 11:26pm

Thanks, @rtmill and @t_abdul_basser

This worked for me:

In RStudio:

Sys.getenv('JAVA')
#no path set
#set path
Sys.setenv(JAVA_HOME='/Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home/jre')

hui_xing_tan · September 6, 2018, 2:15pm

Now I have installed Achilles and SqlRender and but the library failed to load:

library(SqlRender)
Error: package or namespace load failed for ‘SqlRender’:
.onLoad failed in loadNamespace() for ‘rJava’, details:
call: dyn.load(file, DLLpath = DLLpath, …)
error: unable to load shared object ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so’:
dlopen(/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so, 6): Library not loaded: /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/lib/server/libjvm.dylib
Referenced from: /Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so
Reason: image not found

library(SqlRender)
Error: package or namespace load failed for ‘SqlRender’:
.onLoad failed in loadNamespace() for ‘rJava’, details:
call: dyn.load(file, DLLpath = DLLpath, …)
error: unable to load shared object ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so’:
dlopen(/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so, 6): Library not loaded: /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/lib/server/libjvm.dylib
Referenced from: /Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so
Reason: image not found
library(Achilles)
Loading required package: SqlRender
Error: package or namespace load failed for ‘SqlRender’:
.onLoad failed in loadNamespace() for ‘rJava’, details:
call: dyn.load(file, DLLpath = DLLpath, …)
error: unable to load shared object ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so’:
dlopen(/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so, 6): Library not loaded: /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/lib/server/libjvm.dylib
Referenced from: /Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/libs/rJava.so
Reason: image not found
Error: package ‘SqlRender’ could not be loaded

I have read online that it has something to do with rJava, posts in link below suggests setting the directory of Java location and installing 64-bit instead of 32 bit. Have done both

Please let me know if you spot the error, thank you

t_abdul_basser · September 6, 2018, 5:13pm

You appear to still be having issues loading the Java library from RStudio on OS X. Have you tried

sudo ln -f -s $(/usr/libexec/java_home)/jre/lib/server/libjvm.dylib /usr/local/lib as per

rtmill · September 6, 2018, 6:55pm

To narrow it down, you could try loading the libraries in the R console to see if RStudio is in fact the issue.

hui_xing_tan · September 7, 2018, 3:07pm

Hi.

Thanks for the advice. I’ve just tried running the sudo command in terminal but still facing the same error when running library(Achilles) in R-Studio. I’ve also tried loading the libraries in R console but get the same error message.

I’m not sure if this is an issue: I was unable to run sudo comments unless I logged out from the current user and logged in as root user on Mac. But as root user I do notice that my .bash profile is completely blank, whereas when I am logged in as the original user, my JAVA_HOME is set to:

export JAVA_HOME=/usr/libexec/java_home -v 1.8

Further when I type ‘java -version’ into the terminal, the version reflected is different from when I am using the other account

root user:
java 10.0.2 2018-07-17
Java™ SE Runtime Environment 18.3 (build 10.0.2+13)
Java HotSpot™ 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)

original user:
java version “1.8.0_91”
Java™ SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot™ 64-Bit Server VM (build 25.91-b14, mixed mode)

rtmill · September 7, 2018, 3:59pm

I think @t_abdul_basser is on the right path.

When you run ‘library(rJava)’ does it load properly?

Try:
remove.packages(rJava)
(close R studio)
sudo ln -f -s $(/usr/libexec/java_home)/jre/lib/server/libjvm.dylib /usr/local/lib
install.packages(“rJava”, dependencies = TRUE, type=“source”)

hui_xing_tan · September 9, 2018, 5:33am

Thanks @rtmill and @t_abdul_basser for the helpful and fast replies

Sorry for the late response. The thing is, I’m having trouble running the sudo commands, as mac terminal won’t accept my password even though I’m pretty sure I keyed in the correct admin password. argh. Let me figure this password thing out and I’ll let you know if I make any progress!

hui_xing_tan · September 9, 2018, 6:29am

By the way, would it be possible to run Altas without running Achilles?
We have set up WebAPI and Atlas, and also have the CDM tables in our postgresql database, but the data sources are not showing on Atlas.

Please advise if possible. Thanks!

Attached screenshots showing the current status

This is after startup tomcat and type http://localhost:8080/WebAPI/Source/Sources in browser

This is after installing atlas and viewing the jobs (http://localhost:8080/Atlas/#/jobs) and config (http://localhost:8080/Atlas/#/configure) panels:

These are the schema set up in postgresql: webapi (69 tables) and cdm (39 tables). There is also another empty schema called results which is located in the same database as cdm, understand it will be populated once we get achilles working

These are our source and source and source_daimon tables:

Thank you

Chris_Knoll · September 9, 2018, 7:19pm

You can run atlas without running Achilles. Atlas lets you view Achilles results within the Atlas ‘Data Sources’ module, but you can do other tings besides browse Achilles results in Atlas!

That looks like you’ve set up WebAPI enough that you could do vocabulary searches (there is a source with a ‘vocabulary’ daimon set up). You gave a screeesnot of the ‘cdm’ schema, but I do not know if you have an ‘ohdsi’ schema set up in your CDM database (‘ohdsi’ is the name you gave to your results schema). You should create the result schema tables using the WebAPI’s ddl service:
http://localhost:8080/WebAPI/ddl/results?dialect=postgresql&schema=ohdsi

Due to the vast number of RDBMS platforms out there that we support for doing analytics, we do not automatically maintain the results schema. You’ll have to deploy these tables manually, and, if there are schema changes later, perform your own migrations to keep the data intact.

Once you deploy the tables to your ohdsi schema, you can generate cohorts, and do more interesting things like perform an incidence rate analysis.

hui_xing_tan · September 10, 2018, 2:16pm

Thanks, @ Chris Knoll. I think we may have configured the source and source daimons incorrectly. This was what we typed for the WebAPI SOURCE and SOURE_DAIMON Inserts:

INSERT INTO webapi.source (source_id, source_name, source_key, source_connection, source_dialect) VALUES (1, 'My Cdm', 'MY_CDM', 'jdbc:postgresql://127.0.0.1:5433/OHDSI?user=ohdsi_admin_user&password=admin1', 'postgresql');
INSERT INTO webapi.source (source_id, source_name, source_key, source_connection, source_dialect) VALUES (2, 'Default vocabulary', 'vocab', 'jdbc:postgresql://127.0.0.1:5433/OHDSI?user=ohdsi_admin_user&password=admin1', 'postgresql');
INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (1,1,0, 'cdm', 0);
INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (2,1,1, 'cdm', 0);
INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (3,1,2, 'ohdsi', 0);
INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (4,2,1, 'cdm', 1);

Instead of ‘OHDSI’, the database should be ‘CDM’ ?

Also, is there a difference between the results schema created by webapi ddl service and the results schema created by achilles?

Our current structure is as follows

CDM database:
-cdm schema
-results schema (for Achilles)

OHDSI database:
-webapi schema

We don’t have an ‘ohdsi’ schema set up in our CDM database
if we create the ohdsi schema from webapi ddl service, under which database should we keep it and can any user create the tables? Sorry for the multiple questions, and thanks all so much!

Hui Xing

Chris_Knoll · September 10, 2018, 6:26pm

Hi, Hui Xing,
if the name of your CDM database is ‘CDM’ then your connection string in your SOURCE table should be:

INSERT INTO webapi.source (source_id, source_name, source_key, source_connection, source_dialect) VALUES (1, 'My Cdm', 'MY_CDM', 'jdbc:postgresql://127.0.0.1:5433/CDM?user=ohdsi_admin_user&password=admin1', 'postgresql');

You really need to keep the database for WebAPI completely separate from your CDM databases. It looks like you’re using authentication for your CDM from your WebAPI database, which is valid, but can be confusing…you can specify in your SOURCE record a cdm-specfic user/password that has rights to read your cdm schema and write to your results schema.

If your results schema is called ‘results’ then your insert into source_daimon is also incorrect, and it should be:

INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (3,1,2, 'results', 0);

The point of the SOURCE table is to store how to connect to your CDM database
If you see an OHDSI value in your connection string but that is not the name of your CDM database, you have an error.

The point of SOURCE_DAIMON is to store the names of your schemas. If you see an ‘ohdsi’ value in your source daimon table_qualifier, but you don’t have an ‘ohdsi’ schema in your cdm database, then you have an error.

Yes, there are differences. When you run Achilles, it will create an ‘achilles_results’ and ‘achilles_results_dist’ in the specified schema. If you want Atlas/WebAPI to read those tables, then you should set the results schema name in your Achilles call to write to the same schema as the cdm results schema.

I recognize this is confusing: now we are talking about an Achilles results schema? Achilles is something that can be run completely stand-alone. You don’t need WebAPI to run achilles, you don’t need a results schema created based on localhost/WebAPI/ddl/reults since achilles creates their own tables. However, if you do want to browse Achilles results in Atlas, then you will need to put the achilles table into the CDM’s results schema, because that is where Atlas/WebAPI is going to look for the achilles_results and achilles_results_dist tables. You can do that directly in achilles by specifying the ‘resultsSchema’ parameter as your cdm’s results schema. But this will only create achilles-oriented tables, In order for WebAPI to work on the CDM, you need to create the tables in the results schema from localhost/WebAPI/ddl/results.

hui_xing_tan · September 11, 2018, 2:25pm

Ah, i get it, the source and source daimon tables should point to the database where our cdm and results schemas reside, and the username and passwords refer to the user that has the rights to read/write the cdm and results schema, not the ohdsi_admin_user if they are different

Ok, in the end we created a results schema called ‘ohdsi’ under our CDM database, using the ddl from localhost/WebAPI/ddl/reults. For WebApi to read the achilles results In our achilles call we should specify the results schema as ‘ohdsi’, is that correct?

We have changed our connection string and managed to get the vocaulary loading in webapi:

However, the profile is still loading after one hour of waiting. Would this be due to the configuration is still incorrect , or some other reason?

Thank you,
Hui Xing

Chris_Knoll · September 12, 2018, 4:47pm

You got it!

You got it again! Tho, I would have called the schema ‘results’ because ‘ohdsi’ doesn’t really convey what the schema is for, but, it’s just a name, if it is working for you, great. just don’t confuse the ‘ohdsi’ schema that is your results schema with your WebAPI schema (which I think you called webapi)

Congrats! That means your webAPI is looking at the source table to figure out where to find the vocabulary tables, and performing the search. You should also be able to generate cohorts.

Well, in your browser, open up your console (control-shift-i in chrome) and see if you didn’t get some sort of error. Possibly it is a query error. But what it looks like you are doing is trying to pull up a profile for person_id = ‘12’. Does person_id 12 exist in your data?

I’m sorry the error handling isn’t more clear there. You shoudln’t need to wait an hour for a profile to load, it should only take 20-30 seconds depending on the volume of data.

hui_xing_tan · September 13, 2018, 11:06pm

@Chris_Knoll

Ah, I see, ok. Let me start a new discussion thread and get back.

Cheers!

hui_xing_tan · September 13, 2018, 11:28pm

Please see Loading Profiles and Cohort Generation in Atlas

hui_xing_tan · September 30, 2018, 5:41am

@rtmill and @t_abdul_basser

managed to install rJava by following the instructions here:

It turns out that sudo on mac dosent work with blank passwords

Thanks