I’m trying to connect my CDM in Databricks to Atlas.
If I use the URL below running Atlas on a local instance of Tomcat everything seems to work and I am able to see my Databricks data in Atlas (note: ssl=1 in the URL I’m using).
If I try to use the same URL in Broadsea/Docker I get the error shown below (full output from Broadsea/webapi is attached).
URL:
jdbc:spark://dbc-9c1b9c59-a942.cloud.databricks.com:443/default;transportMode=http;UseNativeQuery=1;ssl=1;httpPath=sql/protocolv1/o/2010859959023967/0602-003109-756w8ulw;AuthMech=3;UID=token;PWD=dapi8ae48c35e859579a5531ff2005907544
Error:
2023-06-09 11:47:49 java.sql.SQLException: [Simba][SparkJDBCDriver](500164) Error initialized or created transport for authentication: problem accessing trust store.
2023-06-09 11:47:49 at com.simba.spark.hivecommon.api.TETSSLTransportFactory.createSSLContext(Unknown Source)
2023-06-09 11:47:49 at com.simba.spark.hivecommon.api.TETSSLTransportFactory.getHttpsClient(Unknown Source)
2023-06-09 11:47:49 at com.simba.spark.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
Full Output from Broadsea/webapi:
broadsea-ssl-error.txt (57.6 KB)
@lee_evans
@Ajit_Londhe
@Chris_Knoll
@Sanjay_Udoshi
Hoping you might be able to help.
So this is not a settled topic yet, but here’s where we are on how to solve it:
Likely you’ll need to mount a cacerts file that provides the trust between the WebAPI JVM and Databricks. Your Tomcat instance likely has this by being installed directly on your server/machine, but in Docker, it doesn’t have this inherently.
In Broadsea, there is a blank cacerts file, you can overwrite it with one for your site. To generate this on your machine, here’s one guide:
https://docs.oracle.com/cd/E19798-01/821-1841/gjrgy/
Then add it to your Broadsea folder (overwriting the blank cacerts file), stop and remove the ohdsi-webapi container, and re-deploy it.
I’ve added a Git issue for us to add this info to the readme:
https://github.com/OHDSI/Broadsea/issues/94
Thanks @Ajit_Londhe!!!
I’ll give it a try and let you know how I make out.
Getting closer…
I was able to follow the instructions at To Use keytool to Create a Server Certificate (The Java EE 6 Tutorial).
I ran the following:
keytool -genkey -alias server-alias -keyalg RSA -keypass changeit -storepass changeit -keystore keystore.jks
keytool -export -alias server-alias -storepass changeit -file server.cer -keystore keystore.jks
keytool -import -v -trustcacerts -alias server-alias -file server.cer -keystore cacerts.jks -keypass changeit -storepass changeit
rename cacerts.jks to cacerts and add to Broadsea root
I tried a couple of different approaches to adding the files and finally added all of the generated files to both the Broadsea root and Broadsea/certs folder but I’m still getting the error shown below (full output is attached):
ssl-error.txt (52.4 KB)
2023-06-10 11:29:20 Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
2023-06-10 11:29:20 at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
2023-06-10 11:29:20 at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
2023-06-10 11:29:20 at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
2023-06-10 11:29:20 at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:451)
2023-06-10 11:29:20 ... 148 more
2023-06-10 11:49:04 2023-06-10 15:49:04.077 INFO taskScheduler-5 org.ohdsi.webapi.executionengine.service.ScriptExecutionServiceImpl - [] - Invalidating execution engine based analyses
2023-06-10 11:49:04 2023-06-10 15:49:04.285 INFO taskScheduler-7 org.ohdsi.webapi.tag.TagService - [] - Starting tags statistics refreshing
2023-06-10 11:49:04 2023-06-10 15:49:04.423 INFO taskScheduler-7 org.ohdsi.webapi.tag.TagService - [] - Finishing tags statistics refreshing
Can you try this to build the cacerts file and then add it to broadsea root? Be sure to completely stop and remove the ohdsi-webapi container before redeploying it.
https://knowledge.informatica.com/s/article/000174467?language=en_US
Good morning @Ajit_Londhe. I was able to get this to work with a combination of the guidance in your posts including the latest. I’ll write up my notes and post here when I get a chance. Thanks again!!!
1 Like
Great!
Would it be possible to get this writeup as a pull request in Broadsea’s README file?