OHDSI Home | Forums | Wiki | Github

Using Database Connector Package for Impala

patientprediction
(Kristin Feeney Kostka, MPH) #1

I want to be able to use the awesome skills I learned in the Population-Level Estimation and PLP tutorials on my stack. I’m trying to connect with the DatabaseConnector package to Impala. I’ve read the documentation for DatabaseConnector but want to get some more input from the authors on how they are handling things like Kerberos.

Not sure who authored this part of the package, maybe @tomwhite @shawndolley know? Looking to get an example of someone else using DatabaseConnector with Impala and walking through I have my ducks in a row.

(Tom White) #2

Hi Kristin,

Thanks for the question. The DatabaseConnector package uses JDBC, and Kerberos uses different JDBC authentication parameters to the regular username/password authentication. So DatabaseConnector won’t work as it stands with Kerberos and Impala since it doesn’t know about these parameters.

Do you have a Kerberos Impala environment you are trying to use or is this more hypothetical? It’s generally worth getting things working with Impala standard authentication (at least in a test environment) before enabling Kerberos.

Thanks,
Tom

(Gregory Klebanov) #3

hey @tomwhite. I am glad that Kristin raised this question on forums. Yes, I know of a couple of places already where Kerberos is required to authenticate against Impala. If you do not have a solution for this today - I would love to team up on this - please let me know

Greg

(Kristin Feeney Kostka, MPH) #4

Thanks @tomwhite and @gregk for chiming in. Yes, we have at least one environment where we use Kerberos with Impala and have multiple CDMs loaded in. I’m out of the country this week but perhaps we could schedule a call next week? I had a feeling it wasn’t 1-to-1. Would be great to pick your brains and figure out a way to make it work. I’ve used rImpala but this isn’t ideal for the Methods Library. Happy to work with you all to author something that can fit the bill.

@seslava May be interested in joining as well!

(Sergio Eslava) #5

Thanks @krfeeney for raising this question, which is of great importance for my team as well. I will certainly be very interested in joining the discussion, and willing to work towards finding a solution that benefits other Impala users in the community.
Regards,
Sergio

(Lee Evans) #6

@krfeeney I’m tagging @mvanzandt here because this would be a great topic for discussion in an OHDSI Hadoop Work Group call.

(Kristin Feeney Kostka, MPH) #7

Great! Let’s move the needle. @mvanzandt Is there an opportunity to add this into the working group call agenda? Would be great to keep the post-tutorial momentum.

(Mui Van Zandt) #8

@krfeeney. Yes, this can be added to the agenda. The next Hadoop working group meeting is November 10th. Please join it and we can discuss this.

(Kristin Feeney Kostka, MPH) #9

Hi Mui - sorry to miss today’s call, am traveling internationally again and timing didn’t sync up. Can you send me the dial-in info and when the next call will occur? Still interested in getting into this conversation.

(Sergio Eslava) #10

I wanted to revive this topic in view of some exciting new developments. It’s been a while since we had the chance to discuss the topic of the DatabaseConnector not supporting Kerberos authentication when using Apache Impala, so my team at Celgene took upon ourselves this task as a contribution to the OHDSI community, and to do so we partnered with an external team of developers working with us in other projects, and they successfully implemented changes to DatabaseConnector to support Impala with Kerberos authentication, and I had the chance to test it on our own Cloudera environment, which uses Kerberos, with satisfactory results. One of the developers, Vladislav Kovchug, submitted a pull request yesterday:


Anybody knows who needs to approve that pull request?
Thanks,
Sergio

t