| OHDSI Home | Forums | Wiki | Github |
I would like to get some feedback on establishing a standard set of technologies that we leverage for the development of web clients within the OHDSI suite of applications. For example things we currently use across most applications include jQuery, dataTables and Bootstrap. We also leverage libraries like RequireJS in some of our applications.
Feedback welcome.
I think jQuery (and jQuery UI) and Bootstrap are a no brainer. I’ve been using Angular for some of the Heracles starter code I’ve done, but I’m not married to it.
I have a SpringBoot project (at least for now) that uses Velocity templating, so some of the angular templating I do I could probably find a way to do there. Although, that’s not the only thing angular is good for.
I can say from not using angular to using something like angular, it does seem to save a lot of excess code, and seems cleaner to read.
I’ve also used Bloodhound+typeahead for my autocompleter, and have been pretty impressed with it so far, but I’ve used the jQuery UI autocompleter as well.
I’m happy to explore any other libraries or give feedback if anyone has any others we should try.
We have our local Stash repo mirrored to github, and you’re welcome to look at it, although, it’s still very, very young, and a lot may still change.
I agree with including jQuery, dataTables, Bootstrap and RequireJS.
I would also like to include angular.js in the standard. It is great for developing more complex applications. I used it for my sqlrenderweb app (although jQuery probably would have sufficed for that app) and it is a great framework to build on.
I suggest we include d3.js in the standard too.
I haven’t used angular.js but have built a few AJAX enabled web apps including a bioinformatics platform (http://www.ncbi.nlm.nih.gov/pubmed/19443442) and plugins for a community web based annotation platform (Domeo). Some the projects used very lightweight JS libraries while others used frameworks (e.g, GWT). Based on my experience, my question is, should we be enumerating the libraries or rather general principles?
Libraries will change depending on the tasks but I think we could come up with some standard principles that will be helpful for ensuring long-term browser compatibility over the long run if everyone follows them. For example, these come to mind as possible guidelines (of course, I may be wrong about any of these)
use libraries that are licensed such that they can be served from the WebAPI server - this tends to more efficient and avoids the issue of third party server failures (or URL changes) killing OHDSI code
use libraries that either are simple enough to so that browser compatibility is simply not an issue, or have such wide adoption that we can be confident the libraries will evolve as browsers advance in versions
for those apis that use JS and CSS client-side, make a clean distinction in the source projects (e.g. separate folders). Also, user created JS and CSS should probably be separated from library provided JS and CSS
If possible, avoid platforms that produce obfuscated JS code because of some compilation process that generates browser specific code (e.g., GWT). I have found these technologies to be hard to maintain and even harder to bring new developers in because the projects tend to depend on the IDE and its configuration as much as the libraries themselves!
These are just a few of my own opinions.
-R
Great feedback.
I spent yesterday learning the angular library based on suggestions for its use. It is similar to Knockout and seems to then go on to include things beyond it (like services, some of which are already in jQuery). Given that we’re already heavily invested in Knockout (the entire CIRCES editor and HERMES) I’m going to recommend we leverage it where possible for data binding / templates / ui work. I will likely release modules based on Knockout for things like WebAPI selection, using dataTables, etc.
In addition I plan on releasing a web application starter kit that will include base libraries. The intent would be for anyone who wants to write a web application they would be able to clone the starter kit repository and have a basic template to get started without having to worry about all the underlying details.
If you’re building a web app managed by Maven, it’s pretty simple to use WebJars. The nicest thing about it is that you don’t have to include anything in your resources directory, and upgrades to new versions are pretty seamless. Although, hosting libraries on the OHDSI CDN might be another way for libraries to be served up consistently, and serve the benefit of being cached in the browser.
I haven’t used KnockoutJS, but I can look into integrating it into Olympus (our OHDSI app launcher) and Heracles where and if it makes sense.
I wholeheartedly agree on this point:
If possible, avoid platforms that produce obfuscated JS code because of some compilation process that generates browser specific code (e.g., GWT). I have found these technologies to be hard to maintain and even harder to bring new developers in because the projects tend to depend on the IDE and its configuration as much as the libraries themselves!
We’ve had issues internally at Regenstrief when browsers update, things break, and said frameworks are often difficult to debug and diagnose for issues, not speaking directly to GWT, just in general.
Nothing we’re using generates obfuscated code so we should be OK on that front.
Our current web applications are released as static web content html/js/css. This being the case there isn’t any direct integration required with Maven or plans to release them as part of JAR files.
@Frank, thanks for taking the initiative on this and having a starter kit will be a big boon to folks coming in fresh and wanting to start and OHDSI project quickly. I assume the starter kit itself will evolve through the collective group’s efforts.
I do want to ensure we communicate to folks about where standardization is “advisory” and where it is “enforced.” To the latter, it’s pretty easy-- we have certain database dialects we support, a statistical package we support, and a service layer language we use. Any apps not using these dialects, or R, or Java, will pretty much break. For UI stuff, I see our biggest need for standardization in CSS and components. We want our apps to look consistent and use consistent widgets such as our autocompleters, tables (as you’ve mentioned), pickers etc.
As far as whether someone uses knockout or angular or hacks their own path is not going to break anything either technically or from the user experience perspective. So I would consider making our recommendations in this regard advisory. It should not dissuade someone who really likes XYZ.js from using it to build an OHDSI app. But the look and feel should be consistent.
Hi @Frank, I would highly recommend defining some sort of “packaging” (JAR/WAR) for these html/css/js artifacts/applications. I would also recommend using a common build & dependency management tool (Maven) across OHDSI artifacts. Not only is this consistent, but it facilitates project comprehension/conformance, version management (snaphots, releases), and adoption (via dependency management). We would be glad to assist where needed.
I think coming up with a shared style guide based on bootstrap, would too, be beneficial. OpenMRS has this, and it doesn’t have to go so far as that yet, but it is beneficial when bringing on new devs/creating new applications.
@jon_duke It is open source and community oriented so I don’t feel that anyone is in much of a position to enforce anything in particular. For example, while its true that we only officially support certain database platforms today but if someone were to come along and add access for another database platform (say Teradata, Netezza, etc) then they could request it be added through a pull request, establish their own fork, etc.
I would like to standardize on certain javascript libraries because it would mean it would be easier for someone who works on one application to easily contribute to another given they would be familiar with the technologies that are involved. I think about this much in the same way I do the WebAPI. There are half a dozen frameworks that people could use to connect to databases, we’ve agreed upon using Spring. Someone could come certain use something else and to your point it wouldn’t break anything, but it would complicate things. If your team would like to use angular instead of knockout it complicates things, but doesn’t break anything.
@alfranke I don’t see the need to package html files in a JAR. While we have an installation of tomcat to host our WebAPI our standard is still IIS internally. I’m not opposed to someone establishing build scripts that would create a JAR and make it possible for others to deploy in this fashion but it is not something that would be applicable to our environment.
@alfranke, check out OHDSI/Achilles.Web to see how we’ve managed releases of our HTML apps. I don’t quite understand the advantages of bundling non java assets into a JAR. The webapps can be hosted on servers that don’t know anything about Java. If you’re interested in looking into a bundling toolset such as bower or grunt for html/javascript ‘builds’ that’s something we could look into, since it would allow us to incorporate a testing framework into our build cycle.
-Chris
@Chris_Knoll @Frank - I should have worded my comment differently. The emphasis should have been primarily on embracing Maven across the OHDSI platform (Maven implies packaging – usually a WAR/JAR but can be a pom/assembly producing a zip file. Maven also implies that the artifact is deployed in some artifact repository – allowing other projects to easily depend on it). The reason I mentioned JAR/WAR is that those packaging types are more “native” to Maven (and ultimately just an extension of zip format) and would be a bit more straight forward for another project to depend on…
Some thoughts… I figured it was known that OHDSI is a java based platform… Will there really not be a need for server-side code for these apps (e.g. security, runtime administration)? Why not build Achilles and other applications with Java in mind (e.g. default WAR) and provide options to run in alternative ways (e.g. non-java env) if applicable… For example, use Maven, use WebJars and/or whatever other means that makes development easier and/or more manageable. When it comes time to deploy/release, then build/deploy supported versions of the artifact (e.g. WAR, zip, etc.)**.
Anyway, that’s where I was coming from. I realize it doesn’t get much easier than zipping up a directory but efficiencies could be gained by standardization around how projects are managed (not just what libraries are used).
Note: Any questions herein may be considered rhetorical.
**If you’re interested, here is an example of a Maven WAR project that uses WebJars and then extracts the resource files to create an assembly (which one could extract and run with just a browser). Options… https://github.com/fwienber/webjars-extjs-example/blob/master/pom.xml
I wouldn’t say its a java based platform. It is a platform that supports multiple relational database platforms, has a web service layer with the reference implementation in java, has a methods library with implementations in R, and web applications implemented in HTML5/JS.
@alfranke - I took a look at the WAR project, and while interesting, I don’t think it would be utilized the way you might expect. AchillesWeb, for example, having no Java server-side dependencies you might expect that you would be able to run it right from the unzipped folder. However, the resources it accesses must be served from an HTTP host, and I imagine that most/all of our apps will have this constraint. Additionally, I don’t see our apps serving as dependencies for anything else, however there may be components that exist in multiple apps that would be considered a dependency, but only in cases where we standardize on the client stack (eg: a knockout component i don’t think will be accessable in an angular controller context).
Also, I have pulled a lot of java script libraries (jquery, d3, sammy, knockout, etc) and none of those involve war packaging or maven scripts to bundle their scripts. I like the idea of being consistent but to e consistent for consistency’s sake? Let’s pick the right tools for the job. I fully agree that as our service layer is predominantly java (even with R, most likely go through a java-codepath before invoking the R routine), and maven offers specific advantages that we can use (dependency management, wide IDE support, good online support). I don’t agree that it makes it the tool to use in all aspects of our development, tho. Could we get a set of maven actions implemented to package our client side apps and modules? yes. but what are the tools that are move commonly leveraged, that might have better client-specific capabilities that work more out of the box, and have a community behind it? Some links:
http://www.radcortez.com/javascript-package-management-npm-bower-grunt/
This one has some context:
An interesting passage:
Aside: Wro4j is probably not the tool of choice for hard-core front end developers - they would probably be using a node-based toolchain, with bower and/or grunt. These are definitely excellent tools, and covered in great detail all over the internet, so please feel free to use them if you prefer. If you just put the outputs from those toolchains in “src/main/resources/static” then it will all work. I find wro4j comfortable because I am not a hard-core front end developer and I know how to use Java-based tooling.
So, maven-based tooling may be comfortable to those in the Java stack, but even the author acknowledges that there are tools that are 1) widely covered in great detail all over the internet and 2) specifically geared towards front end development.
I think we have a strong framework for our service layer (java-based, maven, git, etc) that will work well, but let’s not push those decisions into another space that has potentially a different target audience.
@Frank & @Chris_Knoll, I appreciate the information in this thread. It definitely helps me better understand your vision of the OHDSI platform. For you to not consider OHDSI a “java based platform” is eye opening. What is foreign to me is the notion that these applications are managed independently, yet their respective application-specific server side business logic is managed in a completely separate project. I will try to make that adjustment. I will say that if that design (WebAPI) proves too unwieldy, you could consider each application having a separate server side component (e.g. JAR) that defines their services/model/utilities and utilize Spring’s classpath scanning, within WebAPI, and transition to a more modular architecture, where WebAPI source only contains common/core services and the application manages its respective logic. I realize that shift might move the platform to a more “java based” reality. Should that ever be the case, and should there ever be the desire to adopt a single build platform, then we could also consider maven javascript support (e.g. frontend-maven-plugin,etc.wro4j-maven-plugin,etc.).
Bear with me as I become more familiar with the OHDSI vision… Thanks.
Thanks, Al. It was informative for me too. When the group was first discussing WebAPI vs. independent services, one of the considerations was that the webAPI would be servicing different applications so webAPI would have the challenge of keeping up with all the other applications that use it, and the code in the WebAPI sevice would become quite monolithic with all the service functionality being packed in to one place that multiple apps depend on.
On the other hand, because all the code is centrally located, any service that we implement that needs functionality from another service doesn’t have to resort to a web-invoke to get access to the functionality. it means that we can leverage thread-based contexts (transactions? security?) because the invokes are all local. I totally agree and understand putting all application required functionality will require some coordination and discipline to have things play together nicely. What I did on the ‘cohortdefinition’ extention to webAPI was i made a branch out of it and just worked in that branch until the service was far enough along to incorporate into master. I think that all new applicaitons that require new extensions to webAPI will branch off webAPI and add functionality until the applicaiton that uses it is set up and the new services can be folded back into webAPI’s master.
Lastly, the other thing I’ve come to realize/accept/understand is that these front end apps are just specific user experience to expose back-end functionality on webAPI. I say this because thinking this way, you won’t think that we’ll need application specific services: all functions, even if they have only a single applicaiton use, will exist in webAPI, because you never know if a future application will want to leverage the same functionality. Vocabulary services, for example, has wide applications: everyone may want to query up a concept. cohort funcitons are a bit more limited, but they are used across circe and heracles. We’re going to have some statistics services which will be used in hermes, and heracles. One thing that helps in thinking this way is that no application ‘owns’ any particular service. Put it in webAPI and consume it from your application.
-Chris
I notice you’re using Curl (why did they pick that name) over RequireJs is Achilles, since they are very similar, can I safely assume we are using these as roughly equivalent technologies?
We’re using RequireJs going forward. Curl was an artifact of an older app.