use jar-dependencies for requiring jars

[mkristian]

this gives back the control of which jars are loaded when. you can use jbundler or maven or gradle to manage the jars. actually you manage gems and jars almost in the same manner.

it also gives you a chance to work around certain classloader problems when you really need to not delegate the classloading to the parent classloader.

currently I am working on pysch to do the same
https://github.com/mkristian/psych/blob/jruby-build/Rakefile#L41
since we want have psych as default gem in jruby-9k

jruby-openssl is an example to use jar-dependencies with still packing the jars but use jar-dependencies when available (so older jrubies work as well)
https://github.com/jruby/jruby-openssl/blob/master/jruby-openssl.gemspec#L23
https://github.com/jruby/jruby-openssl/blob/master/lib/jopenssl/load.rb#L8

[mkristian]

see also guyboertje/jrjackson#35

[flavorjones]

Hi there,

Sorry again, but you'll need to fully explain what the problem you're trying to solve is. Not being a full-time Java developer, I have no idea what jar-dependencies is, what problem it is supposed to help, what the tradeoffs are to supporting it, nor what alternative solutions were considered.

Let's please just start with a clear description of the problem? We can then proceed to a discussion of solutions. Thanks for your patience.

[mkristian]

in jruby you can use gems and jars almost in the same manner as they provide libraries your code can use. gems are managed by rubygems/bundler and they usually declare their gem-dependencies inside their gemspec - to vendor gems is not that common (any more). in the java world jar can have dependenies and they can be declared as well - used by maven, buildr, ivy, gradle, etc BUT gems like nokogiri vendor their jars, i.e. they get loaded at some point in time into the jruby-classloader. there is no way to find out which jars with which version was loaded since in java the jars do not contain such meta information (those tools above provide thise meta data on side). let say I have a java library depending on xerces or xalan and my jruby project is using it along nokogiri. tools like jbundler or maven or gradle can handle the java libraries for your project but they can NOT inlcude those vendored jars of nokogiri into account. so you end up with loading the same jar TWICE into the jruby classloader - maybe even with different versions ! so jar-dependencies first gives a way to "declare" the jar inside the gemspec in a similar manner as you would declare gem dependencies: https://github.com/jruby/jruby-openssl/blob/master/jruby-openssl.gemspec#L23 now you still can vendor those jars BUT load them through jar-dependencies and this will warn you on version conflict on those jars or not load the jar a second time. declaring the jar inside the gemspec also gives maven-like tools a way to manage those jars as well: http://repo1.maven.org/maven2/rubygems/jruby-openssl/0.9.5/jruby-openssl-0.9.5.pom (dependendies tag) which is derived from the respective gemspec file. on top jbundler uses the maven meta-info of gem to find their jar dependencies. this is all about getting gems and jars on the same level for jruby and help projects which life as much inside the java world as inside the ruby world - jruby being the bridge between them. one last thing: with https://github.com/lookout/leafy we do not vendor the jar anymore (jar dependencies hooks into rubygems and installs them on) but this needs jruby-1.7.13+ for jruby-openssl we do support older jrubies so we vendor those jars for those old jrubies.

[mkristian]

@jvshahid will continue my thingy from #1395 (comment) here

declaring jar dependencies inside the gemspec file

as usual there is a ruby side of things and a java side of things

ruby tools

having those jar dependencies declared inside nokogiri gemspec allows tools like jbundler or jar-dependencies to manage jars in similar manner as gems. both tools can use Jarfile to setup jars for a jruby project - see https://gist.github.com/mkristian/a46851705427f68ce310. they do obey the jars declared inside a gemspec when used with bundler's Gemfile or can lockdown you jar versions for a gem project which has nokogiri as transitive dependency.

loading jars via jar-dependencies gems also ensures that each jar is loaded only with a single version and any attempt to load the jar with another version will produce only a warning without loading the jar. just an extra safety net.

java tools

both maven and gradle have plugins which handle gems and jars in the same manner, gem and jar are just different kind of dependencies with a different packaging forma and are treated alike otherwise. but they depend on having jars declared inside the gemspec to be able to integrate them into their dependency resolution. for embedded jruby this is even more important then for a ruby project. a java application which uses embedded jruby does come with its own set of dependencies. and those dependencies from the java application are already part of the jruby runtime - as per design of jruby classloaders. here it would be important to tell jar-dependencies which jars are already loaded by the underlying container.

possible issues

of course forcing a different version of xerces or using xercesMinimal instead of xerces can cause nokogiri to produce bugs as it uses a different version then the one from nokogiri itself. on the other hand this is common situation with java projects.

what needs to be done

• declare the jars inside the gemspec
• load the jars via require_jar(group_id, artifact_id, version)
• store the jars under ./lib/#{group_id}/#{artifact_id}/#{version}/#{artifact_id}-#{version}.jar

this is the manual way of doing it. look at https://github.com/mkristian/jar-dependencies/tree/master/examples/gem-with-java-extension-and-jar-dependencies/using-rake-compiler for another way of doing it.

the whole thing comes with another gem dependency but jruby-1.7.x and jruby-9k does have this extra gem already as default gem.

in short

• declare the jars inside the gemspec - for dependency management tools
• load jars via jar-dependencies - allow to override the default from the gem which is used by jar dependency management tools as well
• store the jars slightly different

the only impact for the average nokogiri user is that jar-dependencies gem does get activated and used as well. once the jars are loaded there is no further impact on the nokogiri

[jvshahid]

@headius what do you think about this proposal ? I personally like it but I'm worried about wide adoption and edge cases that could arise from using jbundler and jar-dependencies. your thoughts will be very helpfull.

[jvshahid]

/cc @yokolet

[mkristian]

@jvshahid FYI all jars which are embedded in jruby and on maven-central are loaded via jar-dependencies: jline-2.11.jar, the 2 bouncy-castle jars from jruby-openssl, and snakeyaml-1.14 for psych gem

[mkristian]

any decision ?

[flavorjones]

I picked this up with @jvshahid this morning. Won't make the v1.9.0 release this week; but hoping to adopt this approach for the following release.