Configure Aparapi to bundle and load JNI libraries from Aparapi JAR

[GoogleCodeExporter]

As we begin to integrate Aparapi into more generalized and production-ready 
projects, it is becoming obvious that we need a way for Aparapi to bundle all 
of its JNI libraries into the Aparapi JAR and load them automatically, instead 
of relying on the java.library.path to be set by the calling code.

This is entirely possible and has been done by a number of other projects, but 
will require both changes to how we load native libraries and how we build and 
deploy Aparapi.

Original issue reported on code.google.com by [email protected] on 3 Apr 2012 at 5:16

[GoogleCodeExporter]

I will also point out that there are many distro's some are going to include 
AMD stuff some are not, as per normal with linux they will not agree where to 
put the files and each distro that includes them will put them a different 
place, leaving your software flexible to deal with that would probably be a 
good idea. I understand wanting to standardize it this may work for windows 
but,nothing with linux is that way,each distro is different, then think maybe 
someone running freebsd wants to try and run it, a whole new can of worms is 
created.If it is left flexible even the freebsd guy will be able to make it run 
or openindiana for that matter.

Original comment by [email protected] on 13 May 2012 at 3:35

[GoogleCodeExporter]

Please make a post on the front page when you are finished breaking things then 
Ill try it again,this changing and breaking is what has put AMD way behind cuda 

Original comment by [email protected] on 13 May 2012 at 3:43

[GoogleCodeExporter]

This issue ticket is specifically addressing the issue of loading JNI libraries 
"transparently" to the user. There are many instances where it is difficult to 
load JNI libraries externally and this kind of functionality would be ideal. In 
all cases, OpenCL needs to be installed and OpenCL is usually included as part 
of display driver installation.

If a user needs to execute Aparapi on FreeBSD, then they will need to create an 
issue ticket requesting the project to include FreeBSD JNI support. This user 
will also need to supply a FreeBSD build system to provide this project with a 
binary.

Original comment by [email protected] on 11 Aug 2012 at 12:50

[GoogleCodeExporter]

This project performs exactly the same functions I am requesting in this issue 
ticket:

http://one-jar.sourceforge.net/

This project uses a BSD license.

Gary, are you opposed to including this as part of the Aparapi build system? 
Once we have a "one-jar" version of Aparapi, it will vastly simplify all 
deployment types.

Original comment by [email protected] on 11 Sep 2012 at 5:56

[GoogleCodeExporter]

Not opposed at all, this looks a great solution.  

Are you interested in coding this up (in a branch maybe?) so we can try it out? 

Gary

Original comment by [email protected] on 11 Sep 2012 at 6:14

[GoogleCodeExporter]

Gary,

That sounds good. Now that I have more time to work on Aparapi, I was planning 
to check-in my work for Applets/JNLP, but I think this work should be completed 
first. Do you agree?

I've been thinking over how we're going to build and bundle all of the 
necessary native libraries before the One-Jar build is executed. What I'm 
thinking at this point is to add One-Jar as a separate build target to 
com.amd.aparapi. All of the child/demo projects will then be changed to use the 
One-Jar distributable.

The reason I would like to do this before the Applet/JNLP check-in is because 
it will eliminate the need for JNLP to load the Aparapi native libraries. What 
I would like to do is get a nice Applet demo working, ideally nBody since it is 
the most visually appealing. Of course, the nBody code will need to be modified 
to allow the use to select JTP, CPU or GPU instead of passing it in from the 
command-line, since the Applet will not have this option (unless we want it to 
try executing in GPU mode all of the time).

Original comment by [email protected] on 2 Oct 2012 at 12:41

[GoogleCodeExporter]

Hmm, after a lot of work, I have determined that I may be foiled trying to use 
One-Jar. It appears One-Jar circumvents the normal loading of libraries and 
resources by essentially using a custom classloader that starts up first, which 
then starts up user-defined code. The custom classloader then intercepts all 
library and resource loading calls from the user-defined code. In our case, 
Aparapi is simply a resource jar and not a complete application with a main() 
method, so I believe that we'd have to go through a lot of hoops to get this 
working.

Having said that, we definitely could use One-Jar to build applications using 
Aparapi, such as nBody and bundle everything as one nice-and-tidy distributable.

I have some other ideas how to load native libraries embedded within a JAR, so 
I'll start working on those instead.

Original comment by [email protected] on 6 Oct 2012 at 11:17

[GoogleCodeExporter]

After a some searching, there are a ton of good examples for how to do this, 
but I think this example is the best so far:

https://github.com/zeromq/jzmq/blob/master/src/org/zeromq/EmbeddedLibraryTools.j
ava

That class performs exactly the functions I have been working on, especially 
the loadEmbeddedLibrary(). This code snippet is LGPL, which should be 
compatible with Aparapi, correct?

If not, I can re-write all of the logic and submit to Aparapi independently 
under the Aparapi license.

Original comment by [email protected] on 7 Oct 2012 at 12:03

[GoogleCodeExporter]

I think that LGPL is compatible. From my limited 'breakfast reading' I did see 
this 
http://stackoverflow.com/questions/104968/can-i-use-lgpl-library-in-bsd-x11-lice
nsed-project which indicates that we need to make the source and the license 
available. That would be fine.

Would could create a classloader for loading the dll's and then tear it down if 
that helps.  We can't however assume that the users classes will be loadable 
from our class loader.  

Thanks for looking at this. 

Gary

Original comment by [email protected] on 7 Oct 2012 at 3:18

[GoogleCodeExporter]

Thanks Gary. I hope to continue working on this issue this week.

I also spent some time yesterday looking at the FSF's description of GPL vs. 
LGPL vs. BSD etc. and although IANAL it seems that BSD and LGPL licensed code 
should be compatible. Which then got me thinking, it seems that the main 
difference between BSD and LGPL is that LGPL requires any code changes to be 
resubmitted to the original codebase, whereas BSD is basically "public domain". 
Is there a reason a BSD license was chosen instead of LGPL for this project?

Original comment by [email protected] on 7 Oct 2012 at 6:14

[GoogleCodeExporter]

Since I'm looking at some code refactoring/restructuring in 
http://code.google.com/p/aparapi/issues/detail?id=71 I am also investigating a 
reasonable pattern to support additional JNI libraries. To be more specific, 
will we ever have JNI libraries that will support additional functionality, but 
may not be bundled into a single native aparapi library?

For example, would a naming scheme such as the following make sense?

aparapi_opencl_x86_64.dll
aparapi_hsa_x86_64.dll
aparapi_cuda_x86_64.dll
et.al.

Since it appears that most (all?) of the OpenCL specific native methods are 
located in OpenCLJNI.java then I would guess at some point we could have 
HsaJNI.java, CudaJNI.java, etc.?

Or should I not worry about this right now and just focus on natively loading 
exactly what we have, forgoing any thoughts about future flexibility in the 
design?

Original comment by [email protected] on 9 Oct 2012 at 12:56