Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Commit

Permalink
Re-organize scala maven build
Browse files Browse the repository at this point in the history
1. Automatically detect which platform to build for scala.
2. Remove platform dependend submodules
3. Fix cyclic module dependencies
4. Fix scalatype style check
5. Now mvn can be executed in submodule
6. Maven build can be executed from any directory not only in root project
7. Checkin javah header file, and use verify task to detect native API changes
8. Improve incremental build performance
9. Remove unittest and integration-test profile, use proper task instead
10. Delete generated scala file during maven clean.
  • Loading branch information
frankfliu committed Dec 14, 2018
1 parent 77fe96e commit 0603c23
Show file tree
Hide file tree
Showing 41 changed files with 1,651 additions and 1,585 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -678,7 +678,7 @@ scaladeploylocal:
-Dlddeps="$(LIB_DEP) $(ROOTDIR)/lib/libmxnet.a")

jnilint:
3rdparty/dmlc-core/scripts/lint.py mxnet-jnicpp cpp scala-package/native/src
3rdparty/dmlc-core/scripts/lint.py mxnet-jnicpp cpp scala-package/native/src --exclude_path scala-package/native/src/main/native/org_apache_mxnet_native_c_api.h

rclean:
$(RM) -r R-package/src/image_recordio.h R-package/NAMESPACE R-package/man R-package/R/mxnet_generated.R \
Expand Down
2 changes: 1 addition & 1 deletion contrib/clojure-package/project.clj
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
;[org.apache.mxnet/mxnet-full_2.11-linux-x86_64-gpu "1.2.1"]

;;; CI
[org.apache.mxnet/mxnet-full_2.11-linux-x86_64-cpu "1.5.0-SNAPSHOT"]
[org.apache.mxnet/mxnet-full_2.11 "1.5.0-SNAPSHOT"]

[org.clojure/tools.logging "0.4.0"]
[org.apache.logging.log4j/log4j-core "2.8.1"]
Expand Down
44 changes: 24 additions & 20 deletions scala-package/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ It brings flexible and efficient GPU/CPU computing and state-of-art deep learnin
and apply them to tasks such as image classification and data science challenges.

Install
------------
-------

Technically, all you need is the `mxnet-full_2.11-{arch}-{xpu}-{version}.jar` in your classpath.
It will automatically extract the native library to a tempfile and load it.
Expand All @@ -26,45 +26,49 @@ Use the following dependency in maven, change the artifactId according to your o
```HTML
<dependency>
<groupId>org.apache.mxnet</groupId>
<artifactId>mxnet-full_2.10-linux-x86_64-gpu</artifactId>
<version>0.1.1</version>
</dependency>
```

You can also use `mxnet-core_2.10-0.1.1.jar` and put the compiled native library somewhere in your load path.

```HTML
<dependency>
<groupId>org.apache.mxnet</groupId>
<artifactId>mxnet-core_2.10</artifactId>
<version>0.1.1</version>
<artifactId>mxnet-full_2.11-linux-x86_64-gpu</artifactId>
<version>1.5.0</version>
</dependency>
```

If you have some native libraries conflict with the ones in the provided 'full' jar (e.g., you use openblas instead of atlas), this is a recommended way.
Refer to the next section for how to build it from the very source.

Build
------------
-----

Checkout the [Installation Guide](http://mxnet.incubator.apache.org/install/index.html) contains instructions to install mxnet.
### Prerequisites

* Check out MXNet source repository:

Checkout the [Installation Guide](http://mxnet.incubator.apache.org/install/index.html) contains instructions to install mxnet.
Then you can compile the Scala Package by

* Build MXNet shared library

Scala maven build assume you already have a ``lib/libmxnet.so`` file.

* You have JDK installed in your system.

### Build scala package

Scala build will detect your platform (OSX/Linux) and libmxnet.so flavor (CPU/GPU).

```bash
make scalapkg
cd scala-package
mvn install
```

(Optional) run unit/integration tests by

```bash
make scalaunittest
make scalaintegrationtest
mvn integration-test
```

Or run a subset of unit tests by, e.g.,

```bash
make SCALA_TEST_ARGS=-Dsuites=org.apache.mxnet.NDArraySuite scalaunittest
mvn -Dsuites=org.apache.mxnet.NDArraySuite integration-test
```

If everything goes well, you will find jars for `assembly`, `core` and `example` modules.
Expand Down Expand Up @@ -100,7 +104,7 @@ tracker/dmlc_local.py -n 2 -s 2 \
Change the arguments and have fun!

Usage
-------
-----
Here is a Scala example of what training a simple 3-layer multilayer perceptron on MNIST looks like. You can download the MNIST dataset using [get_mnist_data script](https://github.com/apache/incubator-mxnet/blob/master/scala-package/core/scripts/get_mnist_data.sh).

```scala
Expand Down
131 changes: 0 additions & 131 deletions scala-package/assembly/linux-x86_64-cpu/pom.xml

This file was deleted.

This file was deleted.

Loading

0 comments on commit 0603c23

Please sign in to comment.