Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
5b63485
Update README.txt
Dec 13, 2016
39bd5a3
Tensorflow On Yarn: 1. AppMaster 2. Yarn Client 3. Tensorflow Container
muzhongz Dec 16, 2016
c723034
PM: rm useless pom file
muzhongz Dec 26, 2016
9d0d013
AM, CONTAINER: 1. add cluster spec's dynamic creation 2. change conta…
muzhongz Dec 26, 2016
ca01f0d
MAVEN: add tf project to the yarn application list
muzhongz Dec 26, 2016
4633fb7
TIMELINE: clean useless codes "timeline"
muzhongz Dec 26, 2016
78ef3bd
initial commit
huafengw Dec 23, 2016
ff6892c
add impl
huafengw Dec 26, 2016
993dd03
move to the proper module
huafengw Dec 26, 2016
b9e391a
remove AbstractService
huafengw Dec 27, 2016
e16c550
Merge branch 'TF-YARN' of https://github.com/drankye/hadoop into TF-YARN
muzhongz Dec 27, 2016
3750d6c
APP: change options for client and appmaster
muzhongz Dec 27, 2016
ea04b4a
RPC: integrate client-am rpc
muzhongz Dec 27, 2016
a5217cd
Merge branch 'TF-YARN' of https://github.com/drankye/hadoop into TF-YARN
muzhongz Dec 27, 2016
5e1a71f
PM: rm useless files
muzhongz Dec 27, 2016
490c0e8
ClIENT, AM: optimize the options in client side and app master side
muzhongz Dec 28, 2016
00599d0
PM: add license
muzhongz Dec 29, 2016
d5042f6
CLIENT: support minist.py
muzhongz Dec 29, 2016
58385ea
RES: fix colliction in TF worker when copying resource from hdfs
muzhongz Dec 29, 2016
bab3945
SCRIPT: add yarn-tf for running minist.py
muzhongz Dec 29, 2016
61c4f81
Update README.txt
Dec 30, 2016
83a49b9
initial commit of tensorflow jni bridge
huafengw Dec 30, 2016
b0b58e3
add missing pom file
huafengw Dec 30, 2016
4a8f57e
MEM: change jvm memory size to 4096
muzhongz Dec 30, 2016
72379da
remove pre-compiled protobuf java classes
huafengw Jan 5, 2017
4602f00
RES: copy the files to hdfs once on Client endpoint
muzhongz Jan 12, 2017
741f5f7
pass the "container_memory" by jvm parameters
muzhongz Jan 17, 2017
1ad6b80
OPT: convert the raw string options of client and am to constants
muzhongz Jan 17, 2017
7adbda8
Create README.md
huafengw Jan 23, 2017
4d5a247
Update README.md
huafengw Jan 23, 2017
0c2f984
initial commit of tensorflow jni bridge
huafengw Dec 30, 2016
ca0a0b1
add missing pom file
huafengw Dec 30, 2016
e3ce18a
remove pre-compiled protobuf java classes
huafengw Jan 5, 2017
c4282be
Update README.md
huafengw Jan 23, 2017
c40adac
move to proper folder
huafengw Jan 25, 2017
fc82609
remove module tensorflow-bridge
huafengw Jan 25, 2017
3a8abef
Update README.md
huafengw Jan 25, 2017
128b2a2
Merge pull request #1 from huafengw/test
Jan 25, 2017
7f66b48
assembly tensorflow-bridge artifact
huafengw Jan 25, 2017
584f818
documnet and code clean
Jan 25, 2017
a8ae522
merge TF-YARN from zhankun's branch
Feb 7, 2017
bcb5598
delete origin HDL's tensorflow bridge project
Feb 7, 2017
74e48e7
Merge pull request #1 from tangzhankun/zhankun-HDL
Feb 7, 2017
4ca563c
add hadoop-yarn-mxnet project into HDL
Feb 9, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 2 additions & 30 deletions README.txt
Original file line number Diff line number Diff line change
@@ -1,31 +1,3 @@
For the latest information about Hadoop, please visit our website at:
A mirror fork of Apache Hadoop for support of Deep Learning.

http://hadoop.apache.org/core/

and our wiki, at:

http://wiki.apache.org/hadoop/

This distribution includes cryptographic software. The country in
which you currently reside may have restrictions on the import,
possession, use, and/or re-export to another country, of
encryption software. BEFORE using any encryption software, please
check your country's laws, regulations and policies concerning the
import, possession, or use, and re-export of encryption software, to
see if this is permitted. See <http://www.wassenaar.org/> for more
information.

The U.S. Government Department of Commerce, Bureau of Industry and
Security (BIS), has classified this software as Export Commodity
Control Number (ECCN) 5D002.C.1, which includes information security
software using or performing cryptographic functions with asymmetric
algorithms. The form and manner of this Apache Software Foundation
distribution makes it eligible for export under the License Exception
ENC Technology Software Unrestricted (TSU) exception (see the BIS
Export Administration Regulations, Section 740.13) for both object
code and source code.

The following provides more details on the included cryptographic
software:
Hadoop Core uses the SSL libraries from the Jetty project written
by mortbay.org.
We're working on this.
3 changes: 3 additions & 0 deletions hadoop-deeplearning-project/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Hadoop Deep Learning Project
======================
##[YARN-TensorFlow](YARN-TensorFlow/hadoop-yarn-applications-tensorflow/README.md)
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-deeplearning-project</artifactId>
<version>3.0.0-alpha2-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>YARN-MXNet</artifactId>
<name>YARN-MXNet</name>

<dependencies>
<!-- 'mvn dependency:analyze' fails to detect use of this dependency -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<scope>provided</scope>
<exclusions>
<exclusion>
<groupId>commons-el</groupId>
<artifactId>commons-el</artifactId>
</exclusion>
<exclusion>
<groupId>tomcat</groupId>
<artifactId>jasper-runtime</artifactId>
</exclusion>
<exclusion>
<groupId>tomcat</groupId>
<artifactId>jasper-compiler</artifactId>
</exclusion>
<exclusion>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jsp-2.1-jetty</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</dependency>
<dependency>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</dependency>
<!-- 'mvn dependency:analyze' fails to detect use of this dependency -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-annotations</artifactId>
</dependency>
<!-- 'mvn dependency:analyze' fails to detect use of this dependency -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<!-- 'mvn dependency:analyze' fails to detect use of this dependency -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-api</artifactId>
</dependency>
<!-- 'mvn dependency:analyze' fails to detect use of this dependency -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-common</artifactId>
</dependency>
<!-- 'mvn dependency:analyze' fails to detect use of this dependency -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-client</artifactId>
</dependency>
<!-- 'mvn dependency:analyze' fails to detect use of this dependency -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-server-nodemanager</artifactId>
<scope>test</scope>
</dependency>
<!-- 'mvn dependency:analyze' fails to detect use of this dependency -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-server-resourcemanager</artifactId>
<scope>test</scope>
</dependency>
<!-- 'mvn dependency:analyze' fails to detect use of this dependency -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-server-tests</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>jar</goal>
</goals>
<!-- strictly speaking, the unit test is really a regression test. It
needs the main jar to be available to be able to run. -->
<phase>test-compile</phase>
</execution>
</executions>
<configuration>
<archive>
<manifest>
<mainClass>org.apache.hadoop.yarn.dmlc.Client</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<environmentVariables>
<JAVA_HOME>${java.home}</JAVA_HOME>
</environmentVariables>
</configuration>
</plugin>
</plugins>
</build>


</project>
Loading