diff --git a/docs/docs/compute/deployment/index.html b/docs/docs/compute/deployment/index.html index 47a09acd2..e3abdb2fe 100644 --- a/docs/docs/compute/deployment/index.html +++ b/docs/docs/compute/deployment/index.html @@ -330,19 +330,16 @@

Usage

Scheduler: # How often to run a scheduler iteration. - # In nanoseconds. - ScheduleRate: 1000000000 # 1 second + ScheduleRate: 1s # How many tasks to schedule in one iteration. ScheduleChunk: 10 # How long to wait between updates before marking a node dead. - # In nanoseconds. - NodePingTimeout: 60000000000 # 1 minute + NodePingTimeout: 1m # How long to wait for a node to start, before marking the node dead. - # In nanoseconds. - NodeInitTimeout: 300000000000 # 5 minutes + NodeInitTimeout: 5m Node: @@ -351,7 +348,7 @@

Usage

# If the node has been idle for longer than the timeout, it will shut down. # -1 means there is no timeout. 0 means timeout immediately after the first task. - Timeout: -1 + Timeout: -1s # A Node will automatically try to detect what resources are available to it. # Defining Resources in the Node configuration overrides this behavior. @@ -365,8 +362,7 @@

Usage

# For low-level tuning. # How often to sync with the Funnel server. - # In nanoseconds. - UpdateRate: 5000000000 # 5 seconds + UpdateRate: 5s Logger: # Logging levels: debug, info, error @@ -375,10 +371,6 @@

Usage

OutputFile: "" -

Known issues

- -

The config uses nanoseconds for duration values. See issue #342.

- diff --git a/docs/docs/index.xml b/docs/docs/index.xml index f918cae6a..071dc7b2e 100644 --- a/docs/docs/index.xml +++ b/docs/docs/index.xml @@ -173,7 +173,7 @@ Compute: htcondor HTCondor: |universe = vanilla getenv = True executable = {{.Ex https://ohsu-comp-bio.github.io/funnel/docs/storage/http/ HTTP(S) Funnel supports downloading files from public URLs via GET reqests. No authentication mechanism is allowed. This backend can be used to fetch objects from cloud storage providers exposed using presigned URLs. The HTTP storage client is enabled by default, but may be explicitly disabled in the worker config: -HTTPStorage: Disabled: false # Timeout for http(s) GET requests. # In nanoseconds. Timeout: 60000000000 # 60 seconds Example task { "name": "Hello world", "inputs": [{ "url": "http://fakedomain. +HTTPStorage: Disabled: false # Timeout for http(s) GET requests. Timeout: 30s Example task { "name": "Hello world", "inputs": [{ "url": "http://fakedomain. @@ -218,7 +218,7 @@ Database: mongodb MongoDB: # Addresses for the seed servers. Addrs: - " https://ohsu-comp-bio.github.io/funnel/docs/storage/swift/ OpenStack Swift Funnel supports using OpenStack Swift for file storage. The Swift storage client is enabled by default, and will try to automatically load credentials from the environment. Alternatively, you may explicitly set the credentials in the worker config: -Swift: Disabled: false UserName: "" Password: "" AuthURL: "" TenantName: "" TenantID: "" RegionName: "" Example task { "name": "Hello world", "inputs": [{ "url": "swift://funnel-bucket/hello.txt", "path": "/inputs/hello.txt" }], "outputs": [{ "url": "swift://funnel-bucket/output. +Swift: Disabled: false UserName: "" Password: "" AuthURL: "" TenantName: "" TenantID: "" RegionName: "" # 500 MB ChunkSizeBytes: 500000000 Example task { "name": "Hello world", "inputs": [{ "url": "swift://funnel-bucket/hello. diff --git a/docs/docs/storage/http/index.html b/docs/docs/storage/http/index.html index a6b36965d..4daf9307a 100644 --- a/docs/docs/storage/http/index.html +++ b/docs/docs/storage/http/index.html @@ -311,8 +311,7 @@

HTTP(S)

HTTPStorage:
   Disabled: false
   # Timeout for http(s) GET requests.
-  # In nanoseconds.
-  Timeout: 60000000000 # 60 seconds
+  Timeout: 30s

Example task

diff --git a/docs/docs/storage/swift/index.html b/docs/docs/storage/swift/index.html index 9dbf74c42..95450956e 100644 --- a/docs/docs/storage/swift/index.html +++ b/docs/docs/storage/swift/index.html @@ -315,6 +315,8 @@

OpenStack Swift

TenantName: "" TenantID: "" RegionName: "" + # 500 MB + ChunkSizeBytes: 500000000

Example task

diff --git a/docs/download/index.html b/docs/download/index.html index f75fefb0e..40aee9fe0 100644 --- a/docs/download/index.html +++ b/docs/download/index.html @@ -302,8 +302,8 @@

Download

@@ -324,7 +324,7 @@

Build the lastest development version optional

$ go get github.com/ohsu-comp-bio/funnel
-

Funnel requires Go 1.8+. Check out the development docs for more detail.

+

Funnel requires Go 1.10+. Check out the development docs for more detail.

Release History

diff --git a/docs/index.xml b/docs/index.xml index c7930097d..31e5df108 100644 --- a/docs/index.xml +++ b/docs/index.xml @@ -86,11 +86,11 @@ A node is a service which runs on each machine in a cluster. The node connects t Mon, 01 Jan 0001 00:00:00 +0000 https://ohsu-comp-bio.github.io/funnel/download/ - Download linux [funnel-linux-amd64-0.6.0.tar.gz] mac [funnel-darwin-amd64-0.6.0.tar.gz] Windows is not supported (yet), sorry! Funnel is a single binary. + Download linux [funnel-linux-amd64-0.7.0.tar.gz] mac [funnel-darwin-amd64-0.7.0.tar.gz] Windows is not supported (yet), sorry! Funnel is a single binary. Funnel requires Docker. Funnel is beta quality. APIs might break, bugs exist, data might be lost. Homebrew brew tap ohsu-comp-bio/formula brew install funnel Build the lastest development version optional In order to build the latest code, run: -$ go get github.com/ohsu-comp-bio/funnel Funnel requires Go 1.8+. Check out the development docs for more detail. +$ go get github.com/ohsu-comp-bio/funnel Funnel requires Go 1.10+. Check out the development docs for more detail. @@ -186,7 +186,7 @@ Compute: htcondor HTCondor: |universe = vanilla getenv = True executable = {{.Ex https://ohsu-comp-bio.github.io/funnel/docs/storage/http/ HTTP(S) Funnel supports downloading files from public URLs via GET reqests. No authentication mechanism is allowed. This backend can be used to fetch objects from cloud storage providers exposed using presigned URLs. The HTTP storage client is enabled by default, but may be explicitly disabled in the worker config: -HTTPStorage: Disabled: false # Timeout for http(s) GET requests. # In nanoseconds. Timeout: 60000000000 # 60 seconds Example task { "name": "Hello world", "inputs": [{ "url": "http://fakedomain. +HTTPStorage: Disabled: false # Timeout for http(s) GET requests. Timeout: 30s Example task { "name": "Hello world", "inputs": [{ "url": "http://fakedomain. @@ -231,7 +231,7 @@ Database: mongodb MongoDB: # Addresses for the seed servers. Addrs: - " https://ohsu-comp-bio.github.io/funnel/docs/storage/swift/ OpenStack Swift Funnel supports using OpenStack Swift for file storage. The Swift storage client is enabled by default, and will try to automatically load credentials from the environment. Alternatively, you may explicitly set the credentials in the worker config: -Swift: Disabled: false UserName: "" Password: "" AuthURL: "" TenantName: "" TenantID: "" RegionName: "" Example task { "name": "Hello world", "inputs": [{ "url": "swift://funnel-bucket/hello.txt", "path": "/inputs/hello.txt" }], "outputs": [{ "url": "swift://funnel-bucket/output. +Swift: Disabled: false UserName: "" Password: "" AuthURL: "" TenantName: "" TenantID: "" RegionName: "" # 500 MB ChunkSizeBytes: 500000000 Example task { "name": "Hello world", "inputs": [{ "url": "swift://funnel-bucket/hello. diff --git a/util/github-release-notes/main.go b/util/github-release-notes/main.go index 4799b2377..81530acc9 100644 --- a/util/github-release-notes/main.go +++ b/util/github-release-notes/main.go @@ -29,7 +29,7 @@ func main() { } // Stop at this PR number - stopAt := 428 + stopAt := 521 includeCommits := false //true // Iterate over all PRs diff --git a/website/content/download.md b/website/content/download.md index bda7c98f3..1fa7318c0 100644 --- a/website/content/download.md +++ b/website/content/download.md @@ -7,12 +7,12 @@ menu: ### Download -- [linux [funnel-linux-amd64-0.6.0.tar.gz]][linux-64-bin] -- [mac [funnel-darwin-amd64-0.6.0.tar.gz]][mac-64-bin] +- [linux [funnel-linux-amd64-0.7.0.tar.gz]][linux-64-bin] +- [mac [funnel-darwin-amd64-0.7.0.tar.gz]][mac-64-bin] - Windows is not supported (yet), sorry! -[linux-64-bin]: https://github.com/ohsu-comp-bio/funnel/releases/download/0.6.0/funnel-linux-amd64-0.6.0.tar.gz -[mac-64-bin]: https://github.com/ohsu-comp-bio/funnel/releases/download/0.6.0/funnel-darwin-amd64-0.6.0.tar.gz +[linux-64-bin]: https://github.com/ohsu-comp-bio/funnel/releases/download/0.7.0/funnel-linux-amd64-0.7.0.tar.gz +[mac-64-bin]: https://github.com/ohsu-comp-bio/funnel/releases/download/0.7.0/funnel-darwin-amd64-0.7.0.tar.gz Funnel is a single binary. Funnel requires [Docker][docker]. @@ -32,7 +32,7 @@ In order to build the latest code, run: $ go get github.com/ohsu-comp-bio/funnel ``` -Funnel requires Go 1.8+. Check out the [development docs][dev] for more detail. +Funnel requires Go 1.10+. Check out the [development docs][dev] for more detail. ### Release History