Measuring Performance

Starting Funnel Server

funnel server run --config funnel.config.yaml

Funnel Config

funnel.config.json:

Compute: slurm

Slurm:
    Template: |
      #!/bin/bash
      #SBATCH --job-name {{.TaskId}}
      #SBATCH --ntasks 1
      #SBATCH --error {{.WorkDir}}/funnel-stderr
      #SBATCH --output {{.WorkDir}}/funnel-stdout
      {{if ne .Cpus 0 -}}
      {{printf "#SBATCH --cpus-per-task %d" .Cpus}}
      {{- end}}
      {{if ne .RamGb 0.0 -}}
      {{printf "#SBATCH --mem %.0fGB" .RamGb}}
      {{- end}}
      {{if ne .DiskGb 0.0 -}}
      {{printf "#SBATCH --tmp %.0fGB" .DiskGb}}
      {{- end}}

      funnel worker run --taskID {{.TaskId}}

Database: mongodb

MongoDB:
  # Addresses for the seed servers.
  Addrs:
    - "mongodb://localhost"
  # Database name used within MongoDB to store funnel data.
  Database: "funnel"
  Username: "funnel"
  Password: "foobar"

Submitting Tasks

hello-world.json:

{
  "name": "Hello world",
  "description": "Demonstrates the most basic echo task.",
  "executors": [
    {
      "image": "alpine",
      "command": ["echo", "hello world"]
    }
  ],
  "tags": {
    "group": "{sequential,parallel}"
  }
}

Parallel

➜ time ./parallel.sh

...
Submitting job 9998
Submitting job 9999
cnsekfj5hk26ahebe6ag
cnsekfj5hk26ahebe6b0
Submitting job 10000
cnsekfj5hk26ahebe6g0

real    4m2.468s    <--- ~245 seconds
user    6m16.147s
sys     2m24.520s

#!/bin/bash
# parallel.sh

# Generate a sequence of numbers from 0 to 10000
# Use xargs to run the funnel command in parallel
seq 0 10000 |
xargs -n 1 -P 0 -I {} sh -c 'echo "Submitting job {}"; funnel task create hello-world.json'

Sequential

➜ time ./sequential.sh

...
Submitting job 9998: cnsefr35hk26aheb9a6g
Submitting job 9999: cnsefrb5hk26aheb9a70
Submitting job 10000: cnsefrb5hk26aheb9a7g

real    7m56.409s    <--- ~480 seconds
user    4m49.456s
sys     2m15.923s

#!/bin/bash
# sequential.sh

for i in {0..10000}; do
    echo -n "Submitting job $i: "
    funnel task create hello-world.json
done

Listing Tasks

Note: Funnel's default max page count is set to 2048.

➜  time funnel task list --tag group=parallel --page-size 10000
{
  "tasks": [
    <Parallel Tasks>
  ]
}
real    0m0.355s
user    0m0.136s
sys     0m0.037s

➜  time funnel task list --tag group=sequential --page-size 10000
{
  "tasks": [
    <Sequential Tasks>
  ]
}
real    0m0.389s
user    0m0.104s
sys     0m0.061s

Environment

• OS: Ubuntu 22.04.4 LTS
• RAM: 8GB
• CPU: 4
• Host: OpenStack