Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix taskcluster train scripts #895

Merged
merged 1 commit into from
Oct 22, 2024

Fix taskcluster train scripts

2d0fb8c
Select commit
Loading
Failed to load commit list.
Merged

Fix taskcluster train scripts #895

Fix taskcluster train scripts
2d0fb8c
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / score-ru-en succeeded Oct 22, 2024 in 49m 47s

FirefoxCI (pull_request)

score for ru-en

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2024-10-22T18:48:12.844Z
Resolved: 2024-10-22T18:49:55.975Z
Task Execution Time: 1 minute, 43 seconds, 131 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0

Artifacts

- public/build/scores.txt
- public/build/scores.txt.log
- public/logs/live_backing.log
- public/logs/live.log


[taskcluster 2024-10-22T18:48:12.957Z] Worker Type (translations-1/b-linux-v100-gpu-4-300gb) settings:
[taskcluster 2024-10-22T18:48:12.957Z]   {
[taskcluster 2024-10-22T18:48:12.957Z]     "config": {
[taskcluster 2024-10-22T18:48:12.957Z]       "deploymentId": ""
[taskcluster 2024-10-22T18:48:12.957Z]     },
[taskcluster 2024-10-22T18:48:12.957Z]     "generic-worker": {
[taskcluster 2024-10-22T18:48:12.957Z]       "engine": "insecure",
[taskcluster 2024-10-22T18:48:12.957Z]       "go-arch": "amd64",
[taskcluster 2024-10-22T18:48:12.957Z]       "go-os": "linux",
[taskcluster 2024-10-22T18:48:12.957Z]       "go-version": "go1.22.2",
[taskcluster 2024-10-22T18:48:12.957Z]       "release": "https://github.com/taskcluster/taskcluster/releases/tag/v64.2.6",
[taskcluster 2024-10-22T18:48:12.957Z]       "revision": "edab196d7d030a5d625b77335109cd9060ab7e1f",
[taskcluster 2024-10-22T18:48:12.957Z]       "source": "https://github.com/taskcluster/taskcluster/commits/edab196d7d030a5d625b77335109cd9060ab7e1f",
[taskcluster 2024-10-22T18:48:12.957Z]       "version": "64.2.6"
[taskcluster 2024-10-22T18:48:12.957Z]     },
[taskcluster 2024-10-22T18:48:12.957Z]     "image": "projects/taskcluster-imaging/global/images/gw-translations-gcp-googlecompute-2024-04-22t18-22-42z",
[taskcluster 2024-10-22T18:48:12.957Z]     "instance-id": "6053279850141731909",
[taskcluster 2024-10-22T18:48:12.957Z]     "instance-type": "projects/887720501152/machineTypes/custom-40-262144",
[taskcluster 2024-10-22T18:48:12.957Z]     "local-ipv4": "10.164.0.82",
[taskcluster 2024-10-22T18:48:12.957Z]     "project-id": "fxci-production-level1-workers",

...(117 lines hidden)...

[task 2024-10-22T18:49:39.282Z] fetches/vocab.spm
[task 2024-10-22T18:49:39.282Z] fetches/marian-scorer
[task 2024-10-22T18:49:39.282Z] fetches/marian-conv
[task 2024-10-22T18:49:39.282Z] fetches/corpus.ru.zst
[task 2024-10-22T18:49:39.282Z] fetches/final.model.npz.best-chrf.npz
[task 2024-10-22T18:49:39.282Z] fetches/spm_export_vocab
[task 2024-10-22T18:49:39.282Z] fetches/marian
[task 2024-10-22T18:49:39.284Z] + set -euo pipefail
[task 2024-10-22T18:49:39.284Z] + echo '###### Scoring'
[task 2024-10-22T18:49:39.284Z] ###### Scoring
[task 2024-10-22T18:49:39.284Z] + test -v MARIAN
[task 2024-10-22T18:49:39.284Z] + test -v GPUS
[task 2024-10-22T18:49:39.284Z] + test -v SRC
[task 2024-10-22T18:49:39.284Z] + test -v TRG
[task 2024-10-22T18:49:39.284Z] + test -v WORKSPACE
[task 2024-10-22T18:49:39.284Z] + model=/home/ubuntu/tasks/task_172962289270264/fetches/final.model.npz.best-chrf.npz
[task 2024-10-22T18:49:39.284Z] + vocab=/home/ubuntu/tasks/task_172962289270264/fetches/vocab.spm
[task 2024-10-22T18:49:39.284Z] + corpus_prefix=/home/ubuntu/tasks/task_172962289270264/fetches/corpus
[task 2024-10-22T18:49:39.284Z] + output=/home/ubuntu/tasks/task_172962289270264/artifacts/scores.txt
[task 2024-10-22T18:49:39.284Z] + zstdmt --rm -d /home/ubuntu/tasks/task_172962289270264/fetches/corpus.ru.zst
[task 2024-10-22T18:49:39.328Z] 
[task 2024-10-22T18:49:39.329Z] etches/corpus.ru.zst : 0 MB...     
[task 2024-10-22T18:49:39.329Z]                                                                                
[task 2024-10-22T18:49:39.329Z] /home/ubuntu/tasks/task_172962289270264/fetches/corpus.ru.zst: 331233 bytes 
[task 2024-10-22T18:49:39.330Z] + zstdmt --rm -d /home/ubuntu/tasks/task_172962289270264/fetches/corpus.en.zst
[task 2024-10-22T18:49:39.332Z] 
[task 2024-10-22T18:49:39.333Z] etches/corpus.en.zst : 0 MB...     
[task 2024-10-22T18:49:39.333Z]                                                                                
[task 2024-10-22T18:49:39.333Z] /home/ubuntu/tasks/task_172962289270264/fetches/corpus.en.zst: 389255 bytes 
[task 2024-10-22T18:49:39.333Z] ++ dirname /home/ubuntu/tasks/task_172962289270264/artifacts/scores.txt
[task 2024-10-22T18:49:39.335Z] + dir=/home/ubuntu/tasks/task_172962289270264/artifacts
[task 2024-10-22T18:49:39.335Z] + mkdir -p /home/ubuntu/tasks/task_172962289270264/artifacts
[task 2024-10-22T18:49:39.337Z] + /home/ubuntu/tasks/task_172962289270264/fetches/marian-scorer --model /home/ubuntu/tasks/task_172962289270264/fetches/final.model.npz.best-chrf.npz --vocabs /home/ubuntu/tasks/task_172962289270264/fetches/vocab.spm /home/ubuntu/tasks/task_172962289270264/fetches/vocab.spm --train-sets /home/ubuntu/tasks/task_172962289270264/fetches/corpus.en /home/ubuntu/tasks/task_172962289270264/fetches/corpus.ru --mini-batch 32 --mini-batch-words 1500 --maxi-batch 1000 --max-length 250 --max-length-crop --normalize --devices 0 1 2 3 --workspace 12000 --log /home/ubuntu/tasks/task_172962289270264/artifacts/scores.txt.log
[task 2024-10-22T18:49:43.018Z] [2024-10-22 18:49:43] [marian] Marian v1.12.14 2d067af 2024-02-16 11:44:13 -0500
[task 2024-10-22T18:49:43.018Z] [2024-10-22 18:49:43] [marian] Running on translations-1-b-linux-v100-gpu-4-300g-jvxw9tqets-bfnngth-ehq as process 4137 with command line:
[task 2024-10-22T18:49:43.018Z] [2024-10-22 18:49:43] [marian] /home/ubuntu/tasks/task_172962289270264/fetches/marian-scorer --model /home/ubuntu/tasks/task_172962289270264/fetches/final.model.npz.best-chrf.npz --vocabs /home/ubuntu/tasks/task_172962289270264/fetches/vocab.spm /home/ubuntu/tasks/task_172962289270264/fetches/vocab.spm --train-sets /home/ubuntu/tasks/task_172962289270264/fetches/corpus.en /home/ubuntu/tasks/task_172962289270264/fetches/corpus.ru --mini-batch 32 --mini-batch-words 1500 --maxi-batch 1000 --max-length 250 --max-length-crop --normalize --devices 0 1 2 3 --workspace 12000 --log /home/ubuntu/tasks/task_172962289270264/artifacts/scores.txt.log
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] alignment: ""
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] authors: false
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] bert-class-symbol: "[CLS]"
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] bert-mask-symbol: "[MASK]"
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] bert-masking-fraction: 0.15
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] bert-sep-symbol: "[SEP]"
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] bert-train-type-embeddings: true
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] bert-type-vocab-size: 2
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] best-deep: false
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] build-info: ""
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] check-nan: false
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] cite: false
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] comet-augment-bad: 0
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] comet-dropout: 0.1
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] comet-final-sigmoid: false
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] comet-mix: false
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] comet-mix-norm: false
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] comet-mixup: 0
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] comet-mixup-reg: false
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config] comet-pooler-ffn:
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config]   - 2048
[task 2024-10-22T18:49:43.019Z] [2024-10-22 18:49:43] [config]   - 1024
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] comet-prepend-zero: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] comet-stop-grad: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] cpu-threads: 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] data-threads: 8
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] dec-cell: gru
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] dec-cell-base-depth: 2
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] dec-cell-high-depth: 1
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] dec-depth: 1
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] devices:
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - 1
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - 2
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - 3
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] dim-emb: 512
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] dim-rnn: 1024
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] dim-vocabs:
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - 1000
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - 1000
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] dump-config: ""
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] enc-cell: gru
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] enc-cell-depth: 1
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] enc-depth: 1
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] enc-type: bidirectional
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] factors-combine: sum
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] factors-dim-emb: 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] gemm-type: float32
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] ignore-model-config: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] input-types:
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   []
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] interpolate-env-vars: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] layer-normalization: True
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] lemma-dependency: ""
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] lemma-dim-emb: 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] log: /home/ubuntu/tasks/task_172962289270264/artifacts/scores.txt.log
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] log-level: info
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] log-time-zone: ""
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] max-length: 250
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] max-length-crop: true
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] maxi-batch: 1000
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] maxi-batch-sort: trg
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] mini-batch: 32
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] mini-batch-words: 1500
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] model: /home/ubuntu/tasks/task_172962289270264/fetches/final.model.npz.best-chrf.npz
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] n-best: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] n-best-feature: Score
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] no-reload: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] normalize: true
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] optimize: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] output: stdout
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] output-omit-bias: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] precision:
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - float32
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] quantize-range: 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] quiet: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] quiet-translation: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] relative-paths: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] right-left: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] seed: 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] skip: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] summary: ""
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] tied-embeddings: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] tied-embeddings-all: True
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] tied-embeddings-src: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] train-sets:
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - /home/ubuntu/tasks/task_172962289270264/fetches/corpus.en
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - /home/ubuntu/tasks/task_172962289270264/fetches/corpus.ru
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-aan-activation: swish
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-aan-depth: 2
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-aan-nogate: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-decoder-autoreg: self-attention
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-decoder-dim-ffn: 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-decoder-ffn-depth: 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-depth-scaling: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-dim-aan: 2048
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-dim-ffn: 2048
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-ffn-activation: swish
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-ffn-depth: 2
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-guided-alignment-layer: last
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-heads: 8
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-no-affine: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-no-bias: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-no-projection: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-pool: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-postprocess: dan
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-postprocess-emb: d
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-postprocess-top: ""
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-preprocess: ""
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-rnn-projection: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-tied-layers:
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   []
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] transformer-train-position-embeddings: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] tsv: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] tsv-fields: 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] type: s2s
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] ulr: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] ulr-dim-emb: 0
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] ulr-trainable-transformation: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] version: v1.12.14 2d067af 2024-02-16 11:44:13 -0500
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] vocabs:
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - /home/ubuntu/tasks/task_172962289270264/fetches/vocab.spm
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config]   - /home/ubuntu/tasks/task_172962289270264/fetches/vocab.spm
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] word-scores: false
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] workspace: 12000
[task 2024-10-22T18:49:43.020Z] [2024-10-22 18:49:43] [config] Loaded model has been created with Marian v1.12.14 2d067af 2024-02-16 11:44:13 -0500
[task 2024-10-22T18:49:43.021Z] [2024-10-22 18:49:43] [data] Loading SentencePiece vocabulary from file /home/ubuntu/tasks/task_172962289270264/fetches/vocab.spm
[task 2024-10-22T18:49:43.024Z] [2024-10-22 18:49:43] [data] Setting vocabulary size for input 0 to 1,000
[task 2024-10-22T18:49:43.024Z] [2024-10-22 18:49:43] [data] Loading SentencePiece vocabulary from file /home/ubuntu/tasks/task_172962289270264/fetches/vocab.spm
[task 2024-10-22T18:49:43.026Z] [2024-10-22 18:49:43] [data] Setting vocabulary size for input 1 to 1,000
[task 2024-10-22T18:49:43.816Z] [2024-10-22 18:49:43] [memory] Extending reserved space to 12032 MB (device gpu0)
[task 2024-10-22T18:49:43.911Z] [2024-10-22 18:49:43] [memory] Extending reserved space to 12032 MB (device gpu1)
[task 2024-10-22T18:49:44.014Z] [2024-10-22 18:49:44] [memory] Extending reserved space to 12032 MB (device gpu2)
[task 2024-10-22T18:49:44.118Z] [2024-10-22 18:49:44] [memory] Extending reserved space to 12032 MB (device gpu3)
[task 2024-10-22T18:49:44.124Z] [2024-10-22 18:49:44] Loading model from /home/ubuntu/tasks/task_172962289270264/fetches/final.model.npz.best-chrf.npz
[task 2024-10-22T18:49:44.124Z] [2024-10-22 18:49:44] Loading model from /home/ubuntu/tasks/task_172962289270264/fetches/final.model.npz.best-chrf.npz
[task 2024-10-22T18:49:44.124Z] [2024-10-22 18:49:44] Loading model from /home/ubuntu/tasks/task_172962289270264/fetches/final.model.npz.best-chrf.npz
[task 2024-10-22T18:49:44.124Z] [2024-10-22 18:49:44] Loading model from /home/ubuntu/tasks/task_172962289270264/fetches/final.model.npz.best-chrf.npz
[task 2024-10-22T18:49:44.561Z] [2024-10-22 18:49:44] Scoring
[task 2024-10-22T18:49:44.921Z] [2024-10-22 18:49:44] [logits] Applying loss function for 1 factor(s)
[task 2024-10-22T18:49:44.922Z] [2024-10-22 18:49:44] [memory] Reserving 131 MB, device gpu3
[task 2024-10-22T18:49:44.923Z] [2024-10-22 18:49:44] [memory] Reserving 131 MB, device gpu2
[task 2024-10-22T18:49:44.924Z] [2024-10-22 18:49:44] [memory] Reserving 131 MB, device gpu1
[task 2024-10-22T18:49:44.931Z] [2024-10-22 18:49:44] [memory] Reserving 131 MB, device gpu0
[task 2024-10-22T18:49:45.142Z] [2024-10-22 18:49:45] [gpu] 16-bit TensorCores enabled for float32 matrix operations
[task 2024-10-22T18:49:47.183Z] [2024-10-22 18:49:47] Total normalized log probs 640070 : Total sentences 1692 : Total words 99190
[task 2024-10-22T18:49:47.183Z] [2024-10-22 18:49:47] Sum of normalized log probs is a sum of averages
[task 2024-10-22T18:49:47.217Z] [2024-10-22 18:49:47] Total time: 4.19664s wall
[task 2024-10-22T18:49:53.380Z] + echo '###### Done: Scoring'
[task 2024-10-22T18:49:53.380Z] ###### Done: Scoring
[fetches 2024-10-22T18:49:53.381Z] removing /home/ubuntu/tasks/task_172962289270264/fetches
[fetches 2024-10-22T18:49:54.210Z] finished
[taskcluster 2024-10-22T18:49:54.222Z]    Exit Code: 0
[taskcluster 2024-10-22T18:49:54.222Z]    User Time: 20.459751s
[taskcluster 2024-10-22T18:49:54.222Z]  Kernel Time: 11.790084s
[taskcluster 2024-10-22T18:49:54.222Z]    Wall Time: 1m8.268842484s
[taskcluster 2024-10-22T18:49:54.222Z]       Result: SUCCEEDED
[taskcluster 2024-10-22T18:49:54.222Z] === Task Finished ===
[taskcluster 2024-10-22T18:49:54.222Z] Task Duration: 1m8.270848164s
[taskcluster 2024-10-22T18:49:54.416Z] Uploading artifact public/build/scores.txt from file /home/ubuntu/tasks/task_172962289270264/artifacts/scores.txt with content encoding "gzip", mime type "text/plain; charset=utf-8" and expiry 2025-10-17T18:00:07.867Z
[taskcluster 2024-10-22T18:49:54.429Z] Uploading artifact public/build/scores.txt.log from file /home/ubuntu/tasks/task_172962289270264/artifacts/scores.txt.log with content encoding "gzip", mime type "text/plain" and expiry 2025-10-17T18:00:07.867Z
[taskcluster 2024-10-22T18:49:55.203Z] [mounts] Preserving cache: Moving "/home/ubuntu/tasks/task_172962289270264/checkouts" to "/home/ubuntu/caches/JjKW5Jv1RI6dJyq7iYi8rg"
[taskcluster 2024-10-22T18:49:55.415Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2025-10-17T18:00:07.867Z