[taskcluster 2024-10-22T18:53:02.253Z] Worker Type (translations-1/b-linux-large-gcp-1tb-32-256-d2g) settings:
[taskcluster 2024-10-22T18:53:02.253Z]   {
[taskcluster 2024-10-22T18:53:02.253Z]     "config": {
[taskcluster 2024-10-22T18:53:02.253Z]       "deploymentId": ""
[taskcluster 2024-10-22T18:53:02.253Z]     },
[taskcluster 2024-10-22T18:53:02.253Z]     "generic-worker": {
[taskcluster 2024-10-22T18:53:02.253Z]       "engine": "multiuser",
[taskcluster 2024-10-22T18:53:02.253Z]       "go-arch": "amd64",
[taskcluster 2024-10-22T18:53:02.253Z]       "go-os": "linux",
[taskcluster 2024-10-22T18:53:02.253Z]       "go-version": "go1.22.2",
[taskcluster 2024-10-22T18:53:02.253Z]       "release": "https://github.com/taskcluster/taskcluster/releases/tag/v64.3.0",
[taskcluster 2024-10-22T18:53:02.253Z]       "revision": "b66b6614b9b79077f1bfc78f97afb8c25c0c4b30",
[taskcluster 2024-10-22T18:53:02.253Z]       "source": "https://github.com/taskcluster/taskcluster/commits/b66b6614b9b79077f1bfc78f97afb8c25c0c4b30",
[taskcluster 2024-10-22T18:53:02.253Z]       "version": "64.3.0"
[taskcluster 2024-10-22T18:53:02.253Z]     },
[taskcluster 2024-10-22T18:53:02.253Z]     "image": "projects/taskcluster-imaging/global/images/gw-fxci-gcp-l1-gui-googlecompute-2024-08-22t22-48-09z",
[taskcluster 2024-10-22T18:53:02.253Z]     "instance-id": "4816917499024657134",
[taskcluster 2024-10-22T18:53:02.253Z]     "instance-type": "projects/887720501152/machineTypes/n2-highmem-32",
[taskcluster 2024-10-22T18:53:02.253Z]     "local-ipv4": "10.128.0.184",
[taskcluster 2024-10-22T18:53:02.253Z]     "project-id": "fxci-production-level1-workers",

...(203 lines hidden)...

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JOOnKkkKSwe1Qwikcu3ghg/artifacts/public/build/extract_lex.tar.zst to /builds/worker/fetches/extract_lex.tar.zst

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JOOnKkkKSwe1Qwikcu3ghg/artifacts/public/build/extract_lex.tar.zstattempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/V6AHwFI2SFGzHdERkomBNA/artifacts/public/build/corpus.en.zstDownloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/EBhuRvmXRg-JQpAJoHsgsA/artifacts/public/build/fast-align.tar.zst to /builds/worker/fetches/fast-align.tar.zst

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/V6AHwFI2SFGzHdERkomBNA/artifacts/public/build/corpus.ru.zst

attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VwyvswswQmecIxHim7KG7w/artifacts/public/build/marian.tar.zst to /builds/worker/fetches/marian.tar.zstDownloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/EBhuRvmXRg-JQpAJoHsgsA/artifacts/public/build/fast-align.tar.zst
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DL1419gWTqWep_47yjP9hg/artifacts/public/build/vocab.spm to /builds/worker/fetches/vocab.spm

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DL1419gWTqWep_47yjP9hg/artifacts/public/build/vocab.spmDownloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VwyvswswQmecIxHim7KG7w/artifacts/public/build/marian.tar.zst

https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/EBhuRvmXRg-JQpAJoHsgsA/artifacts/public/build/fast-align.tar.zst resolved to 722752 bytes with sha256 f3b22724daab04267d54ff010187e6014c149ed32da9a2f5d3be9800d3bbdbac in 0.092s
Verified size of https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/EBhuRvmXRg-JQpAJoHsgsA/artifacts/public/build/fast-align.tar.zst
Extracting /builds/worker/fetches/fast-align.tar.zst to /builds/worker/fetches
/builds/worker/fetches/fast-align.tar.zst extracted in 0.009s
Removing /builds/worker/fetches/fast-align.tar.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/V6AHwFI2SFGzHdERkomBNA/artifacts/public/build/corpus.en.zst resolved to 7635 bytes with sha256 1a3af6ee2b6325961b0f22b55f12ca7824c54812cc6004b1ad574272d880ac7a in 0.178s
Verified size of https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/V6AHwFI2SFGzHdERkomBNA/artifacts/public/build/corpus.en.zst
Extracting /builds/worker/fetches/corpus.en.zst to /builds/worker/fetches
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/V6AHwFI2SFGzHdERkomBNA/artifacts/public/build/corpus.ru.zst resolved to 83739 bytes with sha256 b397329e91940d1b6e7c0a4674e2f945ec53995240446d6dcb3e4b186b3ee56e in 0.196s
Verified size of https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/V6AHwFI2SFGzHdERkomBNA/artifacts/public/build/corpus.ru.zst
Extracting /builds/worker/fetches/corpus.ru.zst to /builds/worker/fetches
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DL1419gWTqWep_47yjP9hg/artifacts/public/build/vocab.spm resolved to 255318 bytes with sha256 baddcca4f6c928751cc9eba95633a925bc8f631c5cb578d5003a4becd86c14a4 in 0.201s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JOOnKkkKSwe1Qwikcu3ghg/artifacts/public/build/extract_lex.tar.zst resolved to 620577 bytes with sha256 4ac8d20c1f44c9c6de6482502b9fcf83d6114d3664c34b40690d92711d6a48b9 in 0.212s
Verified size of https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JOOnKkkKSwe1Qwikcu3ghg/artifacts/public/build/extract_lex.tar.zst
Extracting /builds/worker/fetches/extract_lex.tar.zst to /builds/worker/fetches
/builds/worker/fetches/extract_lex.tar.zst extracted in 0.008s
Removing /builds/worker/fetches/extract_lex.tar.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VwyvswswQmecIxHim7KG7w/artifacts/public/build/marian.tar.zst resolved to 684170205 bytes with sha256 4c7d64fbb4d7a1edae9a834c53b4de9ac716972ffb460b9d8439fa68a85f5d68 in 8.747s
Verified size of https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VwyvswswQmecIxHim7KG7w/artifacts/public/build/marian.tar.zst
Extracting /builds/worker/fetches/marian.tar.zst to /builds/worker/fetches
/builds/worker/fetches/marian.tar.zst extracted in 5.849s
Removing /builds/worker/fetches/marian.tar.zst
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 14.688188586999999, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-10-22T18:53:44.386Z] finished fetching artifacts
[task 2024-10-22T18:53:44.387Z] executing ['bash', '-c', 'pip3 install --upgrade pip setuptools && pip3 install -r $VCS_PATH/pipeline/alignments/requirements/alignments.txt && export BIN=$MOZ_FETCHES_DIR && export MARIAN=$MOZ_FETCHES_DIR && export PATH=$PATH:$MOZ_FETCHES_DIR && export PYTHONPATH=$PYTHONPATH:$VCS_PATH && $VCS_PATH/pipeline/alignments/generate-shortlist.sh $MOZ_FETCHES_DIR/corpus $MOZ_FETCHES_DIR/vocab.spm $TASK_WORKDIR/artifacts auto']
[task 2024-10-22T18:53:44.727Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2024-10-22T18:53:44.728Z] Defaulting to user installation because normal site-packages is not writeable
[task 2024-10-22T18:53:44.752Z] Requirement already satisfied: pip in /usr/lib/python3/dist-packages (22.0.2)
[task 2024-10-22T18:53:44.919Z] Collecting pip
[task 2024-10-22T18:53:45.013Z]   Downloading pip-24.2-py3-none-any.whl (1.8 MB)
[task 2024-10-22T18:53:45.154Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 13.1 MB/s eta 0:00:00
[task 2024-10-22T18:53:45.164Z] Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (59.6.0)
[task 2024-10-22T18:53:45.460Z] Collecting setuptools
[task 2024-10-22T18:53:45.473Z]   Downloading setuptools-75.2.0-py3-none-any.whl (1.2 MB)
[task 2024-10-22T18:53:45.530Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 22.8 MB/s eta 0:00:00
[task 2024-10-22T18:53:45.684Z] Installing collected packages: setuptools, pip
[task 2024-10-22T18:53:48.306Z] Successfully installed pip-24.2 setuptools-75.2.0
[task 2024-10-22T18:53:48.676Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2024-10-22T18:53:48.677Z] Defaulting to user installation because normal site-packages is not writeable
[task 2024-10-22T18:53:48.839Z] Collecting certifi==2024.7.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 7))
[task 2024-10-22T18:53:48.877Z]   Downloading certifi-2024.7.4-py3-none-any.whl (162 kB)
[task 2024-10-22T18:53:49.011Z] Collecting charset-normalizer==3.3.2 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 11))
[task 2024-10-22T18:53:49.024Z]   Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
[task 2024-10-22T18:53:49.288Z] Collecting cython==3.0.9 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 103))
[task 2024-10-22T18:53:49.303Z]   Downloading Cython-3.0.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
[task 2024-10-22T18:53:49.342Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 101.3 MB/s eta 0:00:00
[task 2024-10-22T18:53:49.391Z] Collecting eflomal==1.0.0b1 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 163))
[task 2024-10-22T18:53:49.404Z]   Downloading eflomal-1.0.0b1.tar.gz (102 kB)
[task 2024-10-22T18:53:49.419Z]   Installing build dependencies: started
[task 2024-10-22T18:53:57.342Z]   Installing build dependencies: finished with status 'done'
[task 2024-10-22T18:53:57.343Z]   Getting requirements to build wheel: started
[task 2024-10-22T18:53:58.718Z]   Getting requirements to build wheel: finished with status 'done'
[task 2024-10-22T18:53:58.720Z]   Preparing metadata (pyproject.toml): started
[task 2024-10-22T18:53:59.348Z]   Preparing metadata (pyproject.toml): finished with status 'done'
[task 2024-10-22T18:53:59.372Z] Collecting idna==3.8 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 166))
[task 2024-10-22T18:53:59.385Z]   Downloading idna-3.8-py3-none-any.whl (66 kB)
[task 2024-10-22T18:53:59.576Z] Collecting numpy==1.26.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 170))
[task 2024-10-22T18:53:59.589Z]   Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
[task 2024-10-22T18:53:59.657Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 284.7 MB/s eta 0:00:00
[task 2024-10-22T18:53:59.737Z] Collecting opus-fast-mosestokenizer==0.0.8.5 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 208))
[task 2024-10-22T18:53:59.751Z]   Downloading opus_fast_mosestokenizer-0.0.8.5-cp310-cp310-manylinux1_x86_64.whl (837 kB)
[task 2024-10-22T18:53:59.756Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 837.2/837.2 kB 528.1 MB/s eta 0:00:00
[task 2024-10-22T18:53:59.800Z] Collecting requests==2.31.0 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 223))
[task 2024-10-22T18:53:59.816Z]   Downloading requests-2.31.0-py3-none-any.whl (62 kB)
[task 2024-10-22T18:53:59.916Z] Collecting tqdm==4.66.4 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 227))
[task 2024-10-22T18:53:59.929Z]   Downloading tqdm-4.66.4-py3-none-any.whl (78 kB)
[task 2024-10-22T18:53:59.983Z] Collecting urllib3==2.2.2 (from -r /builds/worker/checkouts/vcs/pipeline/alignments/requirements/alignments.txt (line 231))
[task 2024-10-22T18:53:59.996Z]   Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
[task 2024-10-22T18:54:00.037Z] WARNING: The candidate selected for download or install is a yanked version: 'eflomal' candidate (version 1.0.0b1 at https://files.pythonhosted.org/packages/1d/cc/2cdf9c90a6bd8fd7323b47bf5a60fe738bfe8465844a2136b0543f5094e8/eflomal-1.0.0b1.tar.gz (from https://pypi.org/simple/eflomal/))
[task 2024-10-22T18:54:00.037Z] Reason for being yanked: <none given>
[task 2024-10-22T18:54:00.039Z] Building wheels for collected packages: eflomal
[task 2024-10-22T18:54:00.040Z]   Building wheel for eflomal (pyproject.toml): started
[task 2024-10-22T18:54:05.051Z]   Building wheel for eflomal (pyproject.toml): finished with status 'done'
[task 2024-10-22T18:54:05.053Z]   Created wheel for eflomal: filename=eflomal-1.0.0b1-cp310-cp310-linux_x86_64.whl size=319770 sha256=fdfa71f36bb98f4f582b91c25de4e859f08b3e1e0734b3322ece8c35cdad2726
[task 2024-10-22T18:54:05.053Z]   Stored in directory: /tmp/pip-ephem-wheel-cache-1asmw6ay/wheels/b4/0c/55/4a2c6f00793a0d2fdaef4376b8cb3d9c6ae8df16366d6e31d1
[task 2024-10-22T18:54:05.056Z] Successfully built eflomal
[task 2024-10-22T18:54:05.104Z] Installing collected packages: urllib3, tqdm, opus-fast-mosestokenizer, numpy, idna, cython, charset-normalizer, certifi, requests, eflomal
[task 2024-10-22T18:54:07.779Z] Successfully installed certifi-2024.7.4 charset-normalizer-3.3.2 cython-3.0.9 eflomal-1.0.0b1 idna-3.8 numpy-1.26.4 opus-fast-mosestokenizer-0.0.8.5 requests-2.31.0 tqdm-4.66.4 urllib3-2.2.2
[task 2024-10-22T18:54:08.165Z] + set -euo pipefail
[task 2024-10-22T18:54:08.165Z] + echo '###### Generating alignments and shortlist'
[task 2024-10-22T18:54:08.165Z] ###### Generating alignments and shortlist
[task 2024-10-22T18:54:08.165Z] + [[ -z /builds/worker/fetches ]]
[task 2024-10-22T18:54:08.165Z] + [[ -z /builds/worker/fetches ]]
[task 2024-10-22T18:54:08.165Z] + [[ -z ru ]]
[task 2024-10-22T18:54:08.165Z] + [[ -z en ]]
[task 2024-10-22T18:54:08.165Z] + corpus_prefix=/builds/worker/fetches/corpus
[task 2024-10-22T18:54:08.165Z] + vocab_path=/builds/worker/fetches/vocab.spm
[task 2024-10-22T18:54:08.165Z] + output_dir=/builds/worker/artifacts
[task 2024-10-22T18:54:08.165Z] + threads=auto
[task 2024-10-22T18:54:08.165Z] + '[' auto = auto ']'
[task 2024-10-22T18:54:08.165Z] ++ nproc
[task 2024-10-22T18:54:08.166Z] + threads=32
[task 2024-10-22T18:54:08.166Z] ++ dirname /builds/worker/checkouts/vcs/pipeline/alignments/generate-shortlist.sh
[task 2024-10-22T18:54:08.167Z] + cd /builds/worker/checkouts/vcs/pipeline/alignments
[task 2024-10-22T18:54:08.167Z] + mkdir -p /builds/worker/artifacts
[task 2024-10-22T18:54:08.168Z] + dir=/builds/worker/artifacts/tmp_shortlist
[task 2024-10-22T18:54:08.169Z] + mkdir -p /builds/worker/artifacts/tmp_shortlist
[task 2024-10-22T18:54:08.170Z] + corpus_src=/builds/worker/fetches/corpus.ru.zst
[task 2024-10-22T18:54:08.170Z] + corpus_trg=/builds/worker/fetches/corpus.en.zst
[task 2024-10-22T18:54:08.170Z] + echo '### Subword segmentation with SentencePiece'
[task 2024-10-22T18:54:08.170Z] ### Subword segmentation with SentencePiece
[task 2024-10-22T18:54:08.170Z] + zstdmt -dc /builds/worker/fetches/corpus.ru.zst
[task 2024-10-22T18:54:08.171Z] + parallel --no-notice --pipe -k -j 32 --block 50M /builds/worker/fetches/spm_encode --model /builds/worker/fetches/vocab.spm
[task 2024-10-22T18:54:08.591Z] + zstdmt -dc /builds/worker/fetches/corpus.en.zst
[task 2024-10-22T18:54:08.592Z] + parallel --no-notice --pipe -k -j 32 --block 50M /builds/worker/fetches/spm_encode --model /builds/worker/fetches/vocab.spm
[task 2024-10-22T18:54:08.823Z] + python3 align.py --corpus_src=/builds/worker/artifacts/tmp_shortlist/corpus.spm.ru --corpus_trg=/builds/worker/artifacts/tmp_shortlist/corpus.spm.en --output_path=/builds/worker/artifacts/corpus.aln
[task 2024-10-22T18:54:08.879Z] [alignments] Running with arguments: ['align.py', '--corpus_src=/builds/worker/artifacts/tmp_shortlist/corpus.spm.ru', '--corpus_trg=/builds/worker/artifacts/tmp_shortlist/corpus.spm.en', '--output_path=/builds/worker/artifacts/corpus.aln']
[task 2024-10-22T18:54:08.881Z] [alignments] Starting generating alignments.
[task 2024-10-22T18:54:08.995Z] [alignments] Splitting corpus into parts
[task 2024-10-22T18:54:08.999Z] [alignments] Processing part aa
[task 2024-10-22T18:54:08.999Z] [alignments] Calculating alignments...
[task 2024-10-22T18:54:09.034Z] [eflomal] Prepared 1609 sentences for alignment
[task 2024-10-22T18:54:09.034Z] /builds/worker/.local/lib/python3.10/site-packages/eflomal/bin/eflomal -m 3 -s /tmp/tmpxfe6mpbp -t /tmp/tmpdx7w1m0p -n 3 -N 0.2 -1 31 -2 31 -3 125 -f /builds/worker/artifacts/tmp_aln/aln.fwd.aa -r /builds/worker/artifacts/tmp_aln/aln.rev.aa
[task 2024-10-22T18:54:09.045Z] Read texts (1609 sentences): 0.010 s
[task 2024-10-22T18:54:09.045Z] Vocabulary sizes are 492 (source), 23 (target)
[task 2024-10-22T18:54:09.046Z] Created alignment structures: 0.000 s
[task 2024-10-22T18:54:09.046Z] Created alignment structures: 0.000 s
[task 2024-10-22T18:54:09.050Z] Randomized alignment: 0.004 s
[task 2024-10-22T18:54:09.050Z] Aligning with model 1 (31 iterations)
[task 2024-10-22T18:54:09.051Z] Randomized alignment: 0.004 s
[task 2024-10-22T18:54:09.051Z] Aligning with model 1 (31 iterations)
[task 2024-10-22T18:54:09.319Z] Done: 0.269 s
[task 2024-10-22T18:54:09.319Z] Aligning with model 2 (31 iterations)
[task 2024-10-22T18:54:09.336Z] Done: 0.286 s
[task 2024-10-22T18:54:09.336Z] Aligning with model 2 (31 iterations)
[task 2024-10-22T18:54:09.634Z] Done: 0.315 s
[task 2024-10-22T18:54:09.634Z] Aligning with model 3 (125 iterations)
[task 2024-10-22T18:54:09.682Z] Done: 0.345 s
[task 2024-10-22T18:54:09.682Z] Aligning with model 3 (125 iterations)
[task 2024-10-22T18:54:11.032Z] Done: 1.398 s
[task 2024-10-22T18:54:11.072Z] Final argmax iteration: 0.040 s
[task 2024-10-22T18:54:11.072Z] Writing alignments to /builds/worker/artifacts/tmp_aln/aln.fwd.aa for 1609 sentencess
[task 2024-10-22T18:54:11.163Z] Done: 1.481 s
[task 2024-10-22T18:54:11.208Z] Final argmax iteration: 0.046 s
[task 2024-10-22T18:54:11.208Z] Writing alignments to /builds/worker/artifacts/tmp_aln/aln.rev.aa for 1609 sentencess
[task 2024-10-22T18:54:11.212Z] [alignments] Merging alignments: ['/builds/worker/artifacts/tmp_aln/aln.fwd.aa']
[task 2024-10-22T18:54:11.214Z] [alignments] Merging alignments: ['/builds/worker/artifacts/tmp_aln/aln.rev.aa']
[task 2024-10-22T18:54:11.215Z] [alignments] Symmetrizing alignments...
[task 2024-10-22T18:54:11.252Z] [alignments] Finished generating alignments.
[task 2024-10-22T18:54:11.273Z] + echo '### Creating shortlist'
[task 2024-10-22T18:54:11.273Z] ### Creating shortlist
[task 2024-10-22T18:54:11.273Z] + /builds/worker/fetches/extract_lex /builds/worker/artifacts/tmp_shortlist/corpus.spm.en /builds/worker/artifacts/tmp_shortlist/corpus.spm.ru /builds/worker/artifacts/corpus.aln /builds/worker/artifacts/tmp_shortlist/lex.s2t /builds/worker/artifacts/tmp_shortlist/lex.t2s
[task 2024-10-22T18:54:11.275Z] Starting...
[task 2024-10-22T18:54:11.308Z] 0 
[task 2024-10-22T18:54:11.308Z] Finished
[task 2024-10-22T18:54:11.309Z] + '[' -f /builds/worker/artifacts/tmp_shortlist/lex.s2t ']'
[task 2024-10-22T18:54:11.309Z] + zstdmt /builds/worker/artifacts/tmp_shortlist/lex.s2t
[task 2024-10-22T18:54:11.311Z] 
[task 2024-10-22T18:54:11.311Z]                                                                                
[task 2024-10-22T18:54:11.311Z] Read :  0 /  0 MB ==> 31%
[task 2024-10-22T18:54:11.311Z]                                                                                
[task 2024-10-22T18:54:11.311Z] /builds/worker/artifacts/tmp_shortlist/lex.s2t : 31.08%   ( 35961 =>  11176 bytes, /builds/worker/artifacts/tmp_shortlist/lex.s2t.zst) 
[task 2024-10-22T18:54:11.311Z] + rm /builds/worker/artifacts/tmp_shortlist/corpus.spm.en
[task 2024-10-22T18:54:11.312Z] + rm /builds/worker/artifacts/tmp_shortlist/corpus.spm.ru
[task 2024-10-22T18:54:11.314Z] + rm /builds/worker/artifacts/corpus.aln
[task 2024-10-22T18:54:11.315Z] + echo '### Shortlist pruning'
[task 2024-10-22T18:54:11.315Z] ### Shortlist pruning
[task 2024-10-22T18:54:11.315Z] + /builds/worker/fetches/spm_export_vocab --model=/builds/worker/fetches/vocab.spm --output=/builds/worker/artifacts/tmp_shortlist/vocab.txt
[task 2024-10-22T18:54:11.320Z] + zstdmt -dc /builds/worker/artifacts/tmp_shortlist/lex.s2t.zst
[task 2024-10-22T18:54:11.320Z] + grep -v NULL
[task 2024-10-22T18:54:11.320Z] + python3 prune_shortlist.py 100 /builds/worker/artifacts/tmp_shortlist/vocab.txt
[task 2024-10-22T18:54:11.320Z] + zstdmt
[task 2024-10-22T18:54:11.342Z] + echo '### Deleting tmp dir'
[task 2024-10-22T18:54:11.342Z] ### Deleting tmp dir
[task 2024-10-22T18:54:11.342Z] + rm -rf /builds/worker/artifacts/tmp_shortlist
[task 2024-10-22T18:54:11.343Z] + echo '###### Done: Generating alignments and shortlist'
[task 2024-10-22T18:54:11.343Z] ###### Done: Generating alignments and shortlist
[fetches 2024-10-22T18:54:11.343Z] removing /builds/worker/fetches
[fetches 2024-10-22T18:54:11.640Z] finished
+ exit_code=0
+ podman cp taskcontainer:/builds/worker/artifacts artifact0
+ podman rm taskcontainer
c904fa0185e1c04df33ea77fa58ce9abe07ce49641b5966224f34f2fa3932539
+ exit 0
[taskcluster 2024-10-22T18:54:13.344Z]    Exit Code: 0
[taskcluster 2024-10-22T18:54:13.344Z]    User Time: 10.0933s
[taskcluster 2024-10-22T18:54:13.344Z]  Kernel Time: 5.065811s
[taskcluster 2024-10-22T18:54:13.344Z]    Wall Time: 1m0.866647713s
[taskcluster 2024-10-22T18:54:13.344Z]       Result: SUCCEEDED
[taskcluster 2024-10-22T18:54:13.344Z] === Task Finished ===
[taskcluster 2024-10-22T18:54:13.344Z] Task Duration: 1m0.867189551s
[taskcluster 2024-10-22T18:54:13.428Z] Uploading artifact public/build/lex.s2t.pruned.zst from file /task_172962308335456/artifact0/lex.s2t.pruned.zst with content encoding "identity", mime type "application/zstd" and expiry 2025-01-20T18:00:07.981Z
[taskcluster 2024-10-22T18:54:13.557Z] [mounts] Preserving cache: Moving "/task_172962308335456/cache0" to "caches/FA3bxU7wTlCMcslC_TXfeg"
[taskcluster 2024-10-22T18:54:13.599Z] [mounts] Denying task_172962308335456 access to 'caches/FA3bxU7wTlCMcslC_TXfeg'
[taskcluster 2024-10-22T18:54:13.660Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2025-01-20T18:00:07.981Z