Thermorawfileparser must not use hard links on AWS!

[jpfeuffer]

See stagein directive in the module

[hendrikweisser]

I've had a problem with the raw->mzML conversion on our server (Ubuntu 20.04) that may be related. The ThermoRawFileParser step failed with the following error message:

Command wrapper:
  ln: failed to create hard link 'test.raw' => '/mnt/MS_Data/Jack/Proteomics/SSTs_for_Hendrik/RAW_files/test.raw': Invalid cross-device link

The raw files were stored on a separate mounted volume.

[hendrikweisser]

Looking into this more, the process script for ThermoRawFileParser actually has code to try staging files in different ways:
https://github.com/bigbio/quantms/blob/8ff9160827c46bd008ea9f191eb73c13a7057bf7/modules/local/thermorawfileparser/main.nf#L11

However, this never takes effect without the errorStrategy 'retry' directive! Adding a line with label 'error_retry' to the top of "thermorawfileparser/main.nf" made it work on my system (with a failure at the first try, but success on the second).

[jpfeuffer]

Yes, one can always overwrite stagein directives. Unfortunately TRFP had problems with symlinks in the past so we default to hardlinks except for AWS now.

[jpfeuffer]

If TRFP proves to be reliable with symlinks we can also revert the change of defaults.

[jpfeuffer]

I mostly have retry strategies enabled anyway.

[hendrikweisser]

I mostly have retry strategies enabled anyway.

How can I do this on the command line?

[hendrikweisser]

The current quantms version (after #28) doesn't fix my problem, because "thermorawfileparser/main.nf" still isn't configured to retry on failure.
Shall I open a separate ticket?

[jpfeuffer]

Yes. The default config https://github.com/nf-core/quantms/blob/dev/conf/base.config#L18
only retries at certain exit codes. On AWS they seem to have triggered. In the new linked PR I now labelled TRFP for unconditional retry https://github.com/nf-core/quantms/blob/dev/conf/base.config#L58

I would use a config file if you want to overwrite the settings of a process:
https://nf-co.re/docs/usage/configuration#custom-configuration-files

[hendrikweisser]

Great, thanks Julianus!
(Did the "Yes" refer to "Shall I open a separate ticket"?)

[jpfeuffer]

Ah no, can you just try again? I submitted and merged a PR for the retries in this step.

[hendrikweisser]

Yep, now that the second PR is merged, it works. Thanks again!