azcopy copy --recursive --follow-symlink=false ignores symbolic links _silently_, not even a warning

[marc-hb]

Which version of the AzCopy was used?

10.16.2

Which platform are you using? (ex: Windows, Mac, Linux)

Linux

What command did you run?

azcopy copy --recursive

What problem was encountered?

Symbolic links are silently ignored, not even a warning or INFO anywhere (maybe some other files are silently ignored too)

How can we reproduce the problem in the simplest way?

Create some symbolic links, run azcopy copy --recursive

Have you found a mitigation/solution?

No.

cc:

• Broken symlinks barely reported (when using --follow-symlink=true) #1974

[marc-hb]

• I know that my current Azure destination does not support symbolic links.
• I don't want to use --follow-symlink=true to duplicate potentially big files and waste money, break tools that rely on symlinks or any problem caused by this change.
• I want to catch symbolic links accidentally created in the transferred area as well as anything that did not transfer and will cause some breakage.
• So, I don't want azcopy copy to silently ignore symbolic links (or anything that it cannot transfer) and pretend everything was transferred fine when it was not.

I don't mind if this requires a new, explicit --actually-fail-on-failures option that is kept "unsafe by default" for backward compability reasons.

The otherwise very verbose .log files in $HOME/.azcopy/ show that --follow-symlink=false should be named --silently-drop-symlink=true: there is not a single line in any log file about any symlink.

Note this copy issue is different from sync issue #718 (but there is likely some overlap)

Other tools like rsync -a or lftp mirror sftp:// report their failures to transfer symbolic links

[marc-hb]

How symbolic links are managed is not just a "user experience" problem. Not silently ignoring some files is at least a new feature. Some would say that silently ignoring what cannot be transferred is a bug.