Scenarios Not Supported by AzCopy
While AzCopy is a versatile and powerful tool for copying data, there are certain scenarios where it may not function as expected or where its use is not recommended, these limitations can help customers avoid unexpected behavior and ensure smooth operation of their data transfer tasks.
AzCopy does not support running multiple instances simultaneously on the same virtual machine or physical machine. Attempting to execute multiple instances concurrently may lead to conflicts, resource contention, and potentially data corruption. For example, if two AzCopy instances attempt to write and to the same destination blob simultaneously, it can result in data inconsistency and corruption.
Let's delve into a scenario to grasp this concept more clearly. Lets say we're executing a copy operation between two containers while simultaneously, on another instance, conducting a similar copy operation between the different two containers. In this scenario, running multiple instances concurrently can lead to unforeseen and possibly problematic behaviors as can be noted below:
The First copy operations succeeds as can be observed
whereas the second copy operation gets failed
When attempting to copy data from a blob that is actively being written to by another agent, AzCopy may encounter unexpected behavior or fail to complete the transfer successfully.
To illustrate this scenario further, let's consider a situation where we're performing a sync operation between two containers. At the same time, on a separate instance, a removal operation is being executed on the destination container. This simultaneous modification of the destination container while AzCopy is attempting to synchronize data can lead to conflicts, inconsistencies, or even data loss.
The First sync operation succeeds:
Whereas, it can be clearly observed the second remove operation completes with errors resulting in inconsistencies:
AzCopy does not support modifying or editing local files that are being transferred during an ongoing job. Attempting to modify files that are part of an active transfer may result in errors, data corruption, or job failure. For example, if a user attempts to rename or delete a file that AzCopy is currently transferring, it may cause the transfer operation to fail or result in unexpected behavior.
In a scenario where we're copying files from our local system to a blob container, it's important to note that while the file transfer is ongoing, AzCopy does not support renaming or deleting files. Any attempts to rename or delete files during the transfer process will not be supported.
As you can see here the copy operation succeeds
During the transfer process, attempting to rename a file will not be supported. For instance, let's say we try to rename a file from "POC" to "TEST" while the transfer is in progress. Despite our attempt, the "POC" folder will continue to be copied. Similarly, during the copy if we try to delete 5 files notably still in wont be supported as can be observed in the destination container as well as in the copy operation.
