Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not stat C:/Users/user/OneDrive due to the folder being a reparsepoint #832

Open
deajan opened this issue Sep 30, 2019 · 31 comments
Open

Comments

@deajan
Copy link
Contributor

deajan commented Sep 30, 2019

Hello,

Reporting a problem that affects like almost all backup programs that use Windows VSS.
Newer OneDrive clients allow to show online files in local filesystem without actually downloading them.
When using that functionnality, it breaks backups because the actual onedrive folder becomes a reparsepoint instead of a standard folder, leading to "Could not stat" error message.

There are at least 3 folders that are known to maybe becomre reparsepoints:

C:\Users\user\OneDrive
C:\Users\user\OneDrive - CompanyName
C:\Users\user\CompanyName

The easy solution is to exclude [A-Z]:/Users/[^/]/OneDrive* from backup, but then there's the 3rd folder that isn't predictable.

Since this FS behavior may be more widespread than Microsoft OneDrive, Is there maybe a way to detect reparsepoints in burp in order to automatically exclude them (with a logging message) ?

Under windows, one can identify a reparsepoint with:

fsutil reparsepoint query c:\users\user\MyFolderNotBeingARealOne

Best regards.

Link to MS forum with that problem
https://social.technet.microsoft.com/Forums/en-US/d612c19a-0f18-431a-9d75-9fb5dfd11aca/windows-10-1803-client-backups-failing-media-write-protected-errors

@grke
Copy link
Owner

grke commented Oct 1, 2019

Hello,
I thought burp already supported junction points and reparse points.
Search for FT_REPARSE in burp.

If you are getting 'Could not stat', it means FT_REPARSE is not getting set.
The setting code is in src/client/find.c:

static void windows_reparse_point_fiddling(struct FF_PKT *ff_pkt)
{
        /*
        * We have set st_rdev to 1 if it is a reparse point, otherwise 0,
        *  if st_rdev is 2, it is a mount point.
        */
        /*
         * A reparse point (WIN32_REPARSE_POINT)
         *  is something special like one of the following:
         *  IO_REPARSE_TAG_DFS              0x8000000A
         *  IO_REPARSE_TAG_DFSR             0x80000012
         *  IO_REPARSE_TAG_HSM              0xC0000004
         *  IO_REPARSE_TAG_HSM2             0x80000006
         *  IO_REPARSE_TAG_SIS              0x80000007
         *  IO_REPARSE_TAG_SYMLINK          0xA000000C
         *
         * A junction point is a:
         *  IO_REPARSE_TAG_MOUNT_POINT      0xA0000003
         * which can be either a link to a Volume (WIN32_MOUNT_POINT)
         * or a link to a directory (WIN32_JUNCTION_POINT)
         *
         * Ignore WIN32_REPARSE_POINT and WIN32_JUNCTION_POINT
         */
        if (ff_pkt->statp.st_rdev == WIN32_REPARSE_POINT) {
                ff_pkt->type = FT_REPARSE;
        } else if (ff_pkt->statp.st_rdev == WIN32_JUNCTION_POINT) {
                ff_pkt->type = FT_JUNCTION;
        }
}
#endif

And st_rdev is set in src/win32/compat/compat.cpp:

           Win32 reparse point (junction point) is like a link
           though it can have many properties (directory link,
           soft link, hard link, HSM, ...
           A mount point is a reparse point where another volume
           is mounted, so it is like a Unix mount point (change of
           filesystem).  */
        if(*pdwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)
                sb->st_rdev=WIN32_MOUNT_POINT;
        else
                sb->st_rdev=0;

        if((*pdwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)
          && (*pdwReserved0 & IO_REPARSE_TAG_MOUNT_POINT))
        {
                sb->st_rdev=WIN32_MOUNT_POINT;
                /* Now to find out if the directory is a mount point or
                   a reparse point, we must do a song and a dance.
                   Explicitly open the file to read the reparse point, then
                   call DeviceIoControl to find out if it points to a Volume
                   or to a directory. */

etc...

So, why would these directories not trigger this code?

@deajan
Copy link
Contributor Author

deajan commented Oct 1, 2019

To be honest, I'm really bad at reading C.
I've tried to check on the file attributes flag and came up with this google magic:
https://docs.microsoft.com/en-us/windows/win32/w8cookbook/placeholder-files

From what I've read, those new file types were introduced in Win8 and are identified by a new flag, eg IO_REPARSE_TAG_FILE_PLACEHOLDER.

Since that flag is unknown to Mingw's winnt.h, it's definition is:

#define IO_REPARSE_TAG_FILE_PLACEHOLDER         (0x80000015L)       // winnt

So I think the code should look like this in src/win32/compat/compat.cpp (sorry, bad C probably):

#define IO_REPARSE_TAG_FILE_PLACEHOLDER         (0x80000015L)       // winnt

[...]

 Win32 reparse point (junction point) is like a link
           though it can have many properties (directory link,
           soft link, hard link, HSM, ...
           A mount point is a reparse point where another volume
           is mounted, so it is like a Unix mount point (change of
           filesystem).  */
        if(*pdwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)
                sb->st_rdev=WIN32_MOUNT_POINT;
        else
                sb->st_rdev=0;

        if((*pdwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)
          && ((*pdwReserved0 & IO_REPARSE_TAG_MOUNT_POINT)
           || (*pdwReserved0 & IO_REPARSE_TAG_FILE_PLACEHOLDER)
           )
        {
                sb->st_rdev=WIN32_MOUNT_POINT;
                /* Now to find out if the directory is a mount point or
                   a reparse point, we must do a song and a dance.
                   Explicitly open the file to read the reparse point, then
                   call DeviceIoControl to find out if it points to a Volume
                   or to a directory. */

Hope that's it :)

@grke
Copy link
Owner

grke commented Oct 2, 2019

Cool, I can try something like that.
Maybe you can test an installer once I have built one?

@deajan
Copy link
Contributor Author

deajan commented Oct 2, 2019

Sure :)
I'm currently working on getting burp-cross-tools to use librsync 2.1.0 to fix a memory leak so I have a full build environment ready. I can even build my own binaries if you commit a fix on a branch.

@grke
Copy link
Owner

grke commented Oct 2, 2019

Here:
https://burp.grke.org/downloads/test/burp-win64-installer-2.3.15.exe
I had a go at updating librsync a couple of days ago, but failed.
I have updated pcre and openssl though.

@deajan
Copy link
Contributor Author

deajan commented Oct 2, 2019

Having fun with librsync here: librsync/librsync#154
I managed to get latest git master compiled as static lib, but now I need to wait for a new release.
Once this is done, I'll happily update the librsync patches on the depkgs in order to compile a static version with libtool.

As for the burp installer you gave me, the "cannot stat" error is random for me (depends if the user updated files I guess), so I need to setup a VM in order to make a quick test.
I'll see to make that tomorrow, really want to find out what the problem is.

@deajan
Copy link
Contributor Author

deajan commented Oct 3, 2019

I've tried to reproduce the error with a machine I experienced the error previously.
Couldn't get it "to show".
I'll try my test VM tomorrow (not myself a onedrive user)...

@grke
Copy link
Owner

grke commented Oct 4, 2019

Sounds good so far.

@deajan
Copy link
Contributor Author

deajan commented Nov 13, 2019

Well f***.
Didn't have that error in a while (and stopped checking too after a week or so), but since a couple of days, using Burp 2.3.16, I noticed the same problem again,eg:

WARNING: Could not stat C:/Users/<user>/OneDrive - <org> FRANCE: Unknown error

Is there any (easy) C code I can compile to dump the *pdwFileAttributes of the directory ?

@deajan
Copy link
Contributor Author

deajan commented Nov 13, 2019

Well, got some easier way to get the attributes of that directory using fsutil:

C:\Users\<user>>fsutil reparsepoint query "c:\Users\<user>\OneDrive - <org> FRANCE"
Valeur de la balise d’analyse : 0x9000701a
Valeur de balise : Microsoft
Valeur de balise : répertoire

Analyser la longueur des données : 0x00000064
Données d’analyse :
0000:  01 00 64 00 46 65 52 70  bf 73 95 b5 60 00 00 00  ..d.FeRp.s..`...
0010:  02 00 09 00 07 00 01 00  58 00 00 00 0a 00 04 00  ........X.......
0020:  5c 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  \...............
0030:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0040:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0050:  00 00 00 00 00 00 00 00  00 00 00 00 01 00 00 00  ................
0060:  76 00 00 00                                       v...

"Valeur de la balise d'analyse" translates to "Reparse Tag Value", so I guessed that's the attribute for that reparse point.
Searching the value 0x9000701a one on google gave me an interresting winnt.h file that comes with current mingw64 in https://github.com/mirror/mingw-w64/blob/master/mingw-w64-tools/widl/include/winnt.h (line 2676):

There are listed all types of reparse point, mine being identified as "cloud 7", whatever this means.

#define IO_REPARSE_TAG_MOUNT_POINT      __MSABI_LONG(0xA0000003)
#define IO_REPARSE_TAG_HSM              __MSABI_LONG(0xC0000004)
#define IO_REPARSE_TAG_DRIVE_EXTENDER   __MSABI_LONG(0x80000005)
#define IO_REPARSE_TAG_HSM2             __MSABI_LONG(0x80000006)
#define IO_REPARSE_TAG_SIS              __MSABI_LONG(0x80000007)
#define IO_REPARSE_TAG_WIM              __MSABI_LONG(0x80000008)
#define IO_REPARSE_TAG_CSV              __MSABI_LONG(0x80000009)
#define IO_REPARSE_TAG_DFS              __MSABI_LONG(0x8000000A)
#define IO_REPARSE_TAG_FILTER_MANAGER   __MSABI_LONG(0x8000000B)
#define IO_REPARSE_TAG_SYMLINK          __MSABI_LONG(0xA000000C)
#define IO_REPARSE_TAG_IIS_CACHE        __MSABI_LONG(0xA0000010)
#define IO_REPARSE_TAG_DFSR             __MSABI_LONG(0x80000012)
#define IO_REPARSE_TAG_DEDUP            __MSABI_LONG(0x80000013)
#define IO_REPARSE_TAG_NFS              __MSABI_LONG(0x80000014)
#define IO_REPARSE_TAG_FILE_PLACEHOLDER __MSABI_LONG(0x80000015)
#define IO_REPARSE_TAG_WOF              __MSABI_LONG(0x80000017)
#define IO_REPARSE_TAG_WCI              __MSABI_LONG(0x80000018)
#define IO_REPARSE_TAG_WCI_1            __MSABI_LONG(0x90001018)
#define IO_REPARSE_TAG_GLOBAL_REPARSE   __MSABI_LONG(0xA0000019)
#define IO_REPARSE_TAG_CLOUD            __MSABI_LONG(0x9000001A)
#define IO_REPARSE_TAG_CLOUD_1          __MSABI_LONG(0x9000101A)
#define IO_REPARSE_TAG_CLOUD_2          __MSABI_LONG(0x9000201A)
#define IO_REPARSE_TAG_CLOUD_3          __MSABI_LONG(0x9000301A)
#define IO_REPARSE_TAG_CLOUD_4          __MSABI_LONG(0x9000401A)
#define IO_REPARSE_TAG_CLOUD_5          __MSABI_LONG(0x9000501A)
#define IO_REPARSE_TAG_CLOUD_6          __MSABI_LONG(0x9000601A)
#define IO_REPARSE_TAG_CLOUD_7          __MSABI_LONG(0x9000701A)
#define IO_REPARSE_TAG_CLOUD_8          __MSABI_LONG(0x9000801A)
#define IO_REPARSE_TAG_CLOUD_9          __MSABI_LONG(0x9000901A)
#define IO_REPARSE_TAG_CLOUD_A          __MSABI_LONG(0x9000A01A)
#define IO_REPARSE_TAG_CLOUD_B          __MSABI_LONG(0x9000B01A)
#define IO_REPARSE_TAG_CLOUD_C          __MSABI_LONG(0x9000C01A)
#define IO_REPARSE_TAG_CLOUD_D          __MSABI_LONG(0x9000D01A)
#define IO_REPARSE_TAG_CLOUD_E          __MSABI_LONG(0x9000E01A)
#define IO_REPARSE_TAG_CLOUD_F          __MSABI_LONG(0x9000F01A)
#define IO_REPARSE_TAG_CLOUD_MASK       __MSABI_LONG(0x0000F000)
#define IO_REPARSE_TAG_APPEXECLINK      __MSABI_LONG(0x8000001B)
#define IO_REPARSE_TAG_GVFS             __MSABI_LONG(0x9000001C)
#define IO_REPARSE_TAG_STORAGE_SYNC     __MSABI_LONG(0x8000001E)
#define IO_REPARSE_TAG_WCI_TOMBSTONE    __MSABI_LONG(0xA000001F)
#define IO_REPARSE_TAG_UNHANDLED        __MSABI_LONG(0x80000020)
#define IO_REPARSE_TAG_ONEDRIVE         __MSABI_LONG(0x80000021)
#define IO_REPARSE_TAG_GVFS_TOMBSTONE   __MSABI_LONG(0xA0000022)

My best guess would be that all the above are reparse points that should be excluded from backups.

@grke
Copy link
Owner

grke commented Nov 14, 2019

OK, thanks for the testing, I will have another go at doing this shortly. :)

@grke
Copy link
Owner

grke commented Nov 28, 2019

I have made a change in master, for the next release, to try to make this better. Maybe you can test the release, when it is out?

@deajan
Copy link
Contributor Author

deajan commented Nov 28, 2019

Sure !
I'll make a before upgrade backup followed by an after upgrade backup and will then compare file manifests in order to make sure everything stays as it should (except for the reparse points of course).

Having read your code, I don't understand at all what it does, although the reparse_or_mount_song_and_dance function is niecely named ;)

@grke
Copy link
Owner

grke commented Dec 29, 2019

My changes for this have been released for a month, and nobody has complained.

I will assume that all is OK and close, please re-open if you find a problem.

@grke grke closed this as completed Dec 29, 2019
@deajan
Copy link
Contributor Author

deajan commented Jan 3, 2020

Sorry, being late for the party, partly because of holidays.

Version 2.3.18 still has the same problem for OneDrive:

2019-12-31 13:54:48 +0100: burp[20153] Backup started
2019-12-31 13:54:48 +0100: burp[20153] Client version: 2.3.18
2019-12-31 13:54:48 +0100: burp[20153] Protocol: 2
2019-12-31 13:54:48 +0100: burp[20153] Client is Windows
2019-12-31 13:54:48 +0100: burp[20153] WARNING: Client 'myclient.local' version '2.3.18' does not match server version '2.3.14'. An upgrade is recommended.
2019-12-31 13:54:48 +0100: burp[20153] Begin phase1 (file system scan)
2019-12-31 14:01:26 +0100: burp[20153] WARNING: Could not stat C:/Users/someuser/OneDrive - Company Name: Unknown error
2019-12-31 14:01:43 +0100: burp[20153] End phase1 (file system scan)
2019-12-31 14:01:43 +0100: burp[20153] forked champ chooser pid 22195
2019-12-31 14:01:46 +0100: burp[20153] Connected to champ chooser.
2019-12-31 14:01:46 +0100: burp[20153] Phase 2 begin (recv backup data)
2019-12-31 14:35:32 +0100: burp[20153] End backup
2019-12-31 14:35:32 +0100: burp[20153] Backup ending - disconnect from client.
2019-12-31 14:35:32 +0100: burp[20153] Begin phase3 (merge manifests)
2019-12-31 14:37:49 +0100: burp[20153] End phase3 (merge manifests)
2019-12-31 14:37:49 +0100: burp[20153] Begin phase4 (sparse generation)
2019-12-31 14:38:34 +0100: burp[20153] locked: sparse index
2019-12-31 14:38:39 +0100: burp[20153] End phase4 (sparse generation)

Can I provide anything, like executing a small c code to get attributes of that folder somehow ?
Best regards.

@deajan
Copy link
Contributor Author

deajan commented Jan 3, 2020

From what I read, FILE_ATTRIBUTE_REPARSE_POINT is 0x400, so none of the list I suggested above has that bit set and applies to be a reparse point.

Could you by any chance include that list / use a newer winnt.h and check files / directories against it ?

@grke grke reopened this Jan 6, 2020
@grke
Copy link
Owner

grke commented Feb 26, 2020

Hello,
Sorry for taking forever.
I have a windows installer that adds this to the top of the reparse_or_mount_song_and_dance function:

printf("reparse_or_mount_song_and_dance: '%s' '%lu' '%lx'\n",
        file,
        reparse_tag,
        reparse_tag);

Maybe this could produce something useful?

https://burp.grke.org/downloads/test/burp-win64-installer-2.3.23.exe

@deajan
Copy link
Contributor Author

deajan commented Feb 29, 2020

No probs ;)
I've updated, but didn't manage to reproduce the problem yet (it's actually kind of random).
Guess you didn't change anything in the function yet ?
I'll retest this week. Thanks.

@grke
Copy link
Owner

grke commented Mar 1, 2020

No, I literally only added a printf.
I don't even know if it will give anything useful.

@mikaku
Copy link

mikaku commented Feb 10, 2022

Hello,

I've just found this issue and I think it is related to what I'm experiencing.
I'm having problems with fresh installations of Windows 10 and Windows 11 that include the C:/Users/<user>/OneDrive directory:

0000037 2022-02-10 16:09:02 +0100

Last 1000 lines of /mnt/burp/win10-sergi/current/log.gz:

2022-02-10 16:09:02 +0100: burp[850881] Backup started
2022-02-10 16:09:02 +0100: burp[850881] Client version: 2.5.4
2022-02-10 16:09:02 +0100: burp[850881] Protocol: 1
2022-02-10 16:09:02 +0100: burp[850881] Using librsync hash blake2
2022-02-10 16:09:02 +0100: burp[850881] Client is Windows
2022-02-10 16:09:02 +0100: burp[850881] WARNING: Client 'win10-sergi' version '2.5.4' does not match server version '2.4.0'. An upgrade is recommended.
2022-02-10 16:09:02 +0100: burp[850881] Begin phase1 (file system scan)
2022-02-10 16:10:03 +0100: burp[850881] WARNING: Will not descend: file system change not allowed C:/Users/sergi/OneDrive: Unknown error
2022-02-10 16:10:03 +0100: burp[850881] End phase1 (file system scan)
2022-02-10 16:10:03 +0100: burp[850881] Begin phase2 (receive file data)
2022-02-10 16:11:30 +0100: burp[850881] End phase2 (receive file data)
2022-02-10 16:11:30 +0100: burp[850881] Backup ending - disconnect from client.
2022-02-10 16:11:30 +0100: burp[850881] Begin phase3 (merge manifests)
2022-02-10 16:11:31 +0100: burp[850881] End phase3 (merge manifests)
2022-02-10 16:11:31 +0100: burp[850881] Begin phase4 (shuffle files)
2022-02-10 16:11:31 +0100: burp[850881] Previous backup is not a hardlinked_archive
2022-02-10 16:11:31 +0100: burp[850881]  will generate reverse deltas
2022-02-10 16:11:31 +0100: burp[850881] Duplicating current backup.
2022-02-10 16:11:33 +0100: burp[850881] New backup is not a hardlinked_archive
2022-02-10 16:11:33 +0100: burp[850881] Doing the atomic data jiggle...
2022-02-10 16:11:44 +0100: burp[850881] End phase4 (shuffle files)
--------------------------------------------------------------------------------
Start time: 2022-02-10 16:09:01
[...]

Same happens if the user has also a Google Drive, in that case I get the following message:

2022-02-09 11:53:57 +0100: burp[767177] Begin phase1 (file system scan)
2022-02-09 11:54:23 +0100: burp[767177] WARNING: Generate VSS snapshot of drive "g:\" failed.
2022-02-09 11:54:23 +0100: burp[767177] got error: Problem with VSS

2022-02-09 11:54:23 +0100: burp[767177] error in phase 1
2022-02-09 11:54:23 +0100: burp[767177] Backup failed

Such errors prevent a complete backup.
Do you have an idea of how we can solve this issue?
Thanks.

@grke
Copy link
Owner

grke commented Feb 10, 2022

Hello,

I don't know much about OneDrive and Google Drive, but I think they are cloud storage services so that your files are backed up "to the cloud". Do you want to back them up to burp, or do you want burp to skip them?
I'm not sure that it makes sense to back them up again, since these things are already backed up "to the cloud".

If you want burp to skip them, you can just add an exclude line for them.

The "Problem with VSS" error is because burp thinks it is just another local drive that it can do a VSS snapshot on.
There is a 'vss_drives' option that you can use to tell burp which drives are VSS snapshottable that you can try.

For the OneDrive reparse point issue, burp ought to detect it and backup that filesystem entry as a reparse point (or whatever it is supposed to be), so that the backup includes that one filesystem entry. I will see if I can borrow somebody's Windows laptop to debug this myself.

@mikaku
Copy link

mikaku commented Feb 11, 2022

I don't know much about OneDrive and Google Drive, but I think they are cloud storage services so that your files are backed up "to the cloud". Do you want to back them up to burp, or do you want burp to skip them?

Same here, I don't know much about these cloud drives, but it seems that some people uses Google Drive as a main storage for small group of people that want to share documents easily. That's why I'd like to include the contents of Google Drive in the backup. Do you think this is possible?

In the case of OneDrive I saw that it comes by default in fresh installs of Windows 10 and Windows 11, and some people might store something there inadvertently. So I'd also want to include it in the backup.

The "Problem with VSS" error is because burp thinks it is just another local drive that it can do a VSS snapshot on.
There is a 'vss_drives' option that you can use to tell burp which drives are VSS snapshottable that you can try.

Hmm, I don't know how exactly vss_drives=[list of drive letters] option works:

  • The list of drive letters must include a separator? if so, a space? a comma? ...
  • It is possible to do a backup of a drive without doing a VSS snapshot?, if no, what is the point of this option?
  • If I set it with a 0 this means that Burp won't backup anything even on C:?

Anyway, for me the must important issue is OneDrive as it comes by default in all new Windows.

For the OneDrive reparse point issue, burp ought to detect it and backup that filesystem entry as a reparse point (or whatever it is supposed to be), so that the backup includes that one filesystem entry. I will see if I can borrow somebody's Windows laptop to debug this myself.

If you cannot get a Windows of someone, try downloading a Windows 10/11 ISO and install a 30days demo on a VM (set it up with "only" 3GB of memory and 20GB of disk). This should suffice for your testings.

Looking forward to your results. 😃

@grke
Copy link
Owner

grke commented Feb 13, 2022

Hello,

As far as I understand, OneDrive and Google Drive allow Windows users to store
and retrieve remote files, like network file systems.

In general, backup software (like burp), operates on local drives, because
there are multiple issues with trying to back up files over network file
systems.

For example, what happens when the remote service is inaccessible?
Does burp just skip that drive and backup all the local files?
That means that when the server comes back in time for the next backup, all
the remote files will look like new files and will have to be backed up again.
Maybe instead, you can just fail the whole backup, but that means the local
files won't be backed up at all.

So this implies that you ought to split the backup into two, so that you backup
the remote files separately to the local files:
You are better off with running the backup software on the actual host to which
the files are local.

If you don't have access to the actual host, because it is some proprietory
cloud system, then you probably need to sync the files to a local directory
first, then backup that copy to your backup server.

Each cloud service will have it's own different set of apis too.
I'm not going to attempt to try to keep up with them all and add native support
for each OS and cloud service to burp.
Maybe something like "rclone" is interesting to you? It claims to support "over
40 cloud products": https://rclone.org/

In conclusion:
The best thing burp can do in the case of 'OneDrive' is to backup the special
file system node within the C: partition that represents the 'OneDrive' link,
rather than trying to download all of the files out of that link (this is
what issue #832 was originally about, and what we have been trying to address).

In the case of Google Drive, which appears as a separate drive letter, as far
as I can tell, it is a similar issue to the backing up of a "Mapped Network
Drive" (which also doesn't support VSS).

  • The list of drive letters must include a separator? if so, a space? a comma? ...

I think it is a list of drive letters, for example: "CDE"
Did you try it? What happened?

  • It is possible to do a backup of a drive without doing a VSS snapshot?, if no, what is the point of this option?

The point is to make it possible to attempt to backup a drive for which it
is impossible to make a VSS snapshot.
For example, some partition using a file system format that doesn't support
VSS. Or some removable storage, like a CD or a USB stick.
Did you try it? What happened?

  • If I set it with a 0 this means that Burp won't backup anything even on C:?

It means that it won't try to make VSS snapshots of any drives before it tries
to back them up.
Did you try it? What happened?

@mikaku
Copy link

mikaku commented Feb 16, 2022

For example, some partition using a file system format that doesn't support
VSS. Or some removable storage, like a CD or a USB stick.
Did you try it? What happened?

I've included the option vss_drives=[G] and the result was:

2022-02-16 09:49:02 +0100: burp[1258771] Backup started
2022-02-16 09:49:02 +0100: burp[1258771] Client version: 2.5.4
2022-02-16 09:49:02 +0100: burp[1258771] Protocol: 1
2022-02-16 09:49:02 +0100: burp[1258771] Using librsync hash blake2
2022-02-16 09:49:02 +0100: burp[1258771] Client is Windows
2022-02-16 09:49:02 +0100: burp[1258771] WARNING: Client 'win10-sergi' version '2.5.4' does not match server version '2.4.0'. An upgrade is recommended.
2022-02-16 09:49:02 +0100: burp[1258771] Begin phase1 (file system scan)
2022-02-16 09:49:07 +0100: burp[1258771] WARNING: Generate VSS snapshot of drive "g:\" failed.
2022-02-16 09:49:07 +0100: burp[1258771] got error: Problem with VSS

2022-02-16 09:49:07 +0100: burp[1258771] error in phase 1
2022-02-16 09:49:07 +0100: burp[1258771] Backup failed

It means that it won't try to make VSS snapshots of any drives before it tries
to back them up.
Did you try it? What happened?

Then, I've included the option vss_drives=[0] and the result was:

2022-02-16 10:09:01 +0100: burp[1259778] Backup started
2022-02-16 10:09:01 +0100: burp[1259778] Client version: 2.5.4
2022-02-16 10:09:01 +0100: burp[1259778] Protocol: 1
2022-02-16 10:09:01 +0100: burp[1259778] Using librsync hash blake2
2022-02-16 10:09:01 +0100: burp[1259778] Client is Windows
2022-02-16 10:09:01 +0100: burp[1259778] WARNING: Client 'win10-sergi' version '2.5.4' does not match server version '2.4.0'. An upgrade is recommended.
2022-02-16 10:09:01 +0100: burp[1259778] Begin phase1 (file system scan)
2022-02-16 10:09:16 +0100: burp[1259778] WARNING: Could not stat G:/: Unknown error
2022-02-16 10:09:16 +0100: burp[1259778] End phase1 (file system scan)

So, there seems no way to backup the Google Drive (G:).

@grke
Copy link
Owner

grke commented Feb 16, 2022

OK, so as I explained above, trying to backup files hosted remotely isn't going to work well.
IMHO, the best thing you can do is either:
a) rely on the cloud provider's backup capabilities to keep backups within the cloud, or
b) use some tool that downloads them to the local computer and then back up those local files.

@mikaku
Copy link

mikaku commented Feb 17, 2022

b) use some tool that downloads them to the local computer and then back up those local files.

Yes, I could use the option backup_script_pre=[path] to use rsync or any other tool to download the contents of G: before the backup. But I need to make sure that there is enough disk space in the local drive.

Anyway, thanks for your assistance.
I'll keep an eye here to see your results on the OneDrive issue.

@mikaku
Copy link

mikaku commented Feb 17, 2022

Yes, I could use the option backup_script_pre=[path] to use rsync or any other tool to download the contents of G: before the backup. But I need to make sure that there is enough disk space in the local drive.

This leads me to the question: it is possible to add an option on the fly when the client connects to the server without having to modify the burp.conf in the client?

I mean, I'd like to define on the server side which clients will execute a script before the backup, but I don't want to have to configure manually the client because I not always have a remote access to it (because it's behind firewall, is a road warrior, etc.).

@grke
Copy link
Owner

grke commented Feb 17, 2022

This leads me to the question: it is possible to add an option on the fly when the client connects to the server without having to modify the burp.conf in the client?

I mean, I'd like to define on the server side which clients will execute a script before the backup, but I don't want to have to configure manually the client because I not always have a remote access to it (because it's behind firewall, is a road warrior, etc.).

I'm not sure what configuration you want to change on the fly.
If it were me, I'd add the script to all clients, and make the script generic.

@mikaku
Copy link

mikaku commented Feb 18, 2022

I'm not sure what configuration you want to change on the fly.

As I said in my previous posts, executing the rsync tool before the backup could be able to include the Google Drive (G:) into the backup since its contents now would be in C:

Following this reasoning, I'd want to be able to include the option backup_script_pre to the client when it connects to the server, but only to an specific client. This would save me from the situation where I'm no longer have access to the client remotely, so I'm unable to change its burp.conf and include there the new backup_script_pre option.

If it were me, I'd add the script to all clients, and make the script generic.

If I configure the same backup_script_pre option to all clients, they will download the contents of Google Drive to their C: and will be included in the backup multiple times, which seems pretty wasteful to me.

@grke
Copy link
Owner

grke commented Feb 19, 2022

Following this reasoning, I'd want to be able to include the option backup_script_pre to the client when it connects to the server, but only to an specific client. This would save me from the situation where I'm no longer have access to the client remotely, so I'm unable to change its burp.conf and include there the new backup_script_pre option.

OK, but if you don't have access to the client, how are you going to add a new script to the client?

@mikaku
Copy link

mikaku commented Feb 21, 2022

OK, but if you don't have access to the client, how are you going to add a new script to the client?

This is not really the case as all clients eventually will include the same script, but will be from the server side when I'd like to decide which client will run the script.

That's why I asked you if it's possible to inject a new option in the burp.conf of the client when it connects to the server every 20 minutes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants