NC | Online Upgrade | Config directory restructure upgrade script

[romayalon]

Explain the changes

1. Added config directory restructure upgrade script based on the following algorithm -

1. Creation of the identities/ directory.
2. Creation of accounts_by_name/ directory.
3. Iterate all accounts under accounts/  directory
  3.1. Creation identity
    3.1.1. Create config_dir/identities/111/ directory
    3.1.2. Create a hard link from accounts/account_name.json to config_dir/identities/111/identity.json
    If failed add to failed_account array and continue the loop
  3.2. Account Name symlink -  Symlink accounts_by_name/account_name.symlink → config_dir/identities/111/identity.json
    If failed add to failed_account array and continue the loop
  3.3. Account access key symlink - 
     3.3.1. link tmp_access_key symlink→ config_dir/identities/111/identity.json
     3.3.2. on GPFS - Linkat() - tmp_access_key -> config_dir/access_keys/access_key.symlink
     3.3.3. On POSIX - Rename() - tmp_access_key -> config_dir/access_keys/access_key.symlink
     If failed add to failed_account array and continue the loop
4. Delete accounts/ to a different directory

Issues: Fixed #xxx / Gap #xxx

1. Automated tests -
   1.1. unit tests
   1.2. test on a GPFS cluster.
   1.3. re-entrancy tests.
   1.4. Scale tests (5000 accounts) and measurements.
2. List accounts in batch is scale tests show bad performance

Testing Instructions:

1. Manual Test -
   1.1. Install NooBaa 5.17 and create accounts, a system, buckets etc.
   1.2. Install NooBaa 5.18, start the noobaa service.
   1.3. Run NooBaa config dir upgrade -

sudo node src/cmd/manage_nsfs.js upgrade start --expected_version 5.18.0 --expected_hosts <host_name>

1.4. Check config dir was upgraded successfully.

• Doc added/updated
• Tests added