Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failover state before initiating restores in pra test #2144

Merged
merged 2 commits into from
Sep 17, 2024
Merged

Failover state before initiating restores in pra test #2144

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
34a47de
Switch to failover during restore in pra CTST test
williamlardier Sep 4, 2024
170d48d
Test object expiration after failback
williamlardier Sep 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion tests/ctst/features/pra.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,20 @@ Feature: PRA operations
And the DATA_ACCESSOR user tries to perform PutObject on "DR" site
Then it "should not" pass Vault authentication

# Switch to failover
When I request the failover state for the DR
Then the DR sink should be in phase "Failover"

# Restore on DR site
When i restore object "obj2-1" for 2 days on "DR" site
When i restore object "obj2-1" for 200000 days on "DR" site
Then object "obj2-1" should "" be "restored" and have the storage class "e2e-cold" on "DR" site
And object "obj2-1" should "" be "transitioned" and have the storage class "e2e-cold" on "Primary" site

# Switch to failback
When I resume operations for the DR
Then the DR sink should be in phase "Running"
Copy link
Contributor

@francoisferrand francoisferrand Sep 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should test that the object is expired (on DR site) during failback (may need to increase the restore duration to ensure we don't succeed thanks to race conditions)

And object "obj2-1" should "" be "transitioned" and have the storage class "e2e-cold" on "DR" site

# Pause / Resume DR
When I pause the DR
Then the DR source should be in phase "Paused"
Expand Down
13 changes: 13 additions & 0 deletions tests/ctst/steps/dr/drctl.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,15 @@ type FailoverConfig = {
sinkZenkoNamespace?: string;
};

type FailbackConfig = {
wait?: boolean;
timeout?: string;
sinkKubeconfigPath?: string;
sinkKubeconfigData?: string;
sinkZenkoInstance?: string;
sinkZenkoNamespace?: string;
};

type UninstallConfig = {
sinkZenkoDrInstance?: string;
sourceZenkoDrInstance?: string;
Expand Down Expand Up @@ -217,6 +226,10 @@ export default class ZenkoDrctl {
return this.runCommand('failover', this.paramToCli(config));
}

async failback(config: FailbackConfig) {
return this.runCommand('failback', this.paramToCli(config));
}

async status(config: StatusConfig) {
return this.runCommand('status', this.paramToCli(config));
}
Expand Down
38 changes: 30 additions & 8 deletions tests/ctst/steps/pra.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,13 +303,14 @@ When('the DATA_ACCESSOR user tries to perform PutObject on {string} site', { tim
this.setResult(await S3.putObject(this.getCommandParameters()));
});

Then('the kafka DR volume exists', { timeout: 60000 }, async function (this: Zenko) {
const volumeTimeout = 60000;
Then('the kafka DR volume exists', { timeout: volumeTimeout + 2000 }, async function (this: Zenko) {
const volumeClaim = await getPVCFromLabel(this, 'kafka_cr', 'end2end-pra-sink-base-queue');
this.logger.debug('kafka volume claim', { volumeClaim });
assert(volumeClaim);
const volume = await this.zenkoDrCtl?.volumeGet({
volumeName: volumeClaim.spec?.volumeName,
timeout: '60s',
timeout: `${volumeTimeout.toString()}ms`,
});
this.logger.debug('kafka volume from drctl', { volume });
assert(volume);
Expand All @@ -320,36 +321,57 @@ Then('the kafka DR volume exists', { timeout: 60000 }, async function (this: Zen
assert(volumeParsed.result!['volume phase'] === 'Bound');
});

When('I pause the DR', { timeout: 360000 }, async function (this: Zenko) {
const failoverTimeout = 360000;
When ('I request the failover state for the DR', { timeout: failoverTimeout + 2000 }, async function (this: Zenko) {
await this.zenkoDrCtl?.failover({
sinkZenkoNamespace: 'default',
wait: true,
timeout: `${failoverTimeout.toString()}ms`,
});
});

const failbackTimeout = 360000;
When ('I resume operations for the DR', { timeout: failbackTimeout + 2000 }, async function (this: Zenko) {
await this.zenkoDrCtl?.failback({
sinkZenkoNamespace: 'default',
wait: true,
timeout: `${failbackTimeout.toString()}ms`,
});
});

const pauseTimeout = 360000;
When('I pause the DR', { timeout: pauseTimeout + 2000 }, async function (this: Zenko) {
await this.zenkoDrCtl?.replicationPause({
sourceZenkoDrInstance: 'end2end-source',
sinkZenkoDrInstance: 'end2end-pra-sink',
sinkZenkoNamespace: 'default',
sourceZenkoNamespace: 'default',
wait: true,
timeout: '6m',
timeout: `${pauseTimeout.toString()}ms`,
});
});

When('I resume the DR', { timeout: 360000 }, async function (this: Zenko) {
const resumeTimeout = 360000;
When('I resume the DR', { timeout: resumeTimeout + 2000 }, async function (this: Zenko) {
await this.zenkoDrCtl?.replicationResume({
sourceZenkoDrInstance: 'end2end-source',
sinkZenkoDrInstance: 'end2end-pra-sink',
sinkZenkoNamespace: 'default',
sourceZenkoNamespace: 'default',
wait: true,
timeout: '6m',
timeout: `${resumeTimeout.toString()}ms`,
});
});

When('I uninstall DR', { timeout: 360000 }, async function (this: Zenko) {
const uninstallTimeout = 360000;
When('I uninstall DR', { timeout: uninstallTimeout + 2000 }, async function (this: Zenko) {
await this.zenkoDrCtl?.uninstall({
sourceZenkoDrInstance: 'end2end-source',
sinkZenkoDrInstance: 'end2end-pra-sink',
sinkZenkoNamespace: 'default',
sourceZenkoNamespace: 'default',
wait: true,
timeout: '6m',
timeout: `${uninstallTimeout.toString()}ms`,
});
});

Expand Down
Loading