Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: Refactored jobrunner #67

Merged
merged 5 commits into from
Jul 9, 2024
Merged

refactor: Refactored jobrunner #67

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
21d03ed
refactored jobrunner
ujwalbukka Jul 2, 2024
43d4eb5
removed non relevant file
ujwalbukka Jul 2, 2024
250f638
removed commented code
ujwalbukka Jul 3, 2024
24e3c1e
Updated with review comments
ujwalbukka Jul 8, 2024
edc1221
Merge remote-tracking branch 'origin' into refactor-job-runner
ujwalbukka Jul 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
732 changes: 84 additions & 648 deletions API.md
View file
Open in desktop

Large diffs are not rendered by default.

49 changes: 27 additions & 22 deletions docs/public/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ import { Construct } from 'constructs';

export class ControlPlaneStack extends Stack {
public readonly regApiGatewayUrl: string;
public readonly eventBusArn: string;
public readonly eventManager: sbt.EventManager;

constructor(scope: Construct, id: string, props?: any) {
super(scope, id, props);
Expand All @@ -113,7 +113,7 @@ export class ControlPlaneStack extends Stack {
systemAdminEmail: 'ENTER YOUR EMAIL HERE',
});

this.eventBusArn = controlPlane.eventManager.busArn;
this.eventManager = controlPlane.eventManager;
this.regApiGatewayUrl = controlPlane.controlPlaneAPIGatewayUrl;
}
}
Expand Down Expand Up @@ -181,7 +181,7 @@ export class ApplicationPlaneStack extends Stack {
});
new sbt.CoreApplicationPlane(this, 'CoreApplicationPlane', {
eventManager: eventManager,
jobRunnerPropsList: [],
jobRunnersList: [],
});
}
}
Expand All @@ -199,7 +199,7 @@ Although entirely optional, SBT includes a utility that lets you define, and run

![sbt-provisioning.png](../../images/sbt-provisioning.png)

Notice the use of the `provisioning.sh` and `deprovisioning.sh` scripts at the top. These scripts are fed to the `JobRunner` as parameters. Internally the `JobRunner` launches an AWS CodeBuild project, wrapped inside an AWS Step Function, to execute the bash scripts. The `JobRunner` also lets you specify what input variables to feed to the scripts, along with what output variables you expect them to return. Note that in this version of SBT, `JobRunner`s are created by the `CoreAppPlane` based on its `jobRunnerPropsList` input (the empty array in the code above). The type of object here is the [`jobRunnerProps`](/API.md#coreapplicationplanejobrunnerprops-). Let's take a simple example: imagine our SaaS application deployed only a single S3 bucket per tenant. Let's create a job runner for that provisioning now.
Notice the use of the `provisioning.sh` and `deprovisioning.sh` scripts at the top. These scripts are fed to the `JobRunner` as parameters. Internally the `JobRunner` launches an AWS CodeBuild project, wrapped inside an AWS Step Function, to execute the bash scripts. The `JobRunner` also lets you specify what input variables to feed to the scripts, along with what output variables you expect them to return. Note that in this version of SBT, you can create `JobRunner`s with [`jobRunnerProps`](/API.md#bashjobrunnerprops-) and configure `CoreAppPlane` with `JobRunner`s using its `jobRunnersList` property. The `CoreAppPlane` will then link these `JobRunner`s to EventBridge. Let's take a simple example: imagine our SaaS application deployed only a single S3 bucket per tenant. Let's create a job runner for that provisioning now.

```typescript
const provisioningJobRunnerProps = {
Expand All @@ -213,23 +213,25 @@ const provisioningJobRunnerProps = {
},
outgoingEvent: sbt.DetailType.PROVISION_SUCCESS,
incomingEvent: sbt.DetailType.ONBOARDING_REQUEST,
eventManager: eventManager /*See below on how to create EventManager*/,
};
```

##### Bash Job Runner Properties

Let's take a moment and dissect this object.

| Key | Type | Purpose |
| ----------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| **name** | string | The **name** key is just a name for this job. |
| **script** | string | A string in bash script format that represents the job to be run (example below) |
| **permissions** | [PolicyDocument](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_iam.PolicyDocument.html) | An IAM policy document giving this job the IAM permisisons it needs to do what it's being asked to do |
| **environmentStringVariablesFromIncomingEvent** | string[] | The environment variables to import into the BashJobRunner from event details field. |
| **environmentVariablesToOutgoingEvent** | string[] | The environment variables to export into the outgoing event once the BashJobRunner has finished. |
| **scriptEnvironmentVariables** | `{ [key: string]: string }` | The variables to pass into the codebuild BashJobRunner. |
| **outgoingEvent** | any | Outgoing EventBridge wiring details |
| **incomingEvent** | any | Incoming EventBridge wiring details |
| Key | Type | Purpose |
| ----------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| **name** | string | The **name** key is just a name for this job. |
| **script** | string | A string in bash script format that represents the job to be run (example below) |
| **permissions** | [PolicyDocument](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_iam.PolicyDocument.html) | An IAM policy document giving this job the IAM permisisons it needs to do what it's being asked to do |
| **environmentStringVariablesFromIncomingEvent** | string[] | The environment variables to import into the BashJobRunner from event details field. |
| **environmentVariablesToOutgoingEvent** | string[] | The environment variables to export into the outgoing event once the BashJobRunner has finished. |
| **scriptEnvironmentVariables** | `{ [key: string]: string }` | The variables to pass into the codebuild BashJobRunner. |
| **outgoingEvent** | any | Outgoing EventBridge wiring details |
| **incomingEvent** | any | Incoming EventBridge wiring details |
| **eventManager** | [IEventManager](/API.md#ieventmanager-) | The EventManager instance that allows connecting to events flowing between the Control Plane and other components. |

The heavy lifting of the `JobRunner` happens with the value of the `script` key. Recall, that this particular example is for provisioning. Also remember that the "SaaS application" we're illustrating here is only provisioning a new S3 bucket for each tenant. Let's take a look at that example provisioning script now:

Expand Down Expand Up @@ -364,11 +366,12 @@ import { EventBus } from 'aws-cdk-lib/aws-events';
import { PolicyDocument, PolicyStatement, Effect } from 'aws-cdk-lib/aws-iam';

export interface AppPlaneProps extends cdk.StackProps {
eventBusArn: string;
eventManager: sbt.EventManager;
}
export class AppPlaneStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props: AppPlaneProps) {
super(scope, id, props);

const provisioningJobRunnerProps = {
name: 'provisioning',
permissions: new PolicyDocument({
Expand Down Expand Up @@ -428,16 +431,18 @@ echo "done!"
},
outgoingEvent: sbt.DetailType.PROVISION_SUCCESS,
incomingEvent: sbt.DetailType.ONBOARDING_REQUEST,
eventManager: props.eventManager,
};

const eventBus = EventBus.fromEventBusArn(this, 'EventBus', props.eventBusArn);
const eventManager = new sbt.EventManager(this, 'EventManager', {
eventBus: eventBus,
});
const provisioningJobRunner: sbt.BashJobRunner = new sbt.BashJobRunner(
this,
'provisioningJobRunner',
provisioningJobRunnerProps
);

new sbt.CoreApplicationPlane(this, 'CoreApplicationPlane', {
eventManager: eventManager,
jobRunnerPropsList: [provisioningJobRunnerProps],
eventManager: props.eventManager,
jobRunnersList: [provisioningJobRunner],
});
}
}
Expand All @@ -455,7 +460,7 @@ import { AppPlaneStack } from '../lib/app-plane';
const app = new cdk.App();
const controlPlaneStack = new ControlPlaneStack(app, 'ControlPlaneStack');
const appPlaneStack = new AppPlaneStack(app, 'AppPlaneStack', {
eventBusArn: controlPlaneStack.eventBusArn,
eventManager: controlPlaneStack.eventManager,
});
```

Expand Down
26 changes: 26 additions & 0 deletions scripts/test-token-vending-machine.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/bin/bash
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0

STACK_NAME="token-vending-machine-integ"

LAMBDA_FUNCTION_NAME=$(aws cloudformation describe-stacks --stack-name $STACK_NAME --query "Stacks[0].Outputs[?OutputKey=='TVMTestFunctionName'].OutputValue" --output text)

TEMP_FILE=$(mktemp)

aws lambda invoke --function-name $LAMBDA_FUNCTION_NAME --payload "{}" $TEMP_FILE

RESPONSE_VALUE=$(cat $TEMP_FILE | jq -r '.statusCode')

rm $TEMP_FILE

echo "Response code: $RESPONSE_VALUE"

# Check the contents of the response
if [[ "$RESPONSE_VALUE" == "200" ]]; then
echo "Token vending machine tests passed successfully!"
exit 0
else
echo "Token vending machine tests failed!"
exit 1
fi
177 changes: 0 additions & 177 deletions src/core-app-plane/bash-job-orchestrator.ts
View file
Open in desktop

This file was deleted.

Loading
Loading