Skip to content

Commit

Permalink
Merge pull request #11 from Cognigy/feature/aws
Browse files Browse the repository at this point in the history
added peter nann nodes
  • Loading branch information
alexteusz authored Aug 24, 2020
2 parents 9d5ebc1 + 7d1ebd5 commit 9e6c7d1
Show file tree
Hide file tree
Showing 9 changed files with 851 additions and 16 deletions.
71 changes: 69 additions & 2 deletions extensions/aws/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# AWS Polly Extension
# AWS SDK Calls

This extension facilitates calls to a number of relevant AWS APIs. Now you can call the AWS SDK from Cognigy, without a single line of code.

**Connection:**
You have to create a user in [IAM](https://console.aws.amazon.com/iam) which has the roles for accessing the [AWS Polly](https://aws.amazon.com/de/polly/) ([Documentation](https://docs.aws.amazon.com/polly/index.html))services.
For most AWS API calls, you must supply [AWS credentials](https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html) as [access keys](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html) which have the required permissions to access the service you are calling, as well at the AWS [Region](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/#Regions).
You will need to supply:
- AWS Region
- key: region
- value: The AWS Region of your Instance
Expand All @@ -12,8 +15,72 @@ You have to create a user in [IAM](https://console.aws.amazon.com/iam) which has
- key: secretAccessKey
- value: Your AWS secret key

Good practice is to create a user in [IAM](https://console.aws.amazon.com/iam) which has only the minimal permissions needed for the Project/Flow. See the Lamda Invoke Node for an example of such minimal permissions.

----
## Node: Lambda Invoke
AWS [Lambda Functions](https://aws.amazon.com/lambda) are serverless, infinitely scalable 'run-time as a service' code functions.

This Node invoke a Lambda function, supplying input data as a JSON object, and receiving return results and output data as a JSON object.

### IAM Setup
To setup an IAM [IAM](https://console.aws.amazon.com/iam) user with just enough permissions to call your Lambda function, give the user the permissions (in JSON) as shown below.
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:<your Region>:<your AccountId>:function:<your Function Name"
}
]
}
```
The Resource details required above can easily be found while viewing the Lambda function from the AWS console - The 'ARN' is the value required for 'Resource'.

----
## Node: S3 Put Object
[S3](https://aws.amazon.com/s3) is the original cloud object(file) storage service from AWS.

This Node 'Puts' (writes) a file into an S3 '[bucket](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html)'.

To write **text data** into the file (object), set the `Data Format` to `Text`.

To write **binary data** (such as an image), first encode as a `base64` string, supply this string for `Data to Put` and set `Data Format` to `base64`. This will result in a binary file, containing any binary data values required for any file type.

**Hint:** S3 paths are really just string indexes to the object. You must always supply the exact same full path, but note that normal S3 convention is to **NOT** start with a "/", instead just like "someTopFolder/someSubFolder/someFile.txt" for example. This Node will write a warning into the project logging if such a path is used, but will still write the object as requested.

### IAM Setup
Example of IAM user permissions to read/write S3 files. Note that we grant both Put and Get actions, and you can specify a path (prefix) to limit where can be read/written:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<your Bucket Name>/<some Path>/*"
]
}
]
}
```

## Node: S3 Get Object
This Node 'Gets' (reads) a file from an S3 '[bucket](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html)'.

The options are similar to [S3 Put Object](#Node-S3-Put-Object).

----
## Node: SayPolly

Uses [AWS Polly](https://aws.amazon.com/polly/) to turn text into audio speech, via the AWS [Polly serice](https://docs.aws.amazon.com/polly/index.html).

As soon as the node gets executed, it will return the following response:
```json
{
Expand Down
6 changes: 3 additions & 3 deletions extensions/aws/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion extensions/aws/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"license": "MIT",
"dependencies": {
"@cognigy/extension-tools": "^0.10.0",
"aws-sdk": "^2.571.0",
"aws-sdk": "^2.738.0",
"tslint": "^6.1.2"
},
"devDependencies": {
Expand Down
8 changes: 7 additions & 1 deletion extensions/aws/src/module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,17 @@ import { createExtension } from "@cognigy/extension-tools";

import { sayPollyNode } from "./nodes/sayPolly";
import { awsConnection } from "./connections/awsConnection";
import { lambdaInvokeNode } from "./nodes/lambdaInvoke";
import { s3GetObjectNode } from "./nodes/s3GetObject";
import { s3PutObjectNode } from "./nodes/s3PutObject";


export default createExtension({
nodes: [
sayPollyNode
sayPollyNode,
lambdaInvokeNode,
s3GetObjectNode,
s3PutObjectNode
],

connections: [
Expand Down
217 changes: 217 additions & 0 deletions extensions/aws/src/nodes/lambdaInvoke.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
import { createNodeDescriptor, INodeFunctionBaseParams } from "@cognigy/extension-tools";
// Load the SDK:
import * as AWS from 'aws-sdk';


export interface ILambdaInvokeParams extends INodeFunctionBaseParams {
config: {
connection: {
region: string;
accessKeyId: string;
secretAccessKey: string;
};

functionName: string;
payload: string;
qualifier: string;
invocationType: "Event" | "RequestResponse" | "DryRun";
logType: "None" | "Tail";
clientContext: string;
outputInputKey: string;
outputContextKey: string;
};
}

export const lambdaInvokeNode = createNodeDescriptor({
type: "lambdaInvoke",
defaultLabel: "Lambda Invoke",
fields: [
{
key: "connection",
label: "AWS Connection",
type: "connection",
params: {
connectionType: "aws",
required: true
}
},
{
key: "functionName",
label: "Function Name/ARN",
type: "cognigyText",
params: {
required: true
}
},
{
key: "payload",
label: "Payload",
type: "json",
defaultValue: "",
},
{
key: "qualifier",
label: "Qualifier (version or alias)",
type: "cognigyText",
params: {
required: false
}
},
{
key: "invocationType",
label: "InvocationType",
type: "select",
defaultValue: "RequestResponse",
params: {
options: [
{
label: "RequestResponse",
value: "RequestResponse"
},
{
label: "Event",
value: "Event"
},
{
label: "DryRun",
value: "DryRun"
},
]
}
},
{
key: "logType",
label: "Log Type",
type: "select",
defaultValue: "None",
params: {
options: [
{
label: "None",
value: "None"
},
{
label: "Tail",
value: "Tail"
}
]
}
},
{
key: "clientContext",
label: "Client Context",
type: "json",
},
{
key: "outputInputKey",
label: "Input Key (in ci) to store Result",
type: "cognigyText",
defaultValue: "awsLambdaInvokeResult",
},
{
key: "outputContextKey",
label: "Context Key (in cc) to store Result",
type: "cognigyText",
defaultValue: "",
}
],
sections: [
{
// Having this in a section is temporary fix to JSON input not being labelled:
key: "payloadSection",
label: "Payload",
defaultCollapsed: false,
fields: [
"payload",
]
},
{
key: "optionsSection",
label: "Lambda Invoke options",
defaultCollapsed: true,
fields: [
"qualifier",
"invocationType",
"logType",
"clientContext"
]
},
{
key: "outputSection",
label: "Output",
defaultCollapsed: false,
fields: [
"outputInputKey",
"outputContextKey",
]
},

],
form: [
{ type: "field", key: "connection" },
{ type: "field", key: "functionName" },
{ type: "section", key: "payloadSection" },
{ type: "section", key: "optionsSection" },
{ type: "section", key: "outputSection" },
],
appearance: {
color: "#FF9900"
},
function: async ({ cognigy, config }: ILambdaInvokeParams) => {

const { connection, functionName, qualifier, payload, invocationType, logType, clientContext, outputInputKey, outputContextKey } = config;
if (!connection) throw new Error('aws.LambdaInvoke: The Connection data is missing.');
if (!functionName) throw new Error('aws.LambdaInvoke: The Function Name is missing.');

const { region, accessKeyId, secretAccessKey } = connection;
if (!region) throw new Error(`aws.LambdaInvoke: AWS 'region' is missing from the connection data.`);
if (!accessKeyId) throw new Error(`aws.LambdaInvoke: AWS 'accessKeyId' is missing from the connection data.`);
if (!secretAccessKey) throw new Error(`aws.LambdaInvoke: AWS 'secretAccessKey' is missing from the connection data.`);


AWS.config.update({ region, accessKeyId, secretAccessKey });
const lambda = new AWS.Lambda();

let realPayload = payload;
// Payload must be a string: Force if not:
if (payload && typeof payload !== 'string') {
if (typeof payload === 'object') {
realPayload = JSON.stringify(payload);
} else {
realPayload = String(payload);
}
}

const result = await lambda.invoke(
{
FunctionName: functionName,
Payload: realPayload || undefined,
Qualifier: qualifier || undefined,
InvocationType: invocationType || undefined,
LogType: logType || undefined,
ClientContext: clientContext || undefined
})
.promise();

// Convert the result Payload from string to object:
if (result) {
if (result.Payload && typeof result.Payload === 'string') {
result.Payload = JSON.parse(result.Payload);
}
// If we have 'LogResult' (by way of LogType:'Tail'), then it's base64 encoded, up to 4kB:
// Let's unencode it:
if (result.LogResult) {
const buff = new Buffer(result.LogResult, 'base64');
result.LogResult = buff.toString('ascii');
}
}

// Store the result in 0, 1 or both output keys, per user config:
if (outputInputKey) {
cognigy.input[outputInputKey] = result;
}
if (outputContextKey) {
cognigy.api.setContext(outputContextKey, result);
}
}
});
Loading

0 comments on commit 9e6c7d1

Please sign in to comment.