Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc-updates in the md files #13

Merged
merged 1 commit into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 22 additions & 12 deletions DEVELOPER.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,38 @@
# aws-unified-firehose
Forwards logs from cloudwatch to NewRelic through Amazon Data Firehose
# AWS unified Firehose

Forwards logs from CloudWatch to New Relic via Amazon Kinesis Data Firehose.

## Requirements

- SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)
- Install the AWS SAM CLI. Refer [SAM CLI Documentation](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)

## Deployment

To try this integration out you can use the `sam` cli to deploy the cloudformation template (`firehose-template.yml`). Make sure **aws is properly authenticated with an account of your choice**.
To try-out this integration, deploy the CloudFormation template (`firehose-template.yml`) using the `sam` CLI. Ensure AWS is authenticated with the desired account.


### Building and packaging

## Building and packaging
To build and package, follow these steps:
1. Authenticate with your aws account details
2. Create an S3 bucket with a unique name, e.g., `test123`.
3. Build the project:

1. Authenticate with your AWS account details.
2. Create an S3 bucket with name. For example, `test123`.
3. To create the project build, run:

```sh
sam build --template-file firehose-template.yaml
sam build --template-file firehose-template.yaml
```
4. The build will be located by default at `.aws-sam/build`, and a template file will be created with the name `template.yaml`.
5. Package the project:

**Note:** By default, build will be available at `.aws-sam/build` with the generated `template.yaml`

4. To package the build, run:

```sh
sam package --s3-bucket test123 --template-file .aws-sam/build/template.yaml --output-template-file firehose-template.yaml --region us-east-2
```
6. Copy the main template file to the S3 bucket:

5. Copy the main template file to the S3 bucket using:

```sh
aws s3 cp .aws-sam/build/firehose-template.yaml s3://test123/firehose-template.yaml
```
48 changes: 22 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,38 +1,34 @@
# aws-unified-firehose
Forwards logs from cloudwatch to NewRelic through Amazon Data Firehose
# AWS unified Firehose

Forwards logs from Amazon CloudWatch to New Relic via Amazon Kinesis Data Firehose.

## Features

- Collects logs from Amazon CloudWatch.
- Forwards logs to NewRelic using Amazon Data Firehose.
- Allows users to attach custom attributes to the logs to make it easier to search, filter, analyze, and parse the logs
- Scalable and reliable log forwarding.
- Stores license key in Secret Manager by default.
- Collects logs from CloudWatch.
- Forwards logs to New Relic using Amazon Kinesis Data Firehose.
- Enables users to attach custom attributes to logs. This will allows users to easily search, filter, analyze, and parse the logs.
- Offers scalable and reliable log forwarding.
- Stores the license key in Secrets Manager by default.

## Requirements

- SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)
- Install the AWS SAM CLI. Refer [SAM CLI Documentation](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)

## Deployment

To try this integration out you can use the `sam` cli to deploy the cloudformation template (`firehose-template.yml`). Make sure **aws is properly authenticated with an account of your choice**.

To try-out this integration, deploy the CloudFormation template (`firehose-template.yml`) using the `sam` CLI. Ensure that the AWS is authenticated with the desired account.

#### CloudFormation Parameters

- `NewRelicRegion` : Can either be `US` or `EU` depending on which endpoint to be used to push logs to New Relic
- For this param `US` is default
- `LicenseKey`: Used when forwarding logs to New Relic
- `LogGroupConfig` : String representation of JSON array of objects of your CloudWatch LogGroup(s) and respective filter (if applicable) to set the Lambda function trigger.
- Example : ```[{"LogGroupName":"group1"}, {"LogGroupName":"group2", "FilterPattern":"ERROR"}, {"LogGroupName":"group3", "FilterPattern":"INFO"}]```
- `LoggingFirehoseStreamName` : Name of new Data Firehose Delivery Stream (must be unique per AWS account in the same AWS Region)
- The default value will be `NewRelic-Logging-Delivery-Stream`
- `LoggingS3BackupBucketName`: S3 Bucket Destination for failed events (must be globally unique across all AWS accounts in all AWS Regions within a partition)
- The default value will be `firehose-logging-backup`
- `EnableCloudWatchLoggingForFirehose`: Can either be `true` or `false` to enable CloudWatch logging for the Amazon Data Firehose stream. Enabling logging can help in troubleshooting issues in pushing data through firehose stream. `false` by default
- `NewRelicAccountId` : The New Relic Account ID to which the logs will be pushed
- `CommonAttributes` : Common attributes to be added to all logs. This should be a JSON object.
- Example : ```[{"AttributeName": "name1", "AttributeValue": "value1"}, {"AttributeName": "name2", "AttributeValue": "value2}]```
- `StoreNRLicenseKeyInSecretManager` : Can either be `true` or `false` depending on which cloud formation stack decides whether to store your license key in the environment variables or to create a new secret in aws secrets manger.
- For this param `true` is default
### CloudFormation Parameters

| Parameter | Description |
|----------------------------------------|-------------|
| `NewRelicRegion` | The New Relic region (`US` or `EU`) for log forwarding. The default value is `US`. |
| `LicenseKey` | Your New Relic license key for log forwarding. |
| `LogGroupConfig` | A JSON array defining CloudWatch LogGroups and filters to set triggers for the Lambda function. For example: `[{"LogGroupName":"group1"}, {"LogGroupName":"group2", "FilterPattern":"ERROR"}, {"LogGroupName":"group3", "FilterPattern":"INFO"}]` |
| `LoggingFirehoseStreamName` | Unique name for the Data Firehose Delivery Stream. The default value is `NewRelic-Logging-Delivery-Stream` |
| `LoggingS3BackupBucketName` | Unique name for S3 bucket for backup of failed events. This name must be globally unique. The default value is `firehose-logging-backup` |
| `EnableCloudWatchLoggingForFirehose` | CloudWatch logging for the Amazon Data Firehose stream. Enabling this can help you to troubleshoot issues in firehose stream. .The default value is `false` |
| `NewRelicAccountId` | The New Relic account ID to push the log. |
| `CommonAttributes` | JSON object of common attributes to add to all logs. For example: `[{"AttributeName": "name1", "AttributeValue": "value1"}, {"AttributeName": "name2", "AttributeValue": "value2"}]` |
| `StoreNRLicenseKeyInSecretManager` | Determines if the license key is stored in AWS Secrets Manager (`true`) or environment variables (`false`). The default value is `true`. |