Skip to content

[Fetch Migration] CDK bugfix and other minor updates#387

Closed
kartg wants to merge 3 commits intoopensearch-project:mainfrom
kartg:fetch-migration-cdk-bugfix
Closed

[Fetch Migration] CDK bugfix and other minor updates#387
kartg wants to merge 3 commits intoopensearch-project:mainfrom
kartg:fetch-migration-cdk-bugfix

Conversation

@kartg
Copy link
Copy Markdown
Member

@kartg kartg commented Nov 3, 2023

Description

  • Category: Bug fix

Why these changes are required?

The Fetch Migration stack now uses StringParameter.valueFromLookup to resolve the target cluster endpoint from Parameter Store. This is required instead of the StringParameter.valueForStringParameter that was previously used because valueForStringParameter returns a Token that is only resolved at deployment time.

This PR also includes two other minor updates:

  • Bumping the Fetch Migration task definition to 1 CPU and 4 GB of memory
  • Additional documentation in the pipeline template file to aid readability

What is the old behavior before changes and new behavior after changes?

Since the value of the target cluster endpoint is substituted into the pipeline template file prior to deployment time, a string representation of the Token object (eg. ${Token[TOKEN.123]}) was being written instead of the actual endpoint URI. Using valueFromLookup looks up the value immediately, so the substitution occurs correctly with the bugfix.

Issues Resolved

N/A

Testing

Bug identified while using CDK. Bugfix verified by re-deploying CDK and verifying that the correct value is now used.

Check List

  • New functionality includes testing
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

kartg added 3 commits November 3, 2023 12:21
…r endpoint in Fetch Migration stack

This is required instead of StringParameter.valueForStringParameter because the latter returns a token that is only resolved at deployment time. Since the value of the target cluster endpoint is substituted into the Data Prepper pipeline file prior to the deployment, it incorrectly wrote a string representation of the Token object - "${Token[TOKEN.123]}". Using StringParameter.valueFromLookup looks up the value immediately, so now the substitution occurs correctly.

Signed-off-by: Kartik Ganesh <gkart@amazon.com>
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
…ntation

Signed-off-by: Kartik Ganesh <gkart@amazon.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented Nov 3, 2023

Codecov Report

Merging #387 (2b9f50f) into main (2993d19) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main     #387   +/-   ##
=========================================
  Coverage     64.23%   64.23%           
  Complexity      726      726           
=========================================
  Files            82       82           
  Lines          3277     3277           
  Branches        306      306           
=========================================
  Hits           2105     2105           
  Misses          983      983           
  Partials        189      189           
Flag Coverage Δ
unittests 64.23% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


// Import required values
const targetClusterEndpoint = StringParameter.valueForStringParameter(this, `/migration/${props.stage}/${props.defaultDeployId}/osClusterEndpoint`)
const targetClusterEndpoint = StringParameter.valueFromLookup(this, `/migration/${props.stage}/${props.defaultDeployId}/osClusterEndpoint`)
Copy link
Copy Markdown
Contributor

@lewijacn lewijacn Nov 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this makes sense. Value from lookup requires that the value be there at synthesis time. If there is anyway we can retrieve this from the Migration Console it seems preferable

@kartg
Copy link
Copy Markdown
Member Author

kartg commented Nov 5, 2023

Moving to draft PR based on this comment

@kartg
Copy link
Copy Markdown
Member Author

kartg commented Nov 9, 2023

Closing in favor of #398

@kartg kartg closed this Nov 9, 2023
@kartg kartg deleted the fetch-migration-cdk-bugfix branch November 10, 2023 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants