Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cumulus 3307 pg version pin #3386

Merged
merged 9 commits into from
May 24, 2023
Merged

Cumulus 3307 pg version pin #3386

merged 9 commits into from
May 24, 2023

Conversation

etcart
Copy link
Contributor

@etcart etcart commented May 18, 2023 •
edited
Loading

Summary:
Addresses local bootstrap failures stemming from bug introduced in node pg version 8.11

Changes

  • pg version pinned to 8.10x in cumulus/db

PR Checklist

  • Update CHANGELOG
  • Unit tests
  • Ad-hoc testing - Deploy changes and test manually
  • Integration tests

@etcart etcart self-assigned this May 18, 2023
@etcart etcart added needs review dependencies Pull requests that update a dependency file labels May 18, 2023
Jkovarik
Jkovarik approved these changes May 18, 2023
View reviewed changes
Copy link
Member

@Jkovarik Jkovarik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a fine change, although we're going to need to make a point in considering it each time we touch this file/update knex/etc.

@npauzenga
Copy link
Contributor

Might we also need this in the release-16.0.x branch? I suspect bootstrapping will be blocked there as well. Probably need to cherry-pick the commit from main to the release branch once this PR is merged.

@charleshuang80
Copy link
Contributor

In one of the slack threads that included Vahn, you also mentioned needing to change lambdas/db-migration/package.json as well. Is that not the case any longer?

@charleshuang80
Copy link
Contributor

Ok -- updating the pg version in db and in db-migration did not work for me. I had to update the pg version in all the package.json files. Then it all worked.

@etcart etcart merged commit cbb4088 into master May 24, 2023
Jkovarik added a commit that referenced this pull request May 25, 2023
@Jkovarik
Backport CUMULUS-3307/PR #3386
f65532a
@Jkovarik Jkovarik mentioned this pull request May 25, 2023
Merged
4 tasks
Jkovarik added a commit that referenced this pull request May 25, 2023
@Jkovarik
Jk/cumulus 3307 (#3394)
0dee3c0 
* Backport CUMULUS-3307/PR #3386

* Fixup
npauzenga added a commit that referenced this pull request Jul 14, 2023
@npauzenga @nasamoduyebo
@jennyhliu @Jkovarik @Nnaga1 @etcart
Release 16 with master (#3432)
49f3b08 
* [CUMULUS-3226] Remove refs to async operations dynamo table (#3331)

* Remove refs to async operations dynamo table

* add changelog entry

* Remove additional async operation table refs

* Update process_dead_letter_archive role

* CUMULUS-3053: removing references to dynamodb from docs (#3322)

* CUMULUS-3053: removing references to dynamodb from docs

* resolving lint errors

* CUMULUS-3285: Updated isAuthBearTokenRequest to handle non-Bearer authorization header (merge to master) (#3350) (#3352)

* CUMULUS-3285: Updated isAuthBearTokenRequest to handle non-Bearer authorization header (#3341)

* Jk rel16/cumulus 3079 (#3357)

* Update endpoints/granules :granuleName parameter to :granuleId (#3333)

* Move PUT logic to PATCH.  Implement error for PUT endpoint

* Add put units/refactor

* Update API client to use PATCH protocol

cumulus's api is moving all existing PUT requests to PATCH.  This
commit updates the api-client to use those, adds 'PATCH' to typings
and updates the test fixtures for the thin unit tests.

* Fix broken API unit

* Update CHANGELOG

* Fix naming/minor refactor

* Fix Dynamodb granule.files default bug

* Fix elasticsearch default array write value

* Update/add PUT endpoint and relevant tests

* Update CHANGELOG

* Update unit test to take advantage of fixed unit helper

* Update API version

* Add api-client method to use updated PUT endpoint

* Add fixed annotations to CHANGELOG

* Update API version unit

* Minor comment refactor

* Add PUT API spec test

* Fix incorrect test label

* Add missing unit tests

* Add removed unit tests

* Jk/cumulus 3072 add header check (#3257)

* Add API version compliance checks to app middleware

* Update middleware/add test defaults/fix typing

* Fix broken unit tests

* cleanup

* Add middleware units

* Add endpoint tests to validate middleware

* Fix unit test

* Remove unneeded type exports

* Remove unneeded test

* Remove async

* Minor formatting fix/changeset reduction

* Add missing ts-check

* Update CHANGELOG

* Fix bad import

* Update 'version' header to 'Cumulus-API-Version'

* Respond to PR feedback

* validateApiCompliance -> requireApiVersion

* Update packages/api/lib/request.js

Co-authored-by: Marc <[email protected]>

* Fix isMinVersionApi expression

* Update granule units/fix prior test fixtures

* Update packages/api/app/middleware.js

Co-authored-by: Marc <[email protected]>

* Fix spacing

* Add api-client required headers

* Update header restriction with new name, fix api-package header
inclusions

* Fix broken unit

* Bring in fix from #3270

* Update packages/api/app/middleware.js

Co-authored-by: Marc <[email protected]>

* isNumber -> Number.isFinite

---------

Co-authored-by: Marc <[email protected]>

* Update PUT to use collectionId endpoint *only*

* Update CHANGELOG

* Fix api-client to call right PUT endpoint

* Remove unneeded log output

* Update tests, add granuleId parameter updates

* Fix bad logic assertion

* Update endpoints/granules :granuleName parameter to :granuleId

* Fix lint 🔔

* Fix lint

* Fix merged unit test

* Fix merged comments

---------

Co-authored-by: Marc <[email protected]>

* Reorder/updated CHANGELOG

* Minor P3 release edits

---------

Co-authored-by: Marc <[email protected]>

* Cumulus 3120 bugfix (#3364)

* [CUMULUS-3226] Remove refs to async operations dynamo table (#3331)

* Remove refs to async operations dynamo table

* add changelog entry

* Remove additional async operation table refs

* Update process_dead_letter_archive role

* Update endpoints/granules :granuleName parameter to :granuleId (#3333)

* Move PUT logic to PATCH.  Implement error for PUT endpoint

* Add put units/refactor

* Update API client to use PATCH protocol

cumulus's api is moving all existing PUT requests to PATCH.  This
commit updates the api-client to use those, adds 'PATCH' to typings
and updates the test fixtures for the thin unit tests.

* Fix broken API unit

* Update CHANGELOG

* Fix naming/minor refactor

* Fix Dynamodb granule.files default bug

* Fix elasticsearch default array write value

* Update/add PUT endpoint and relevant tests

* Update CHANGELOG

* Update unit test to take advantage of fixed unit helper

* Update API version

* Add api-client method to use updated PUT endpoint

* Add fixed annotations to CHANGELOG

* Update API version unit

* Minor comment refactor

* Add PUT API spec test

* Fix incorrect test label

* Add missing unit tests

* Add removed unit tests

* Jk/cumulus 3072 add header check (#3257)

* Add API version compliance checks to app middleware

* Update middleware/add test defaults/fix typing

* Fix broken unit tests

* cleanup

* Add middleware units

* Add endpoint tests to validate middleware

* Fix unit test

* Remove unneeded type exports

* Remove unneeded test

* Remove async

* Minor formatting fix/changeset reduction

* Add missing ts-check

* Update CHANGELOG

* Fix bad import

* Update 'version' header to 'Cumulus-API-Version'

* Respond to PR feedback

* validateApiCompliance -> requireApiVersion

* Update packages/api/lib/request.js

Co-authored-by: Marc <[email protected]>

* Fix isMinVersionApi expression

* Update granule units/fix prior test fixtures

* Update packages/api/app/middleware.js

Co-authored-by: Marc <[email protected]>

* Fix spacing

* Add api-client required headers

* Update header restriction with new name, fix api-package header
inclusions

* Fix broken unit

* Bring in fix from #3270

* Update packages/api/app/middleware.js

Co-authored-by: Marc <[email protected]>

* isNumber -> Number.isFinite

---------

Co-authored-by: Marc <[email protected]>

* Update PUT to use collectionId endpoint *only*

* Update CHANGELOG

* Fix api-client to call right PUT endpoint

* Remove unneeded log output

* Update tests, add granuleId parameter updates

* Fix bad logic assertion

* Update endpoints/granules :granuleName parameter to :granuleId

* Fix lint 🔔

* Fix lint

* Fix merged unit test

* Fix merged comments

---------

Co-authored-by: Marc <[email protected]>

* CUMULUS-3053: removing references to dynamodb from docs (#3322)

* CUMULUS-3053: removing references to dynamodb from docs

* resolving lint errors

* CUMULUS-3285: Updated isAuthBearTokenRequest to handle non-Bearer authorization header (merge to master) (#3350)

* CUMULUS-3285: Updated isAuthBearTokenRequest to handle non-Bearer authorization header (#3341)

* CUMULUS-3121/3120 (#3360)

* CUMULUS-3121/3120 v15.1.0 backport (#3346)

* backport PR

* finalizing docs and changing remaining groups

* PR feedback + testing changes

* PR feedback

* reverting change

* Revert "update changelog"

This reverts commit ae4627c.

* reverting changes

* PR feedback

* removing EgressLambda from doc

* Update xml2js 0.4.22->0.5 strict (#3330) (#3339)

* Update xml2js 0.4.22->0.5 strict

* Address GHSA-776f-qx25-q3cc/update allow list

* Update CHANGELOG

* Update package pins to 0.5.0 for xmljs

Co-authored-by: Jonathan Kovarik <[email protected]>

* PR feedback

* PR feedback

* PR feedback

* fixing documentation linting

* PR feedback

* PR feedback

* PR feedback

* adding variables to tf-modules/workflow

* PR feedback

* PR feedback

* reverting previous change

---------

Co-authored-by: jennyhliu <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>

* PR feedback (docs+changelog)

* fixing docs

* undoing commit

* PR feedback

---------

Co-authored-by: jennyhliu <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>

* CUMULUS-3121/3120 (#3360)

* CUMULUS-3121/3120 v15.1.0 backport (#3346)

* backport PR

* finalizing docs and changing remaining groups

* PR feedback + testing changes

* PR feedback

* reverting change

* Revert "update changelog"

This reverts commit ae4627c.

* reverting changes

* PR feedback

* removing EgressLambda from doc

* Update xml2js 0.4.22->0.5 strict (#3330) (#3339)

* Update xml2js 0.4.22->0.5 strict

* Address GHSA-776f-qx25-q3cc/update allow list

* Update CHANGELOG

* Update package pins to 0.5.0 for xmljs

Co-authored-by: Jonathan Kovarik <[email protected]>

* PR feedback

* PR feedback

* PR feedback

* fixing documentation linting

* PR feedback

* PR feedback

* PR feedback

* adding variables to tf-modules/workflow

* PR feedback

* PR feedback

* reverting previous change

---------

Co-authored-by: jennyhliu <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>

* PR feedback (docs+changelog)

* fixing docs

* undoing commit

* PR feedback

---------

Co-authored-by: jennyhliu <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>

* CUMULUS-3121/3120 (#3360)

* CUMULUS-3121/3120 v15.1.0 backport (#3346)

* backport PR

* finalizing docs and changing remaining groups

* PR feedback + testing changes

* PR feedback

* reverting change

* Revert "update changelog"

This reverts commit ae4627c.

* reverting changes

* PR feedback

* removing EgressLambda from doc

* Update xml2js 0.4.22->0.5 strict (#3330) (#3339)

* Update xml2js 0.4.22->0.5 strict

* Address GHSA-776f-qx25-q3cc/update allow list

* Update CHANGELOG

* Update package pins to 0.5.0 for xmljs

Co-authored-by: Jonathan Kovarik <[email protected]>

* PR feedback

* PR feedback

* PR feedback

* fixing documentation linting

* PR feedback

* PR feedback

* PR feedback

* adding variables to tf-modules/workflow

* PR feedback

* PR feedback

* reverting previous change

---------

Co-authored-by: jennyhliu <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>

* PR feedback (docs+changelog)

* fixing docs

* undoing commit

* PR feedback

---------

Co-authored-by: jennyhliu <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>

* CUMULUS-3121/3120 (#3360)

* CUMULUS-3121/3120 v15.1.0 backport (#3346)

* backport PR

* finalizing docs and changing remaining groups

* PR feedback + testing changes

* PR feedback

* reverting change

* Revert "update changelog"

This reverts commit ae4627c.

* reverting changes

* PR feedback

* removing EgressLambda from doc

* Update xml2js 0.4.22->0.5 strict (#3330) (#3339)

* Update xml2js 0.4.22->0.5 strict

* Address GHSA-776f-qx25-q3cc/update allow list

* Update CHANGELOG

* Update package pins to 0.5.0 for xmljs

Co-authored-by: Jonathan Kovarik <[email protected]>

* PR feedback

* PR feedback

* PR feedback

* fixing documentation linting

* PR feedback

* PR feedback

* PR feedback

* adding variables to tf-modules/workflow

* PR feedback

* PR feedback

* reverting previous change

---------

Co-authored-by: jennyhliu <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>

* PR feedback (docs+changelog)

* fixing docs

* undoing commit

* PR feedback

---------

Co-authored-by: jennyhliu <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>

* fixing CHANGELOG.md

* removing docs + updating changelog

* changing doc link

---------

Co-authored-by: Nate Pauzenga <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>
Co-authored-by: Marc <[email protected]>
Co-authored-by: nasamoduyebo <[email protected]>
Co-authored-by: jennyhliu <[email protected]>

* CUMULUS-3243:Updated granule delete logic (#3338) (#3366) (#3367)

* CUMULUS-3243:Updated granule delete logic (#3338)

* CUMULUS-3243:Updated granule delete logic to delete granule which is not in DynamoDB

* add unit tests

* delete files not in master

* update CHANGELOG

* remove dynamodb from test

* Release 16.0.0 (#3376)

* update docs

* Version up to v16.0.0

* Minor edits/update release doc

* Add PI release version notes

* Minor CL edit

* Update CHANGELOG.md

Co-authored-by: Nate Pauzenga <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Nate Pauzenga <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Nate Pauzenga <[email protected]>

* Address PR feedback

---------

Co-authored-by: Nate Pauzenga <[email protected]>

* [CUMULUS-3172] Update data integrity/migration docs (#3387)

* update docs

* update docs

* add new doc to sidebars

* Update docs/upgrade-notes/rds-phase-3-data-migration-guidance.md

Co-authored-by: Jonathan Kovarik <[email protected]>

* Update docs/upgrade-notes/rds-phase-3-data-migration-guidance.md

Co-authored-by: Jonathan Kovarik <[email protected]>

* Update docs/upgrade-notes/rds-phase-3-data-migration-guidance.md

Co-authored-by: Jonathan Kovarik <[email protected]>

* Update docs/upgrade-notes/rds-phase-3-data-migration-guidance.md

Co-authored-by: Jonathan Kovarik <[email protected]>

---------

Co-authored-by: Jonathan Kovarik <[email protected]>

* Fix lint issues (#3392)

* Jk/cumulus 3307 (#3394)

* Backport CUMULUS-3307/PR #3386

* Fixup

* Re-issue v16 docs (#3400)

* CUMULUS-3223: Fix failed granule stuck in queued (#3373) (#3402)

* CUMULUS-3223:Fix failed granule stuck in queued

* skip sqsQueueExists test

* update getGranuleTemporalInfo

* update test match schema

* update getGranuleTemporalInfo

* remove extra await

* remove skip sqsQueueExists

* update `@cumulus/cumulus-message-adapter-js` to `2.0.5`

* update sfEventSqsToDbRecords to return partial batch failure

* fix typo

* handle process error seperately, multiple message test

* update test to process multiple messages

* Jk/cumulus 3315 (#3407)

* CUMULUS-3135 - Update integration test scripts to fail on test timeout (#3401)

* Update integration test scripts to fail on test timeout

* Fixup

* Fixup

* Update script interpreter for test runs

* Fix script

* Fixup

* Fixup

* Update timeout pass/fail conditional

* Jk/cumulus 3135 fix integration tests (#3403)

* Update integration test scripts to fail on test timeout

* Fixup

* Fixup

* Update script interpreter for test runs

* Fix script

* Fixup

* Fixup

* Update api/client and integration test usage of it to fix test failures

* Fix formatting/lint/etc

* Update test/minor fix

---------

Co-authored-by: etcart <[email protected]>

---------

Co-authored-by: etcart <[email protected]>

* v16.0.1 alpha release for testing (#3409)

* bump to 16.0.1-alpha.0

* resolve conflicts

---------

Co-authored-by: Jonathan Kovarik <[email protected]>

* Update serve.js to match main

* bump version

* Add missing method for erasing PG tables (#3419)

* add missing method for erasing PG tables

* Remove duplicate declaration

* Remove duplicate declaration

* Take serve.js and serveUtils.js from main

* Update schema endpoint to utilize P3 api/lib schema

---------

Co-authored-by: Jonathan Kovarik <[email protected]>

* Release 16.0.0 (#3428)

* version bump

* update changelog version

* changelog fixup

* changelog fixup

* update cumulus versions in new lambda

* bump cumulus versions

---------

Co-authored-by: nasamoduyebo <[email protected]>
Co-authored-by: jennyhliu <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>
Co-authored-by: Marc <[email protected]>
Co-authored-by: Naga Nages <[email protected]>
Co-authored-by: etcart <[email protected]>
Co-authored-by: Jonathan Kovarik <[email protected]>
@1proprogrammerchant 1proprogrammerchant mentioned this pull request Jul 15, 2023
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@charleshuang80 charleshuang80 charleshuang80 approved these changes

@Jkovarik Jkovarik Jkovarik approved these changes

Assignees

@etcart etcart

Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@etcart @npauzenga @charleshuang80 @Jkovarik