auth: store state in postgres instance #1420

iulianbarbu · 2023-11-23T19:04:54Z

Description of change

Store the auth service state inside Postgres.

Note: This PR should NOT be merged before we set up the migration logic for the existing SQLite-based state to Postgres.

How has this been tested? (if applicable)

it was tested locally by starting the entire stack and deploying a project with a newly created admin user.
test on staging on a brand new empty db.

Kazy

Nice work, didn't expect to see this that fast 💪 Maybe switch the PR as draft to make sure we don't inadvertently merge it ?

auth/migrations/0000_init.sql

auth/src/args.rs

oddgrd

LGTM! Just a couple of questions and minor comments.

test-utils/src/lib.rs

test-utils/Cargo.toml

test-utils/src/lib.rs

Kazy · 2023-12-08T09:14:51Z

test-utils/src/lib.rs

+    }
+}
+
+impl PostgresDockerInstance {


No need to change now that it is done, but we might want to look into https://testcontainers.com/ to avoid doing all that work ourselves.

I just saw it today, looks promising.

Oh, I didn't know about that, cool!

Opened #1479. It shouldn't be difficult and might be a nice OSS contribution. What do you think?

I haven't looked closely at test containers, but it does seem like it would simplify this a lot. For the GitHub issue we might want to expand a bit on what should change, and where, since we use this strategy in logger, provisioner and now auth. Provisioner does not use the code you moved to common-tests, but maybe we don't need to share code in common-tests if we use test containers.

We might want to suggest that the contributor breaks it up into separate PRs, as well.

Thanks @oddgrd ! I added more details under the definition of done for the associated issue. I missed that we have the docker tests in the provisioner too. In common-tests we created API to create/get unique conn strings, referring to unique databases, that can be used in parallel by test cases. I think this is the outstanding part shared by logger and auth soon. Not sure if the provisioner needs this functionality for its integration tests, and if not, it can probably avoid using common-tests, integrating directly with testcontainers.

test-utils/src/lib.rs

oddgrd

LGTM! 🥳

.circleci/config.yml

Also removed the build & push approval, since it is not used for the production images.

iulianbarbu force-pushed the feature/eng-1920-switch-auth-service-state-from-sqlitedb-to-a-postgres branch from b512b07 to 5fb2848 Compare November 23, 2023 19:13

iulianbarbu changed the title ~~auth: store state in postgre instance~~ auth: store state in postgres instance Nov 23, 2023

iulianbarbu self-assigned this Nov 23, 2023

iulianbarbu requested review from chesedo, oddgrd and Kazy November 23, 2023 19:14

iulianbarbu force-pushed the feature/eng-1920-switch-auth-service-state-from-sqlitedb-to-a-postgres branch 3 times, most recently from 9332bda to fcc27fc Compare November 23, 2023 19:30

Kazy reviewed Nov 23, 2023

View reviewed changes

auth/migrations/0000_init.sql Outdated Show resolved Hide resolved

auth/src/args.rs Outdated Show resolved Hide resolved

iulianbarbu marked this pull request as draft November 24, 2023 10:57

iulianbarbu force-pushed the feature/eng-1920-switch-auth-service-state-from-sqlitedb-to-a-postgres branch 6 times, most recently from 5fce012 to 051b472 Compare November 29, 2023 17:40

iulianbarbu force-pushed the feature/eng-1920-switch-auth-service-state-from-sqlitedb-to-a-postgres branch 4 times, most recently from 4333ca3 to 7716b4c Compare December 7, 2023 19:33

iulianbarbu marked this pull request as ready for review December 8, 2023 08:02

iulianbarbu requested a review from Kazy December 8, 2023 08:03

oddgrd reviewed Dec 8, 2023

View reviewed changes

test-utils/src/lib.rs Outdated Show resolved Hide resolved

test-utils/Cargo.toml Outdated Show resolved Hide resolved

test-utils/src/lib.rs Outdated Show resolved Hide resolved

test-utils/src/lib.rs Outdated Show resolved Hide resolved

Kazy reviewed Dec 8, 2023

View reviewed changes

iulianbarbu commented Dec 8, 2023

View reviewed changes

test-utils/src/lib.rs Outdated Show resolved Hide resolved

iulianbarbu added 4 commits December 8, 2023 19:20

test-utils: added crate for testing utilities

1055b2f

logger: testing based on test-utils

cbaa16f

auth: move from sqlite to postgres

7a3f008

auth: use the control db postgres uri when deploying

1d98c4d

iulianbarbu added 6 commits December 8, 2023 19:20

auth: updated DEVELOPING.md

3e1a1e7

auth: handle AccountTier conversion from TEXT

7845b80

auth: removed arg default value

0b0ed1f

test-utils: fix the public docs example

9f64ccb

test-utils: simplify get_unique_uri

0c1f443

auth: update deploy recipe

9025378

iulianbarbu force-pushed the feature/eng-1920-switch-auth-service-state-from-sqlitedb-to-a-postgres branch from cc29418 to 04dc9ef Compare December 8, 2023 17:21

iulianbarbu added 8 commits December 8, 2023 19:28

fix(auth): pass over the control db uri directly

65e1ad2

auth: readded complete arg version for state

bb73341

auth: init-admin query tier should be stringified

a8fa6c9

circleci: add approve step for deploying

75018b7

test-utils: remove prepended fn underline

26ce1a0

common-tests: merged with test-utils

da844f1

common-tests: adjust postgres docker instance docs

0950e6d

common-tests: enable postgres docker instance syntax highlight

de5e417

iulianbarbu force-pushed the feature/eng-1920-switch-auth-service-state-from-sqlitedb-to-a-postgres branch from 04dc9ef to de5e417 Compare December 8, 2023 17:28

circleci: uncomment main filter

5300add

iulianbarbu requested a review from oddgrd December 8, 2023 21:01

oddgrd approved these changes Dec 11, 2023

View reviewed changes

.circleci/config.yml Outdated Show resolved Hide resolved

circleci: rename to build & push approve

0ea78b8

Also removed the build & push approval, since it is not used for the production images.

iulianbarbu merged commit da538ac into shuttle-hq:main Dec 12, 2023
35 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

auth: store state in postgres instance #1420

auth: store state in postgres instance #1420

iulianbarbu commented Nov 23, 2023 •

edited

Loading

Kazy left a comment

oddgrd left a comment

Kazy Dec 8, 2023

iulianbarbu Dec 8, 2023

oddgrd Dec 8, 2023

iulianbarbu Dec 8, 2023

oddgrd Dec 11, 2023

oddgrd Dec 11, 2023

iulianbarbu Dec 11, 2023 •

edited

Loading

oddgrd left a comment

auth: store state in postgres instance #1420

auth: store state in postgres instance #1420

Conversation

iulianbarbu commented Nov 23, 2023 • edited Loading

Description of change

How has this been tested? (if applicable)

Kazy left a comment

Choose a reason for hiding this comment

oddgrd left a comment

Choose a reason for hiding this comment

Kazy Dec 8, 2023

Choose a reason for hiding this comment

iulianbarbu Dec 8, 2023

Choose a reason for hiding this comment

oddgrd Dec 8, 2023

Choose a reason for hiding this comment

iulianbarbu Dec 8, 2023

Choose a reason for hiding this comment

oddgrd Dec 11, 2023

Choose a reason for hiding this comment

oddgrd Dec 11, 2023

Choose a reason for hiding this comment

iulianbarbu Dec 11, 2023 • edited Loading

Choose a reason for hiding this comment

oddgrd left a comment

Choose a reason for hiding this comment

iulianbarbu commented Nov 23, 2023 •

edited

Loading

iulianbarbu Dec 11, 2023 •

edited

Loading