-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use SQLite instead of bbolt #779
Conversation
763b8ed
to
99bfaab
Compare
99bfaab
to
79ce337
Compare
143ef36
to
1c7b268
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oof, this was a huge one. Overall, amazing work! I have some comments and nitpicks (of course 🙃)...
nss/integration-tests/testdata/golden/TestIntegration/Get_all_entries_from_group
Outdated
Show resolved
Hide resolved
136001f
to
4a8bb53
Compare
internal/users/db/testutils.go
Outdated
// Z_ForTests_DumpNormalizedYAML gets the content of the database, normalizes it | ||
// (so that it can be compared with a golden file) and returns it as a YAML string. | ||
// | ||
// nolint:revive,nolintlint // We want to use underscores in the function name here. | ||
func Z_ForTests_DumpNormalizedYAML(t *testing.T, c *Manager) string { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to be safe: I remember you mentioning that these symbols are left out of the release code, but what happens to the dependencies? Are they still included in the package or they are also left out? Here you are importing the testing
package inside the db
one and that's something we avoid at all costs... If it's not included in the release, I guess we are fine, but if it is, we should probably go back to return errors and let the test function deal with them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not in the binary:
go build -o authd ./cmd/authd && go tool nm authd | grep Z_ForTests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm aware that the function isn't part of it, but does it pull the testing
package into the binary even though the function itself is trimmed out?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I understand what you mean now. Yes, there are indeed some symbols from the testing package:
go build -o authd ./cmd/authd && go tool nm authd | grep testing
1234e28 d os.testingForceReadDirLstat
11d6160 d testing..inittask
d48f60 r testing..stmp_19
d48f70 r testing..stmp_20
d472fc r testing..stmp_60
97a240 t testing.init
1210ad8 d testing.supportedTypes
I'll change the functions to return errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, that's probably something for a follow-up task, because the testing
package is already used in that file on main
:
authd/internal/users/cache/testutils.go
Line 13 in 0b9e98f
"testing" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only line I see testing
being used there is here:
authd/internal/users/cache/testutils.go
Line 157 in 0b9e98f
func Z_ForTests_CreateDBFromYAML(t *testing.T, src, destDir string) { |
So the problem is the same
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, so we need to remove it from that function as well, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I still think that could have happened in a follow-up task, but I pushed a commit which removes the dependency on the testing package.
bea2eab
to
981dfe2
Compare
6a6377d
to
f9f5d85
Compare
Changes done by running: git rm "internal/*/testdata/golden/*" TESTS_UPDATE_GOLDEN=1 go test ./internal/... git add -A
We don't use our database as a cache anymore. It's time to reflect this in the package name.
We can't use the "AAAAATIME" placeholders anymore when using SQLite, because the schema only allows actual timestamps.
We don't store the current UID in golden files anymore since 3f81332.
These files were forgotten in 5555836. Also simplify the testdata for the "pam_unix_non_existent" test case, because we don't need multiple users and groups for that test case.
Our database requirements are better met by a relational database than a key value store, so we now use SQLite instead of bbolt.
Some cases were not tested.
And inline the `userRow` struct into `UserDB`, because it now doesn't have any additional fields anymore.
Explain what a local group is
* Make groupRow exported by renaming it to to GroupRow. This allows us to use it whenever we don't need the members of the group. * Add GroupWithMembersByID and GroupWithMembersByName, to allow the users manager to fetch a group with its members in a single transaction. * Rename GroupDB to GroupWithMembers, which is more descriptive. * Rename UserDB to UserRow. More descriptive and consistent with GroupRow.
To make it a bit more clear from the variable name why we use the GID of this group for the user.
... but don't fail if the database has other permissions than 0600, because it doesn't contain any secrets, so it's fine if it's readable by other users than the owner. Check that it's not *writable* by other users though.
f9f5d85
to
df8b137
Compare
Our database requirements are better met by a relational database than a key value store, so we now use SQLite instead of bbolt.
UDENG-4890