Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

moderation: migrate away from gorm to sqlboiler #1658

Conversation

jo3-l
Copy link
Contributor

@jo3-l jo3-l commented May 30, 2024

No description provided.

moderation/commands.go Outdated Show resolved Hide resolved
@jo3-l jo3-l force-pushed the refactor/moderation-plugin-to-sqlboiler branch from fa05e29 to be169b1 Compare May 31, 2024 01:08
jo3-l added 3 commits May 30, 2024 18:17
We currently use gorilla/schema to parse HTML form values into structs -- this allows us to specify, e,g., `<input ... name=SomeField>` and have `payload.SomeField` set correctly in Go handlers. This approach mostly works nicely, with the major exception of null.Int64 (formerly sql.NullInt64 with gorm) fields, which show up in the moderation plugin, e.g., Config.DefaultMuteDuration.

Particularly, since gorilla/schema has no native support for null.Int64, we have instead instructed gorilla/schema to directly set the Int64 value using <input ... name=DefaultMuteDuration.Int64> and then manually set DefaultMuteDuration.Valid=true in the web handler, without which the changes are not saved to database. This approach is, for obvious reasons, rather brittle. In this commit we therefore register a custom gorilla/schema decoder for null.Int64, obviating the need for both pointing to the Int64 field and manually setting Valid=true. While at it, we also fix the validator so that it checks null.Int64 fields properly.
gorm uses pq.Int64Array but sqlboiler uses types.Int64Array, so we need also support the latter now.
@jo3-l jo3-l force-pushed the refactor/moderation-plugin-to-sqlboiler branch from be169b1 to 5ac8c34 Compare May 31, 2024 01:25
@jo3-l
Copy link
Contributor Author

jo3-l commented May 31, 2024

I am tentatively marking this PR as ready to review. The obvious mistakes that I found have been fixed and the features I have tested so far do work. But as the previous (incorrect) comment demonstrated, there are some subtle aspects here that need to be reviewed and tested very carefully.

@jo3-l jo3-l marked this pull request as ready for review May 31, 2024 01:27
@jo3-l jo3-l changed the title wip: moderation: migrate away from gorm to sqlboiler moderation: migrate away from gorm to sqlboiler May 31, 2024
@ashishjh-bst ashishjh-bst merged commit 2c6f65a into botlabs-gg:gorm-to-sqlboiler Jun 16, 2024
1 check passed
ashishjh-bst pushed a commit that referenced this pull request Jun 19, 2024
* moderation: decouple from configstore

* moderation: add database schemas

* moderation: set up and run sqlboiler

* moderation: migrate to sqlboiler

* web: support null.Int64 in forms and validator

We currently use gorilla/schema to parse HTML form values into structs -- this allows us to specify, e,g., `<input ... name=SomeField>` and have `payload.SomeField` set correctly in Go handlers. This approach mostly works nicely, with the major exception of null.Int64 (formerly sql.NullInt64 with gorm) fields, which show up in the moderation plugin, e.g., Config.DefaultMuteDuration.

Particularly, since gorilla/schema has no native support for null.Int64, we have instead instructed gorilla/schema to directly set the Int64 value using <input ... name=DefaultMuteDuration.Int64> and then manually set DefaultMuteDuration.Valid=true in the web handler, without which the changes are not saved to database. This approach is, for obvious reasons, rather brittle. In this commit we therefore register a custom gorilla/schema decoder for null.Int64, obviating the need for both pointing to the Int64 field and manually setting Valid=true. While at it, we also fix the validator so that it checks null.Int64 fields properly.

* web: support validating types.Int64Array

gorm uses pq.Int64Array but sqlboiler uses types.Int64Array, so we need also support the latter now.
ashishjh-bst added a commit that referenced this pull request Jun 19, 2024
* all: migrate executed command logging away from gorm to sqlboiler (#1654)

* common: add schemas for executed command logs

* common: configure and run sqlboiler for executed command log model

* all: migrate executed command logging to sqlboiler

* reminders: migrate away from gorm to sqlboiler (#1652)

* reminders: add database schemas

* reminders: set up and run sqlboiler

* reminders: migrate to sqlboiler

While at it, lightly refactor and simplify the code as applicable. The most substantial (non-database) refactor is renaming and moving strReminders to DisplayReminders.

To aid in the refactor, also add discordgo.ParseID as a shortcut for strconv.ParseInt(...). This operation is often needed to convert string IDs in database to int64's and having a shorter name helps.

* reminders: auto-delete old entries with null guild_id as necessary

* soundboard: remove stray references to gorm (#1651)

The soundboard module used to use gorm, but was migrated to sqlboiler some years ago in commit 628ea9a. There are still two (erroneous) references to gorm remaining which were not caught since this section of code ignores errors. This commit changes these to use sqlboiler as well.

* schemamigration: skip executing ALTER COLUMN SET NOT NULL queries when possible (#1650)

* schemamigration: skip executing ALTER COLUMN SET NOT NULL queries when possible

* Use quantifier * not + in regex for consistency

It really should be +, but the current code works and hey -- when in Rome, do as the Romans.

* youtube: migrate away from gorm to sqlboiler (#1660)

* youtube: add database schemas

* youtube: set up and run sqlboiler

* youtube: migrate to sqlboiler

* notifications: migrate away from gorm to sqlboiler (#1659)

* notifications: decouple from configstore

* notifications: add database schemas

* notifications: set up and run sqlboiler

* notifications: migrate to sqlboiler

* moderation: migrate away from gorm to sqlboiler (#1658)

* moderation: decouple from configstore

* moderation: add database schemas

* moderation: set up and run sqlboiler

* moderation: migrate to sqlboiler

* web: support null.Int64 in forms and validator

We currently use gorilla/schema to parse HTML form values into structs -- this allows us to specify, e,g., `<input ... name=SomeField>` and have `payload.SomeField` set correctly in Go handlers. This approach mostly works nicely, with the major exception of null.Int64 (formerly sql.NullInt64 with gorm) fields, which show up in the moderation plugin, e.g., Config.DefaultMuteDuration.

Particularly, since gorilla/schema has no native support for null.Int64, we have instead instructed gorilla/schema to directly set the Int64 value using <input ... name=DefaultMuteDuration.Int64> and then manually set DefaultMuteDuration.Valid=true in the web handler, without which the changes are not saved to database. This approach is, for obvious reasons, rather brittle. In this commit we therefore register a custom gorilla/schema decoder for null.Int64, obviating the need for both pointing to the Int64 field and manually setting Valid=true. While at it, we also fix the validator so that it checks null.Int64 fields properly.

* web: support validating types.Int64Array

gorm uses pq.Int64Array but sqlboiler uses types.Int64Array, so we need also support the latter now.

* common/configstore: remove package (#1668)

* all: remove last traces of gorm (#1670)

* common: connect to postgres via database/sql

...instead of through gorm.

* common: remove gorm log hook

* common: remove gorm SmallModel

* deps: run go mod tidy

---------

Co-authored-by: Joe L <[email protected]>
ashishjh-bst added a commit to ashishjh-bst/yagpdb that referenced this pull request Jun 20, 2024
…g#1655)

* all: migrate executed command logging away from gorm to sqlboiler (botlabs-gg#1654)

* common: add schemas for executed command logs

* common: configure and run sqlboiler for executed command log model

* all: migrate executed command logging to sqlboiler

* reminders: migrate away from gorm to sqlboiler (botlabs-gg#1652)

* reminders: add database schemas

* reminders: set up and run sqlboiler

* reminders: migrate to sqlboiler

While at it, lightly refactor and simplify the code as applicable. The most substantial (non-database) refactor is renaming and moving strReminders to DisplayReminders.

To aid in the refactor, also add discordgo.ParseID as a shortcut for strconv.ParseInt(...). This operation is often needed to convert string IDs in database to int64's and having a shorter name helps.

* reminders: auto-delete old entries with null guild_id as necessary

* soundboard: remove stray references to gorm (botlabs-gg#1651)

The soundboard module used to use gorm, but was migrated to sqlboiler some years ago in commit 628ea9a. There are still two (erroneous) references to gorm remaining which were not caught since this section of code ignores errors. This commit changes these to use sqlboiler as well.

* schemamigration: skip executing ALTER COLUMN SET NOT NULL queries when possible (botlabs-gg#1650)

* schemamigration: skip executing ALTER COLUMN SET NOT NULL queries when possible

* Use quantifier * not + in regex for consistency

It really should be +, but the current code works and hey -- when in Rome, do as the Romans.

* youtube: migrate away from gorm to sqlboiler (botlabs-gg#1660)

* youtube: add database schemas

* youtube: set up and run sqlboiler

* youtube: migrate to sqlboiler

* notifications: migrate away from gorm to sqlboiler (botlabs-gg#1659)

* notifications: decouple from configstore

* notifications: add database schemas

* notifications: set up and run sqlboiler

* notifications: migrate to sqlboiler

* moderation: migrate away from gorm to sqlboiler (botlabs-gg#1658)

* moderation: decouple from configstore

* moderation: add database schemas

* moderation: set up and run sqlboiler

* moderation: migrate to sqlboiler

* web: support null.Int64 in forms and validator

We currently use gorilla/schema to parse HTML form values into structs -- this allows us to specify, e,g., `<input ... name=SomeField>` and have `payload.SomeField` set correctly in Go handlers. This approach mostly works nicely, with the major exception of null.Int64 (formerly sql.NullInt64 with gorm) fields, which show up in the moderation plugin, e.g., Config.DefaultMuteDuration.

Particularly, since gorilla/schema has no native support for null.Int64, we have instead instructed gorilla/schema to directly set the Int64 value using <input ... name=DefaultMuteDuration.Int64> and then manually set DefaultMuteDuration.Valid=true in the web handler, without which the changes are not saved to database. This approach is, for obvious reasons, rather brittle. In this commit we therefore register a custom gorilla/schema decoder for null.Int64, obviating the need for both pointing to the Int64 field and manually setting Valid=true. While at it, we also fix the validator so that it checks null.Int64 fields properly.

* web: support validating types.Int64Array

gorm uses pq.Int64Array but sqlboiler uses types.Int64Array, so we need also support the latter now.

* common/configstore: remove package (botlabs-gg#1668)

* all: remove last traces of gorm (botlabs-gg#1670)

* common: connect to postgres via database/sql

...instead of through gorm.

* common: remove gorm log hook

* common: remove gorm SmallModel

* deps: run go mod tidy

---------

Co-authored-by: Joe L <[email protected]>
ashishjh-bst added a commit to ashishjh-bst/yagpdb that referenced this pull request Jun 27, 2024
…g#1655)

* all: migrate executed command logging away from gorm to sqlboiler (botlabs-gg#1654)

* common: add schemas for executed command logs

* common: configure and run sqlboiler for executed command log model

* all: migrate executed command logging to sqlboiler

* reminders: migrate away from gorm to sqlboiler (botlabs-gg#1652)

* reminders: add database schemas

* reminders: set up and run sqlboiler

* reminders: migrate to sqlboiler

While at it, lightly refactor and simplify the code as applicable. The most substantial (non-database) refactor is renaming and moving strReminders to DisplayReminders.

To aid in the refactor, also add discordgo.ParseID as a shortcut for strconv.ParseInt(...). This operation is often needed to convert string IDs in database to int64's and having a shorter name helps.

* reminders: auto-delete old entries with null guild_id as necessary

* soundboard: remove stray references to gorm (botlabs-gg#1651)

The soundboard module used to use gorm, but was migrated to sqlboiler some years ago in commit 628ea9a. There are still two (erroneous) references to gorm remaining which were not caught since this section of code ignores errors. This commit changes these to use sqlboiler as well.

* schemamigration: skip executing ALTER COLUMN SET NOT NULL queries when possible (botlabs-gg#1650)

* schemamigration: skip executing ALTER COLUMN SET NOT NULL queries when possible

* Use quantifier * not + in regex for consistency

It really should be +, but the current code works and hey -- when in Rome, do as the Romans.

* youtube: migrate away from gorm to sqlboiler (botlabs-gg#1660)

* youtube: add database schemas

* youtube: set up and run sqlboiler

* youtube: migrate to sqlboiler

* notifications: migrate away from gorm to sqlboiler (botlabs-gg#1659)

* notifications: decouple from configstore

* notifications: add database schemas

* notifications: set up and run sqlboiler

* notifications: migrate to sqlboiler

* moderation: migrate away from gorm to sqlboiler (botlabs-gg#1658)

* moderation: decouple from configstore

* moderation: add database schemas

* moderation: set up and run sqlboiler

* moderation: migrate to sqlboiler

* web: support null.Int64 in forms and validator

We currently use gorilla/schema to parse HTML form values into structs -- this allows us to specify, e,g., `<input ... name=SomeField>` and have `payload.SomeField` set correctly in Go handlers. This approach mostly works nicely, with the major exception of null.Int64 (formerly sql.NullInt64 with gorm) fields, which show up in the moderation plugin, e.g., Config.DefaultMuteDuration.

Particularly, since gorilla/schema has no native support for null.Int64, we have instead instructed gorilla/schema to directly set the Int64 value using <input ... name=DefaultMuteDuration.Int64> and then manually set DefaultMuteDuration.Valid=true in the web handler, without which the changes are not saved to database. This approach is, for obvious reasons, rather brittle. In this commit we therefore register a custom gorilla/schema decoder for null.Int64, obviating the need for both pointing to the Int64 field and manually setting Valid=true. While at it, we also fix the validator so that it checks null.Int64 fields properly.

* web: support validating types.Int64Array

gorm uses pq.Int64Array but sqlboiler uses types.Int64Array, so we need also support the latter now.

* common/configstore: remove package (botlabs-gg#1668)

* all: remove last traces of gorm (botlabs-gg#1670)

* common: connect to postgres via database/sql

...instead of through gorm.

* common: remove gorm log hook

* common: remove gorm SmallModel

* deps: run go mod tidy

---------

Co-authored-by: Joe L <[email protected]>
ashishjh-bst added a commit to ashishjh-bst/yagpdb that referenced this pull request Jun 27, 2024
…g#1655)

* all: migrate executed command logging away from gorm to sqlboiler (botlabs-gg#1654)

* common: add schemas for executed command logs

* common: configure and run sqlboiler for executed command log model

* all: migrate executed command logging to sqlboiler

* reminders: migrate away from gorm to sqlboiler (botlabs-gg#1652)

* reminders: add database schemas

* reminders: set up and run sqlboiler

* reminders: migrate to sqlboiler

While at it, lightly refactor and simplify the code as applicable. The most substantial (non-database) refactor is renaming and moving strReminders to DisplayReminders.

To aid in the refactor, also add discordgo.ParseID as a shortcut for strconv.ParseInt(...). This operation is often needed to convert string IDs in database to int64's and having a shorter name helps.

* reminders: auto-delete old entries with null guild_id as necessary

* soundboard: remove stray references to gorm (botlabs-gg#1651)

The soundboard module used to use gorm, but was migrated to sqlboiler some years ago in commit 628ea9a. There are still two (erroneous) references to gorm remaining which were not caught since this section of code ignores errors. This commit changes these to use sqlboiler as well.

* schemamigration: skip executing ALTER COLUMN SET NOT NULL queries when possible (botlabs-gg#1650)

* schemamigration: skip executing ALTER COLUMN SET NOT NULL queries when possible

* Use quantifier * not + in regex for consistency

It really should be +, but the current code works and hey -- when in Rome, do as the Romans.

* youtube: migrate away from gorm to sqlboiler (botlabs-gg#1660)

* youtube: add database schemas

* youtube: set up and run sqlboiler

* youtube: migrate to sqlboiler

* notifications: migrate away from gorm to sqlboiler (botlabs-gg#1659)

* notifications: decouple from configstore

* notifications: add database schemas

* notifications: set up and run sqlboiler

* notifications: migrate to sqlboiler

* moderation: migrate away from gorm to sqlboiler (botlabs-gg#1658)

* moderation: decouple from configstore

* moderation: add database schemas

* moderation: set up and run sqlboiler

* moderation: migrate to sqlboiler

* web: support null.Int64 in forms and validator

We currently use gorilla/schema to parse HTML form values into structs -- this allows us to specify, e,g., `<input ... name=SomeField>` and have `payload.SomeField` set correctly in Go handlers. This approach mostly works nicely, with the major exception of null.Int64 (formerly sql.NullInt64 with gorm) fields, which show up in the moderation plugin, e.g., Config.DefaultMuteDuration.

Particularly, since gorilla/schema has no native support for null.Int64, we have instead instructed gorilla/schema to directly set the Int64 value using <input ... name=DefaultMuteDuration.Int64> and then manually set DefaultMuteDuration.Valid=true in the web handler, without which the changes are not saved to database. This approach is, for obvious reasons, rather brittle. In this commit we therefore register a custom gorilla/schema decoder for null.Int64, obviating the need for both pointing to the Int64 field and manually setting Valid=true. While at it, we also fix the validator so that it checks null.Int64 fields properly.

* web: support validating types.Int64Array

gorm uses pq.Int64Array but sqlboiler uses types.Int64Array, so we need also support the latter now.

* common/configstore: remove package (botlabs-gg#1668)

* all: remove last traces of gorm (botlabs-gg#1670)

* common: connect to postgres via database/sql

...instead of through gorm.

* common: remove gorm log hook

* common: remove gorm SmallModel

* deps: run go mod tidy

---------

Co-authored-by: Joe L <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants