Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean-up commands table #2

Open
arielmoraes opened this issue Dec 10, 2024 · 2 comments
Open

Clean-up commands table #2

arielmoraes opened this issue Dec 10, 2024 · 2 comments
Assignees

Comments

@arielmoraes
Copy link

When the command is successfully sent its state is updated to Completed. Is it possible to delete the records instead of updating to avoid growing the table?

@mookid8000 mookid8000 self-assigned this Dec 16, 2024
@mookid8000
Copy link
Member

I intentionally made it like this to have some kind of trail of executed commands. But it's pretty clear that it's not sustainable to keep adding commands to the table, so I imagined that one would have some kind of cleanup procedure run once in a while that would execute the "truncate table trick"* and remove finished commands older than some cutoff.

If commands were deleted immediately after being executed, I would be worried that it would be very inefficient, because deleting rows in SQL Server is usually pretty heavy.

What do you think about this?


(*) In a transaction: Selecting rows to keep over into a temp table, truncating the original table, selecting the rows to keep back into the original table, commit! Fast way to remove lots of rows from a table in a situation where most of the rows must be deleted.

@arielmoraes
Copy link
Author

This should work if the number of rows that should be kept is always small (which should be the case), as we were using Rebus before Freakout we had to create our own Outbox solution, whereas we wanted to keep some trail too, the problem is we wanted to keep based on a timespan, ie 30 days. To do the clean-up, we execute a paginated delete job so that we can define the balance between transaction size and speed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants