Perform batch deletion of invalid records before applying new foreign keys

[zackgalbreath]

Perform batch deletion of invalid records before applying new foreign keys

[williamjallen]

Is there a reason why doing this is preferable to simply leaving the migrations as-is?

Two concerns here:

1. Broadly speaking, modifying old migrations adds additional risk, even if the changes are small. If there isn't a meaningful reason why editing a given migration is needed (e.g., a critical failure or performance issue), I don't like the idea of introducing additional risks.
2. One of the biggest issues with containerized deployments of CDash is the fact that deploying an older version does not automatically roll back newer migrations, or give the option to do so manually. One solution to this problem used by some migration systems is to store the text of the migration in the migrations table itself, so older versions of CDash can roll back newer migrations. If we ever want to do that, compatibility with code from outside the migration will be a major concern. As such, I think it's better to keep all of the relevant code (excluding a handful of library functions from Laravel) contained within each migration file whenever possible. Even if we don't end up storing migration logic in the database, there is still the issue of not wanting to edit old migrations if the functions being referenced are ever refactored.

[zackgalbreath]

Is there a reason why doing this is preferable to simply leaving the migrations as-is?

Broadly speaking, the motivation behind this proposed change is to improve the performance of these recent migrations. In particular, the buildfailure2argument table is quite large on some of the production instances we support (> 100 million rows).

[williamjallen]

What do you think about copying and pasting the code into each of the migrations so they are independent of the rest of the codebase? It feels wrong to do in principle, but I think it's acceptable given that migrations are essentially one-time use code snippets. I could be convinced otherwise if you have strong opinions on this though.