We talked about this briefly offline, but in taking a good look at this, I’m not sure I feel especially great about it. That’s NOT a knock on the code you wrote here — the original issue was poorly thought through on my part. I feel bad about that.

My big concern here is safety: we can (and usually do) have multiple import jobs running simultaneously, and the large gap of time between creating a page and then potentially removing it means a removing it the way it’s happening here could potentially break another import that’s happening in parallel. For example:

1. Import job A creates Page X in order to handle Version 1, then
2. Import job B sees Page X and use it to handle Version 2, then
3. Import A might fail to create Version 1 and delete Page X, and then
4. Import B will fail on saving Version 2 (even though it was valid) because the ID of Page X refers to an object that no longer exists.

That’s totally an edge case, but it’s one we don’t have to worry about at all right now, so I’m a little worried about introducing it.

The more I think about it, the transaction idea isn’t ideal either. I think you might be thinking the same, since you already pointed out:

If I changed to a transaction, I'd want to do a bit of reordering in that method so I could open the transaction after the API call so that the open transaction is as brief as possible.

A transaction could cause us to created two records for the same page (while import A is creating a page + version, import B is doing the same).

Another thing we could do while rearranging code here is leverage the fact that version_for_record goes out of its way to avoid persisting the version model it constructs. We could proactively construct the version model and check its validity first (side note: we’d have to temporarily assign it to a dummy page for validations to pass) before bothering to create a page, and never create a page at all if the version would fail to save. That still leaves the super rare scenario of Rails validations succeeding but Postgres constraints failing. We could either just not solve that, or use the transaction to handle it.

Bigger picture, though, all the above is a lot of work and rearranging to handle a rare problem. It would probably make the import code much more prone to accidental breakage in the future, too, because it would be so carefully tip-toeing around a race condition. You’ve already solved the real serious part of this by doing edgi-govdata-archiving/web-monitoring-ui#363, so I think you should only worry about doing all this stuff if you really want to. Let me know whether you plan to continue on this or drop it for now.

Again, sorry my original issue was poorly thought through and set you up for this scenario :(