Improve DB indexes for better worst case accounting time

[tolikzinovyev]

Summary

Closes https://github.com/algorand/go-algorand-internal/issues/1718.

Test Plan

Ran block generator. No noticeable difference in performance.

[tolikzinovyev]

There is already a non-partial index on addr - the primary key.

[codecov-commenter]

Codecov Report

Merging #760 (5194422) into develop (04a0dcf) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #760   +/-   ##
========================================
  Coverage    55.20%   55.20%           
========================================
  Files           26       26           
  Lines         3817     3817           
========================================
  Hits          2107     2107           
  Misses        1435     1435           
  Partials       275      275           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04a0dcf...5194422. Read the comment docs.

[gmalouf]

Question: Do we need to manage migration for existing deployments, or are we just assuming no backwards compatibility with upcoming release / simply deleting references to pre-existing indexes? cc @winder

I.E. Some of these indexes being updated it seems we would want runners to drop before applying the new ones, yet we give no on/off ramp to do this at first glance.

[tolikzinovyev]

The coming "refactoring" release will no backward compatibility with the previous ones. The users will have to create a new database and do catch up from round 0. This is simply because 2.6.* has accounting problems, and 2.7 will in all likelihood crash if we try using a 2.6 database. Hence, we should use the opportunity to make any changes that would require a migration if we were to support backward compatibility.

[gmalouf]

I'm approving, but have we run tests with these indexes at least locally to show the performance improvement?

[gmalouf]

We need to keep in mind that not everyone uses the same indexes, although these in particular -should- be universal. We'll need to work with release eng to communicate the changes in existing ones for people who pick and choose which ones they apply today..

[winder]

Seems fine to me.

[tolikzinovyev]

I'm approving, but have we run tests with these indexes at least locally to show the performance improvement?

The performance is asymptotically better in the worst case (when the number of deleted items is much larger than the number of non-deleted items). I'm not sure testing performance is even necessary.