[Core] Owernership-based Object Directory - enabled by default

[clarkzinzow]

Enable the ownership-based object directory by default.

Remaining work:

• Consolidate location table and reference table: Ownership-based Object Directory - consolidate location table and reference table #13203
• Be resilient to owner failures: [Ownership-based Object Directory] Not resilient to owner failures. #13654
• Remove infinite polling loop from object location subscription: [Core] Owernership-based Object Directory - remove infinite polling loop from object location subscription #12551
• Add support for object spilling (blocker if object spilling is enabled by default): [Core] Ownership-based Object Directory - add support for object spilling #13701
• Add unit tests: [Core] Owernership-based Object Directory - remove GCS-based object directory, add OBOD unit tests #12550
• Confirm that benchmarks show no signs of performance regressions.

Benchmarking plan

1. Test on standard microbenchmarks, fix regressions.
2. Test on workloads that should stress the object directory, such as the multi-node shuffle release test.

[clarkzinzow]

Microbencmarks

No obvious performance regressions on the standard microbenchmarks. Tested on commit c9a9d42.

Before

single client get calls per second 52502.79 +- 502.02
single client put calls per second 37916.51 +- 583.05
multi client put calls per second 91735.1 +- 862.65
single client get calls (Plasma Store) per second 7843.15 +- 27.56
single client put calls (Plasma Store) per second 2800.99 +- 14.16
multi client put calls (Plasma Store) per second 4165.41 +- 77.58
single client put gigabytes per second 13.85 +- 7.25
multi client put gigabytes per second 6.93 +- 0.64
single client tasks sync per second 1281.47 +- 14.06
single client tasks async per second 8988.62 +- 142.92
multi client tasks async per second 10881.14 +- 65.02
1:1 actor calls sync per second 2155.07 +- 108.99
1:1 actor calls async per second 6839.06 +- 115.76
1:1 actor calls concurrent per second 6016.65 +- 80.69
1:n actor calls async per second 11936.93 +- 330.19
n:n actor calls async per second 13193.37 +- 350.18
n:n actor calls with arg async per second 5106.75 +- 62.84
1:1 async-actor calls sync per second 1534.55 +- 35.14
1:1 async-actor calls async per second 3622.73 +- 36.27
1:1 async-actor calls with args async per second 2527.81 +- 30.93
1:n async-actor calls async per second 7156.61 +- 71.37
n:n async-actor calls async per second 7971.14 +- 142.77
client: get calls per second 1744.45 +- 18.85
client: put calls per second 1027.22 +- 15.46
client: remote put calls per second 4162.08 +- 29.28
client: 1:1 actor calls sync per second 544.51 +- 6.11
client: 1:1 actor calls async per second 574.51 +- 7.65
client: 1:1 actor calls concurrent per second 584.16 +- 1.02

After

single client get calls per second 51396.49 +- 704.03
single client put calls per second 39866.91 +- 779.77
multi client put calls per second 92103.79 +- 1045.44
single client get calls (Plasma Store) per second 7751.17 +- 284.5
single client put calls (Plasma Store) per second 2949.25 +- 14.19
multi client put calls (Plasma Store) per second 4219.19 +- 46.06
single client put gigabytes per second 13.96 +- 7.22
multi client put gigabytes per second 6.73 +- 0.71
single client tasks sync per second 1283.59 +- 9.76
single client tasks async per second 8830.53 +- 67.72
multi client tasks async per second 10805.59 +- 163.89
1:1 actor calls sync per second 2216.58 +- 24.64
1:1 actor calls async per second 6781.82 +- 122.73
1:1 actor calls concurrent per second 6011.66 +- 64.84
1:n actor calls async per second 12332.01 +- 540.22
n:n actor calls async per second 13061.85 +- 187.92
n:n actor calls with arg async per second 5126.7 +- 74.11
1:1 async-actor calls sync per second 1537.02 +- 11.16
1:1 async-actor calls async per second 3544.57 +- 82.38
1:1 async-actor calls with args async per second 2520.55 +- 20.46
1:n async-actor calls async per second 7138.91 +- 70.26
n:n async-actor calls async per second 8421.67 +- 230.56
client: get calls per second 1810.06 +- 6.87
client: put calls per second 1025.82 +- 14.59
client: remote put calls per second 4096.26 +- 23.18
client: 1:1 actor calls sync per second 525.07 +- 19.39
client: 1:1 actor calls async per second 596.13 +- 6.23
client: 1:1 actor calls concurrent per second 588.01 +- 9.14