Store only the resolved reference in DocumentationContext.symbolIndex

[d-ronnqvist]

Bug/issue #, if applicable: rdar://106654403

Summary

This changes DocumentationContext.symbolIndex from a lookup that return DocumentationNode values to a lookup that return ResolvedTopicReference values.

Storing all nodes both in the documentation cache and in the symbol index makes it easy to accidentally hold on to old versions of nodes if the value in the documentation cache is updated without also updating the value in the symbol index. By instead only storing the node's resolved reference in the symbol cache and using it to find the node in the documentation cache, various places in the code that modify documentation nodes only need to update the value in one place.

This makes up for the previous peak memory usage increase from #498 (comment).

Dependencies

None.

Testing

None. This is a non-functional change. Everything should continue to work the same as before.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• Added tests
• Ran the ./bin/test script and it succeeded
• Updated documentation if necessary

[d-ronnqvist]

In my local benchmarking across two different machines this lowers peak memory usage by ~1.5% while also making small improvements to the total covert time:

┌──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Metric                                   │ Change          │ main                 │ current              │
├──────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Duration for 'bundle-registration'       │ -1.184%¹        │ 8.723 sec            │ 8.619 sec            │
│ Duration for 'convert-total-time'        │ -0.6942%²       │ 14.529 sec           │ 14.428 sec           │
│ Duration for 'documentation-processing'  │ no change³      │ 5.376 sec            │ 5.377 sec            │
│ Duration for 'finalize-navigation-index' │ no change⁴      │ 0.228 sec            │ 0.229 sec            │
│ Peak memory footprint                    │ -1.412%⁵        │ 1,019.1 MB           │ 1,004.7 MB           │
│ Data subdirectory size                   │ no change       │ 280.8 MB             │ 280.8 MB             │
│ Index subdirectory size                  │ no change       │ 2.6 MB               │ 2.6 MB               │
│ Total DocC archive size                  │ no change       │ 401.3 MB             │ 401.3 MB             │
│ Topic Anchor Checksum                    │ no change       │ a8c40d797627683c5d7c │ a8c40d797627683c5d7c │
│ Topic Graph Checksum                     │ no change       │ 665713509084b5131a37 │ 665713509084b5131a37 │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────┘

 1: There's a statistically significant difference between the two benchmark measurements.
    The values are different enough that the most probable explanation is that they're random samples from two different data sets.
    t-statistic : +3.770          degrees of freedom : 28       95% confidence critical value : +2.042

 2: There's a statistically significant difference between the two benchmark measurements.
    The values are different enough that the most probable explanation is that they're random samples from two different data sets.
    t-statistic : +2.143          degrees of freedom : 28       95% confidence critical value : +2.042

 3: No statistically significant difference.
    The values are similar enough that the most probable explanation is that they're random samples from the same data set.
    t-statistic : -0.026          degrees of freedom : 28       95% confidence critical value : +2.042

 4: No statistically significant difference.
    The values are similar enough that the most probable explanation is that they're random samples from the same data set.
    t-statistic : -0.443          degrees of freedom : 28       95% confidence critical value : +2.042

 5: There's a statistically significant difference between the two benchmark measurements.
    The values are different enough that the most probable explanation is that they're random samples from two different data sets.
    t-statistic : +9.061          degrees of freedom : 28       95% confidence critical value : +2.042

[d-ronnqvist]

@swift-ci please test

[franklinsch]

Nice find! Is there a test that we can write that asserts that the references from the different stores are the consistent? Similar to https://github.com/apple/swift-docc/blob/c74237d83e542864033e0194b8f4bdf553c12485/Tests/SwiftDocCTests/Infrastructure/DocumentationContext/DocumentationContextTests.swift#L3282

[d-ronnqvist]

Is there a test that we can write that asserts that the references from the different stores are the consistent?

Good idea. I think that'd be easy to test with one of the test bundles.

[d-ronnqvist]

@swift-ci please test