Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal: Build source map for hir_def::TypeRefs #18074

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ChayimFriedman2
Copy link
Contributor

@ChayimFriedman2 ChayimFriedman2 commented Sep 8, 2024

So that given a TypeRef we will be able to trace it back to source code.

This is necessary to be able to provide diagnostics for lowering to chalk tys, since the input to that is TypeRef. This is the first step towards diagnostics for ty lowering. The next steps will be smaller :P

This means that TypeRefs now have an identity, which means storing them in arena and not interning them, which is an unfortunate (but necessary) loss but also a pretty massive change. Luckily, because of the separation layer we have for IDE and HIR, this change never crosses the IDE boundary.

This gives an unfortunate regression of ~170mb in analysis-stats ..

Sorry for the size of the PR; I did some extra things but they are trivial - most of the code is necessary and impossible to split.

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 8, 2024
@ChayimFriedman2 ChayimFriedman2 marked this pull request as draft September 8, 2024 19:28
@bors
Copy link
Collaborator

bors commented Sep 11, 2024

☔ The latest upstream changes (presumably #18075) made this pull request unmergeable. Please resolve the merge conflicts.

@ChayimFriedman2 ChayimFriedman2 force-pushed the typeref-source-map branch 2 times, most recently from 9af921a to 3c34665 Compare September 12, 2024 21:44
@ChayimFriedman2 ChayimFriedman2 marked this pull request as ready for review September 12, 2024 21:44
@ChayimFriedman2
Copy link
Contributor Author

Okay, this is ready for review.

I intend to submit a follow-up PR with few cleanups that I don't want to insert here to not increase the size of this already-large PR.

@ChayimFriedman2 ChayimFriedman2 changed the title Build source map for hir_def::TypeRefs internal: Build source map for hir_def::TypeRefs Sep 12, 2024
@Veykril
Copy link
Member

Veykril commented Sep 25, 2024

This gives an unfortunate regression of ~170mb in analysis-stats ..

Whew that's quite a bit 😞 Wonder if we can shrink TypeRef some more (its 40 bytes on master)

crates/hir-def/src/db.rs Show resolved Hide resolved
}

/// A single generic argument.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum GenericArg {
Type(TypeRef),
Type(TypeRefId),
Lifetime(LifetimeRef),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside: I imagine we might wanna give lifetimes the same treatment (given they also only appear in type position anyways)

@Veykril
Copy link
Member

Veykril commented Sep 25, 2024

I guess part of the memory increase comes from the new source map queries not being LRU'd. Gonna think a bit aabout whether we can reduce the memory impact of this change to some degree (aside from LRUing those queries)

@lnicola
Copy link
Member

lnicola commented Sep 25, 2024

Can we make block_item_tree etc. transparent to save some memory? Because block_item_tree_query just calls block_item_tree_with_source_map now.

@Veykril
Copy link
Member

Veykril commented Sep 25, 2024

The split exists for incrementality purposes, and making that one transparent won't really save much given it just clones the Arc

@ChayimFriedman2
Copy link
Contributor Author

If anything, it's the opposite: we can make the source maps query transparent (not just this one), since everyone that wants the source map is not perf-sensitive (IDE layer).

@ChayimFriedman2
Copy link
Contributor Author

I guess part of the memory increase comes from the new source map queries not being LRU'd

What is the difference between LRU'd and not LRU'd queries?

@ChayimFriedman2
Copy link
Contributor Author

I'll try to find how each type consumes memory, and as a result of this analysis what can we do to shrink it.

@Veykril
Copy link
Member

Veykril commented Sep 25, 2024

I guess part of the memory increase comes from the new source map queries not being LRU'd

What is the difference between LRU'd and not LRU'd queries?

LRU'd ones will just have their caches evicted if they overflow the LRU limit (up to the limit).

@ChayimFriedman2
Copy link
Contributor Author

@Veykril Adding #[salsa::lru] around the source map queries does not help analysis-stats . at all (neither it makes it worse, though). I'll explore other ways. Should I still LRU them?

So that given a `TypeRef` we will be able to trace it back to source code.

This is necessary to be able to provide diagnostics for lowering to chalk tys, since the input to that is `TypeRef`.

This means that `TypeRef`s now have an identity, which means storing them in arena and not interning them, which is an unfortunate (but necessary) loss but also a pretty massive change. Luckily, because of the separation layer we have for IDE and HIR, this change never crosses the IDE boundary.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants