Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Encode spans relative to the enclosing item #443

Closed
1 of 3 tasks
cjgillot opened this issue Jun 29, 2021 · 3 comments
Closed
1 of 3 tasks

Encode spans relative to the enclosing item #443

cjgillot opened this issue Jun 29, 2021 · 3 comments
Labels
major-change A proposal to make a major change to rustc major-change-accepted A major change proposal that was accepted T-compiler Add this label so rfcbot knows to poll the compiler team

Comments

@cjgillot
Copy link

Proposal

Right now, spans are hashed for incremental compilation using their byte offset in their respective file. As a consequence, each time a function is modified, all the code following it in the same file must be recompiled.

The aim of this MCP is to avoid recomputing queries when code is moved without modification.

This is achieved by :

  1. storing the HIR owner LocalDefId information inside the span;
  2. encoding and decoding spans relative to the enclosing item in the incremental on-disk cache;
  3. creating a source_span(LocalDefId) returning the absolute span;
  4. marking a dependency to the source_span(LocalDefId) query when we translate a span from the short (Span) representation to its explicit (SpanData) representation.

Since all client code uses Span, step 3 ensures that all manipulations of span byte positions actually create the dependency edge between the caller and the source_span(LocalDefId). As this query returns the actual absolute span of the parent item, any source code motion that changes the absolute byte position of a node will either:

  • modify the distance to the parent's beginning, so change the relative span's hash;
  • dirty source_span, and trigger the incremental recomputation of all code that depends on the span's absolute byte position.

With this scheme, I believe the dependency tracking to be accurate.

Mentors or Reviewers

@petrochenkov started to review the first PR.

Process

The main points of the Major Change Process are as follows:

  • File an issue describing the proposal.
  • A compiler team member or contributor who is knowledgeable in the area can second by writing @rustbot second.
    • Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a -C flag, then full team check-off is required.
    • Compiler team members can initiate a check-off via @rfcbot fcp merge on either the MCP or the PR.
  • Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.

You can read more about Major Change Proposals on forge.

Comments

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.
@cjgillot cjgillot added major-change A proposal to make a major change to rustc T-compiler Add this label so rfcbot knows to poll the compiler team labels Jun 29, 2021
@rustbot
Copy link
Collaborator

rustbot commented Jun 29, 2021

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

cc @rust-lang/compiler @rust-lang/compiler-contributors

@rustbot rustbot added the to-announce Announce this issue on triage meeting label Jun 29, 2021
@cjgillot cjgillot mentioned this issue Jun 29, 2021
Merged
@apiraino apiraino removed the to-announce Announce this issue on triage meeting label Jul 1, 2021
@estebank
Copy link

@rustbot second

@rustbot rustbot added the final-comment-period The FCP has started, most (if not all) team members are in agreement label Jul 18, 2021
@apiraino
Copy link
Contributor

@rustbot label -final-comment-period +major-change-accepted

@rustbot rustbot added major-change-accepted A major change proposal that was accepted to-announce Announce this issue on triage meeting and removed final-comment-period The FCP has started, most (if not all) team members are in agreement labels Jul 31, 2021
@apiraino apiraino removed the to-announce Announce this issue on triage meeting label Aug 5, 2021
bors added a commit to rust-lang-ci/rust that referenced this issue Sep 11, 2021
@bors
Auto merge of rust-lang#84373 - cjgillot:resolve-span, r=michaelwoeri…
547d937 
…ster,petrochenkov

Encode spans relative to the enclosing item

The aim of this PR is to avoid recomputing queries when code is moved without modification.

MCP at rust-lang/compiler-team#443

This is achieved by :
1. storing the HIR owner LocalDefId information inside the span;
2. encoding and decoding spans relative to the enclosing item in the incremental on-disk cache;
3. marking a dependency to the `source_span(LocalDefId)` query when we translate a span from the short (`Span`) representation to its explicit (`SpanData`) representation.

Since all client code uses `Span`, step 3 ensures that all manipulations
of span byte positions actually create the dependency edge between
the caller and the `source_span(LocalDefId)`.
This query return the actual absolute span of the parent item.
As a consequence, any source code motion that changes the absolute byte position of a node will either:
- modify the distance to the parent's beginning, so change the relative span's hash;
- dirty `source_span`, and trigger the incremental recomputation of all code that
  depends on the span's absolute byte position.

With this scheme, I believe the dependency tracking to be accurate.

For the moment, the spans are marked during lowering.
I'd rather do this during def-collection,
but the AST MutVisitor is not practical enough just yet.
The only difference is that we attach macro-expanded spans
to their expansion point instead of the macro itself.
flip1995 pushed a commit to flip1995/rust-clippy that referenced this issue Sep 28, 2021
@bors
Auto merge of #84373 - cjgillot:resolve-span, r=michaelwoerister,petr…
e6455ea 
…ochenkov

Encode spans relative to the enclosing item

The aim of this PR is to avoid recomputing queries when code is moved without modification.

MCP at rust-lang/compiler-team#443

This is achieved by :
1. storing the HIR owner LocalDefId information inside the span;
2. encoding and decoding spans relative to the enclosing item in the incremental on-disk cache;
3. marking a dependency to the `source_span(LocalDefId)` query when we translate a span from the short (`Span`) representation to its explicit (`SpanData`) representation.

Since all client code uses `Span`, step 3 ensures that all manipulations
of span byte positions actually create the dependency edge between
the caller and the `source_span(LocalDefId)`.
This query return the actual absolute span of the parent item.
As a consequence, any source code motion that changes the absolute byte position of a node will either:
- modify the distance to the parent's beginning, so change the relative span's hash;
- dirty `source_span`, and trigger the incremental recomputation of all code that
  depends on the span's absolute byte position.

With this scheme, I believe the dependency tracking to be accurate.

For the moment, the spans are marked during lowering.
I'd rather do this during def-collection,
but the AST MutVisitor is not practical enough just yet.
The only difference is that we attach macro-expanded spans
to their expansion point instead of the macro itself.
calebcartwright pushed a commit to calebcartwright/rustfmt that referenced this issue Oct 20, 2021
@bors
Auto merge of #84373 - cjgillot:resolve-span, r=michaelwoerister,petr…
8743472 
…ochenkov

Encode spans relative to the enclosing item

The aim of this PR is to avoid recomputing queries when code is moved without modification.

MCP at rust-lang/compiler-team#443

This is achieved by :
1. storing the HIR owner LocalDefId information inside the span;
2. encoding and decoding spans relative to the enclosing item in the incremental on-disk cache;
3. marking a dependency to the `source_span(LocalDefId)` query when we translate a span from the short (`Span`) representation to its explicit (`SpanData`) representation.

Since all client code uses `Span`, step 3 ensures that all manipulations
of span byte positions actually create the dependency edge between
the caller and the `source_span(LocalDefId)`.
This query return the actual absolute span of the parent item.
As a consequence, any source code motion that changes the absolute byte position of a node will either:
- modify the distance to the parent's beginning, so change the relative span's hash;
- dirty `source_span`, and trigger the incremental recomputation of all code that
  depends on the span's absolute byte position.

With this scheme, I believe the dependency tracking to be accurate.

For the moment, the spans are marked during lowering.
I'd rather do this during def-collection,
but the AST MutVisitor is not practical enough just yet.
The only difference is that we attach macro-expanded spans
to their expansion point instead of the macro itself.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
major-change A proposal to make a major change to rustc major-change-accepted A major change proposal that was accepted T-compiler Add this label so rfcbot knows to poll the compiler team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@estebank @cjgillot @apiraino @rustbot