Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework ProjectSnapshotManagerDispatcher to avoid creating a dedicated thread #9984

Merged
merged 47 commits into from
Feb 28, 2024
Merged
Changes from 1 commit
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
246db4a
Assert the dispatcher thread in DefaultProjectSnapshotManager
DustinCampbell Feb 23, 2024
b1cc8e4
CSharpVirtualDocumentFactoryTest: Use real dispatcher
DustinCampbell Feb 23, 2024
b79a2ae
Clean up CSharpVirtualDocumentFactoryTest
DustinCampbell Feb 23, 2024
e9cf196
WorkspaceSemanticTokensRefreshTriggerTest: Use real dispatcher
DustinCampbell Feb 23, 2024
bd43542
Clean up WorkspaceSemanticTokensRefreshTriggerTest
DustinCampbell Feb 23, 2024
776fb79
Benchmarks: Use real dispatcher
DustinCampbell Feb 23, 2024
6e04ffb
Remove TestProjectSnapshotManagerDispatcher
DustinCampbell Feb 23, 2024
cbccdcb
RazorProjectInfoPublisherTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
a2d1fa4
Clean up RazorProjectInfoPublisherTest
DustinCampbell Feb 23, 2024
d8e2d94
BackgroundDocumentGeneratorTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
806ce2b
Unskip test
DustinCampbell Feb 23, 2024
38a4cb0
Clean up BackgroundDocumentGeneratorTest
DustinCampbell Feb 23, 2024
6000ada
DefaultProjectSnapshotManagerTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
8e9bd62
Clean up DefaultProjectSnapshotManagerTest
DustinCampbell Feb 23, 2024
89b9d02
OutOfProcTagHelperResolverTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
881404e
Clean up OutOfProcTagHelperResolverTest
DustinCampbell Feb 23, 2024
ba1b78f
VisualStudioFileChangeTrackerTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
4ad44e3
Clean up VisualStudioFileChangeTrackerTest
DustinCampbell Feb 23, 2024
41ad40c
FallbackProjectManagerTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
9efea52
Clean up FallbackProjectManagerTest
DustinCampbell Feb 23, 2024
d4f3fc4
RazorDynamicFileInfoProviderTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
64f850e
Clean up RazorDynamicFileInfoProviderTest
DustinCampbell Feb 23, 2024
5f45f41
ProjectWorkspaceStateGeneratorTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
57c315c
Clean up ProjectWorkspaceStateGeneratorTest
DustinCampbell Feb 23, 2024
d268749
ProjectSnapshotManagerProxyTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
13fefbe
Clean up ProjectSnapshotManagerProxyTest
DustinCampbell Feb 23, 2024
4520cd1
ImportDocumentManagerIntegrationTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
4763864
ImportDocumentManagerTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
23c4e5a
RazorDocumentManagerTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
583ae4a
Clean up RazorDocumentManagerTest
DustinCampbell Feb 23, 2024
c09eace
VisualStudioDocumentTrackerTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
db0560f
GeneratedDocumentPublisherTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
ddea43c
Clean up GeneratedDocumentPublisherTest
DustinCampbell Feb 23, 2024
8d4b9f3
RazorProjectServiceTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
1c3d0c3
Clean up RazorProjectServiceTest
DustinCampbell Feb 23, 2024
76be965
RazorDiagnosticsPublisherTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
a4f828e
DocumentContextFactoryTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
2e8ea76
Clean up DocumentContextFactoryTest
DustinCampbell Feb 23, 2024
a760a34
SnapshotResolverTest: Update to use dispatcher
DustinCampbell Feb 23, 2024
501ffae
Clean up SnapshotResolverTest
DustinCampbell Feb 24, 2024
c7fe0d9
Unify ProjectSnapshotManagerDispatcher implementations on IErrorReporter
DustinCampbell Feb 24, 2024
8fc1d8e
Fix break in ProjectMutationBenchmark
DustinCampbell Feb 24, 2024
f4ce9af
Merge ProjectSnapshotManagerDispatcher and ProjectSnapshotManagerDisp…
DustinCampbell Feb 24, 2024
97852da
WorkspaceProjectStateChangeDetectorTest: Remove odd static dispatcher
DustinCampbell Feb 24, 2024
39976f5
Clean up WorkspaceProjectStateChangeDetectorTest a bit
DustinCampbell Feb 24, 2024
80ea498
Use ThreadPool rather than dedicated dispatcher thread
DustinCampbell Feb 24, 2024
06734a5
Rename dispatcher's RunOnDispatcherAsync methods to RunAsync
DustinCampbell Feb 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -381,26 +381,16 @@ internal override void ReportError(Exception exception, ProjectKey projectKey)

private void NotifyListeners(IProjectSnapshot? older, IProjectSnapshot? newer, string? documentFilePath, ProjectChangeKind kind)
{
// Change notifications should always be sent on the dispatcher.
_dispatcher.AssertDispatcherThread();

NotifyListeners(new ProjectChangeEventArgs(older, newer, documentFilePath, kind, IsSolutionClosing));
}

// virtual so it can be overridden in tests
protected virtual void NotifyListeners(ProjectChangeEventArgs e)
{
// For now, consumers of the Changed event often assume the threaded
// behavior and can error. Once that is fixed we can remove this.
// https://github.com/dotnet/razor/issues/9162
if (_dispatcher.IsDispatcherThread)
{
NotifyListenersCore(e);
}
else
{
_ = _dispatcher.RunOnDispatcherThreadAsync(() =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm slightly concerned with this going away, since this method was originally written without it, and it had to come back due to bugs. Perhaps we can move this to a test accessor somehow, and make the method not protected (or realistically, get rid of it since it is a one-liner), and then we'd know we can trust the assert in the method above?

(also I'm reviewing commit-at-a-time, and this is the first commit, so my whole comment could be moot)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can go away because the only callers assert that they're running on the dispatcher. I've run integration tests and all seems to be working fine.

{
NotifyListenersCore(e);
}, CancellationToken.None);
}
NotifyListenersCore(e);
}

private void NotifyListenersCore(ProjectChangeEventArgs e)
Expand Down Expand Up @@ -449,6 +439,9 @@ private bool TryChangeEntry_UsesLock(
return false;
}

// We're about to mutate, so assert that we're on the dispatcher thread.
_dispatcher.AssertDispatcherThread();

var state = ProjectState.Create(
_projectEngineFactoryProvider,
projectAddedAction.HostProject,
Expand Down Expand Up @@ -485,6 +478,10 @@ private bool TryChangeEntry_UsesLock(
{
oldSnapshot = entry.GetSnapshot();
newSnapshot = newEntry?.GetSnapshot() ?? oldSnapshot;

// We're about to mutate, so assert that we're on the dispatcher thread.
_dispatcher.AssertDispatcherThread();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it doesn't hurt to assert close to the mutation, but given its not an async method, why not just assert once at the top of the method, rather than twice? Also, this whole thing is in a lock, and I thought the lock was added to remove the dispatcher requirement. Does this PR change anything in that regard? Or will we just always have both?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dispatcher is still a requirement until it is removed. Essentially, it's an all or nothing situation. If nothing runs on the dispatcher, mutations are pushed directly onto the project snapshot manager queue. If everything runs on the dispatcher, mutations are pushed onto the dispatcher queue and are pushed onto the project snapshot manager queue later. However, if some mutations go directly to the project snapshot manager and some go to the dispatcher, mutations are likely to get processed in the wrong order because those on the dispatcher will be handled later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The addition of the lock allowed us to retrieve snapshot information without having to use the dispatcher, which was the win there. It looks like this just makes sure mutation happens on the dispatcher thread. That's not technically required from the perspective of the snapshot manager, but is logically required by dependents of snapshot changes. That's the part that will need to be cleaned up before we can completely remove

Copy link
Member Author

@DustinCampbell DustinCampbell Feb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not technically required from the perspective of the snapshot manager.

It might not be required from the project snapshot manager's perspective, but it's required by the current system. Everything that calls mutation methods of the project snapshot manager does so by scheduling the work on the dispatcher. By placing assertions here, I found a race condition where one component (the FallbackProjectManager) was updating the project snapshot manager directly without going through the dispatcher, before other components that had already queued updates on the dispatcher.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The addition of the lock allowed us to retrieve snapshot information without having to use the dispatcher, which was the win there.

Yes! And I really appreciate your work on this! The separation provided by the ReaderWriterLock and the refactoring you did made things so much clearer.


using (upgradeableLock.EnterWriteLock())
{
if (newEntry is null)
Expand Down