Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin solutions when doing fix all operation #72909

Merged
merged 13 commits into from
Apr 7, 2024

Conversation

CyrusNajmabadi
Copy link
Member

@CyrusNajmabadi CyrusNajmabadi commented Apr 6, 2024

Previously fix-all would do thingsl ike the following:

  1. compute the initial set of doc changes to make to all the files in the solution
  2. apply those changes, then go over the solution again to 'clean' the files (format/simplify/etc.)
  3. this involves then asking for things like compilations for a project, whch then involves syncing over to OOP to do things like run generators.
  4. However, this is all operating on a forked solution. So each time we do this, we'll sync that forked solution over to oop.

This is normally not a problem, but it opens ourselves up to an issue (which i repro'ed) when the operation is very long running. Namely, we sync the solution over, do an op on OOP with it, then come back. Before the next operation calls out to oop, oop drops that solution. So on the next call, we have to sync it again. this is normally cheap in terms of the sync, but it means we get a fresh solution instance on the oop side. This fresh solution will not have the cached compilations from the last call, causing them to be recreated. With SG and skeletons, this can eb super expensive.

The fix here, before doing very many operations on the same solution, is to pin that solution on the remote side. That way, it won't get dropped between calls, and we can benefit from all teh compilations and wahtnot staying cached.

This shaved off a couple of minutes doing a 'fix all in solution' change that i was trying out.

@dotnet-issue-labeler dotnet-issue-labeler bot added Area-IDE untriaged Issues and PRs which have not yet been triaged by a lead labels Apr 6, 2024
@@ -93,7 +93,7 @@ private partial class SuggestedActionsSource : IAsyncSuggestedActionsSource
// especially important as we are sending disparate requests for diagnostics, and we do not want the
// individual diagnostic requests to redo all the work to run source generators, create skeletons,
// etc.
using var _1 = RemoteKeepAliveSession.Create(document.Project.Solution, _listener);
using var _1 = await RemoteKeepAliveSession.CreateAsync(document.Project.Solution, cancellationToken).ConfigureAwait(false);
Copy link
Member Author

@CyrusNajmabadi CyrusNajmabadi Apr 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the async entrypoint is nicer for async clients, ensuring that we don't proceed until actually makign the remote host client, and issuing the keepalive call. it also means an async caller doesn't need access to an async listener.

@@ -20,26 +20,31 @@ namespace Microsoft.CodeAnalysis.CodeFixesAndRefactorings;
/// </summary>
internal static class DefaultFixAllProviderHelpers
{
public static async Task<CodeAction?> GetFixAsync<TFixAllContext>(
public static CodeAction GetFix<TFixAllContext>(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

view with whitespace off.

@CyrusNajmabadi CyrusNajmabadi marked this pull request as ready for review April 6, 2024 21:53
@CyrusNajmabadi CyrusNajmabadi requested a review from a team as a code owner April 6, 2024 21:53
@CyrusNajmabadi
Copy link
Member Author

@dibarbet @mavasani ptal

/// <summary>
/// Gets a new <see cref="FixAllContext"/> with the given cancellationToken.
/// </summary>
public FixAllContext WithCancellationToken(CancellationToken cancellationToken)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved into the generic With helper below.

@@ -39,7 +45,7 @@ internal static class DefaultFixAllProviderHelpers
return null;

return CodeAction.Create(
title, c => Task.FromResult(solution));
title, _ => Task.FromResult(solution));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mavasani Aside: why do we do all teh expensive comptuation above, instead of doing it in the async callback passed to CodeAction.Create?

@@ -40,32 +42,52 @@ internal static class DocumentBasedFixAllProviderHelpers
var workItemCount = fixAllKind == FixAllKind.CodeFix ? 3 : 2;
progressTracker.AddItems(fixAllContexts.Length * workItemCount);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file is hte meat of hte change.

// First, iterate over all contexts, and collect all the changes for each of them. We'll be making a lot of
// calls to the remote server to compute diagnostics and changes. So keep a single connection alive to it
// so we never resync or recompute anything.
using var _2 = await RemoteKeepAliveSession.CreateAsync(solution, originalFixAllContext.CancellationToken).ConfigureAwait(false);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the first large scale operation we do. where we process the entire initial solution to determine the first pass of documents changed. for this, we want things pinned so nothing drops between local and oop.

currentSolution = newRoot != null
? currentSolution.WithDocumentSyntaxRoot(docId, newRoot)
: currentSolution.WithDocumentText(docId, newText!);
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we then go and form the next full solution, with all those initial changes applied.

// We're about to making a ton of calls to this new solution, including expensive oop calls to get up to
// date compilations, skeletons and SG docs. Create and pin this solution so that all remote calls operate
// on the same fork and do not cause the forked solution to be created and dropped repeatedly.
using var _2 = await RemoteKeepAliveSession.CreateAsync(currentSolution, originalFixAllContext.CancellationToken).ConfigureAwait(false);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we then do a final cleanup pass on that next-solution. this will again be a ton of work, with lots of oop calls. so this also needs to pin the solution during this time.

{
// First, compute and apply the fixes.
var docIdToNewRootOrText = await getFixedDocumentsAsync(fixAllContext, progressTracker).ConfigureAwait(false);
// TODO: consider computing this in parallel.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// TODO: consider computing this in parallel.

Seems like a good idea, have you measured on your really slow scenario if this would help a lot?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i will, but i'd like to put in another PR :)

Copy link
Contributor

@ToddGrun ToddGrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@CyrusNajmabadi CyrusNajmabadi merged commit 951f5e6 into dotnet:main Apr 7, 2024
27 checks passed
@dotnet-policy-service dotnet-policy-service bot added this to the Next milestone Apr 7, 2024
@CyrusNajmabadi CyrusNajmabadi deleted the pinRemote branch April 7, 2024 21:45
@CyrusNajmabadi
Copy link
Member Author

@jasonmalinowski For review when you get back.

@dibarbet dibarbet modified the milestones: Next, 17.11 P1 Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-IDE untriaged Issues and PRs which have not yet been triaged by a lead
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants