-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DO NOT MERGE] Rebase to v2.46.0 #5062
Conversation
Typically, forcing a sparse index to expand to a full index means that Git could not determine the status of a file outside of the sparse-checkout and needed to expand sparse trees into the full list of sparse blobs. This operation can be very slow when the sparse-checkout is much smaller than the full tree at HEAD. When users are in this state, there is usually a modified or untracked file outside of the sparse-checkout mentioned by the output of 'git status'. There are a number of reasons why this is insufficient: 1. Users may not have a full understanding of which files are inside or outside of their sparse-checkout. This is more common in monorepos that manage the sparse-checkout using custom tools that map build dependencies into sparse-checkout definitions. 2. In some cases, an empty directory could exist outside the sparse-checkout and these empty directories are not reported by 'git status' and friends. 3. If the user has '.gitignore' or 'exclude' files, then 'git status' will squelch the warnings and not demonstrate any problems. In order to help users who are in this state, add a new advice message to indicate that a sparse index is expanded to a full index. This message should be written at most once per process, so add a static global 'give_advice_on_expansion' to sparse-index.c. Further, there is a case in 'git sparse-checkout set' that uses the sparse index as an in-memory data structure (even when writing a full index) so we need to disable the message in that kind of case. The t1092-sparse-checkout-compatibility.sh test script compares the behavior of several Git commands across full and sparse repositories, including sparse repositories with and without a sparse index. We need to disable the advice in the sparse-index repo to avoid differences in stderr. By leaving the advice on in the sparse-checkout repo (without the sparse index), we can test the behavior of disabling the advice in convert_to_sparse(). (Indeed, these tests are how that necessity was discovered.) Add a test that reenables the advice and demonstrates that the message is output. The advice message is defined outside of expand_index() to avoid super- wide lines. It is also defined as a macro to avoid compile issues with -Werror=format-security. Signed-off-by: Derrick Stolee <[email protected]> Signed-off-by: Junio C Hamano <[email protected]>
This reverts commit b7d6f23. This commit caused a regression in the incremental-repack maintenance step where not all objects in a pack-file are repacked, thus 'git multi-pack-index expire' cannot delete the pack-files. This will require some careful explanation upstream and some careful untangling of how server-side repacks may be depending on the new logic. Since microsoft/git only serves client users, it is safe to revert this in isolation for now. Signed-off-by: Derrick Stolee <[email protected]>
These seem to be custom tests to microsoft/git as they break without these changes, but these changes are not needed upstream. Signed-off-by: Derrick Stolee <[email protected]>
This topic reverts a regression in v2.45.x where it no longer expires multi-pack indices correctly, resulting in way too many packs that cannot be expired. This work-around will be upstreamed at a later date. Signed-off-by: Johannes Schindelin <[email protected]>
This backports the `ds/advice-sparse-index-expansion` patches into `microsoft/git` which _just_ missed the v2.46.0 window. Signed-off-by: Johannes Schindelin <[email protected]>
Start work on a new `git survey` command to scan the repository for monorepo performance and scaling problems. The goal is to measure the various known "dimensions of scale" and serve as a foundation for adding additional measurements as we learn more about Git monorepo scaling problems. Results will be logged to the console and to Trace2. The initial goal is to complement the scanning and analysis performed by the GO-based `git-sizer` (https://github.com/github/git-sizer) tool. It is hoped that by creating a builtin command, we may be able to take advantage of internal Git data structures and code that is not accessible from GO to gain further insight into potential scaling problems. Signed-off-by: Jeff Hostetler <[email protected]>
By default we will scan all references in "refs/heads/", "refs/tags/" and "refs/remotes/". Add command line opts let the use ask for all refs or a subset of them and to include a detached HEAD. Signed-off-by: Jeff Hostetler <[email protected]>
Collect the set of requested branches, tags, and etc into a ref_array and collect the set of requested patterns into a strvec. Signed-off-by: Jeff Hostetler <[email protected]>
Calculate stats on the set of refs. This includes the number of branches, plain and annotated tags, remotes, and etc. Calculate the number of packed vs loose refs. Calculate the size of the set of refnames. Print results on the console in JSON format. Add Trace2 logging of the results as a data_json event. Signed-off-by: Jeff Hostetler <[email protected]>
Add treewalk on the commits and objects reachable from the set of refs. This commit sets up the treewalk, but only stubs in the traverse callbacks. We'll actually look at the commit and object data in the next commit. Signed-off-by: Jeff Hostetler <[email protected]>
Add callback to handle commit objects during the treewalk. Count the number of commits and group them by the number of parents. Signed-off-by: Jeff Hostetler <[email protected]>
Create `struct large_item` and `struct large_item_vec` to capture the n largest commits, trees, and blobs under various scaling dimensions, such as size in bytes, number of commit parents, or number of entries in a tree. Each of these have a command line option to set them independently. Signed-off-by: Jeff Hostetler <[email protected]>
Include the pathname of each blob or tree in the large_item_vec to help identify the file or directory associated with the OID and size information. This pathname is computed during the treewalk, so it reflects the first observed pathname seen for that OID during the traversal over all of the refs. Since the file or directory could have moved (without being modified), there may be multiple "correct" pathnames for a particular OID. Since we do not control the ref traversal order, we should consider it to be a "suggested pathname" for the OID. Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
Computing `git name-rev` on each commit, tree, and blob in each of the various large_item_vec can be very expensive if there are too many refs, especially if the user doesn't need the result. Lets make it optional. The `--no-name-rev` option can save 50 calls to `git name-rev` since we have 5 large_item_vec's and each defaults to 10 items. Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
While using the reset --stdin feature on windows path added may have a \r at the end of the path that wasn't getting removed so didn't match the path in the index and wasn't reset. Signed-off-by: Kevin Willford <[email protected]>
It has been a long-standing practice in Git for Windows to append `.windows.<n>`, and in microsoft/git to append `.vfs.0.0`. Let's keep doing that. Signed-off-by: Johannes Schindelin <[email protected]>
Since we really want to be based on a `.vfs.*` tag, let's make sure that there was a new-enough one, i.e. one that agrees with the first three version numbers of the recorded default version. This prevents e.g. v2.22.0.vfs.0.<some-huge-number>.<commit> from being used when the current release train was not yet tagged. It is important to get the first three numbers of the version right because e.g. Scalar makes decisions depending on those (such as assuming that the `git maintenance` built-in is not available, even though it actually _is_ available). Signed-off-by: Johannes Schindelin <[email protected]>
This header file will accumulate GVFS-specific definitions. Signed-off-by: Kevin Willford <[email protected]>
This does not do anything yet. The next patches will add various values for that config setting that correspond to the various features offered/required by GVFS. Signed-off-by: Kevin Willford <[email protected]> gvfs: refactor loading the core.gvfs config value This code change makes sure that the config value for core_gvfs is always loaded before checking it. Signed-off-by: Kevin Willford <[email protected]>
This topic branch brings in a new, experimental built-in command to assess the dimensions of a local repository. It is experimental and subject to change! It might grow new options, change its output, or even be moved into `git diagnose --analyze` or something like that. The hope is that this command, which was inspired by `git sizer` (https://github.com/github/git-sizer), will be helpful not only in diagnosing issues with large repositories, but also in modeling what shapes and sizes of repositories can be handled by Git (and as a corollary: where Git needs to improve to be able to accommodate the natural growth of repositories). Signed-off-by: Johannes Schindelin <[email protected]>
This takes a substantial amount of time, and if the user is reasonably sure that the files' integrity is not compromised, that time can be saved. Git no longer verifies the SHA-1 by default, anyway. Signed-off-by: Kevin Willford <[email protected]> Update for 2023-02-27: This feature was upstreamed as the index.skipHash config option. This resulted in some changes to the struct and some of the setup code. In particular, the config reading was moved to prepare_repo_settings(), so the core.gvfs bit check was moved there, too. Signed-off-by: Derrick Stolee <[email protected]>
While this command is definitely something we _want_, chances are that upstreaming this will require substantial changes. We still want to be able to experiment with this before that, to focus on what we need out of this command: To assist with diagnosing issues with large repositories, as well as to help monitoring the growth and the associated painpoints of such repositories. To that end, we are about to integrate this command into `microsoft/git`, to get the tool into the hands of users who need it most, with the idea to iterate in close collaboration between these users and the developers familar with Git's internals. However, we will definitely want to avoid letting anybody have the impression that this command, its exact inner workings, as well as its output format, are anywhere close to stable. To make that fact utterly clear (and thereby protect the freedom to iterate and innovate freely before upstreaming the command), let's mark its output as experimental in all-caps, as the first thing we do. Signed-off-by: Johannes Schindelin <[email protected]>
…x-built-in-fsmonitor Fix the built-in FSMonitor, and run Scalar's Functional Tests as part of the automated builds
This is random stuff that probably all got upstream in the meantime.
When scripts or background maintenance wish to perform HTTP(S) requests, there is a risk that our stored credentials might be invalid. At the moment, this causes the credential helper to ping the user and block the process. Even if the credential helper does not ping the user, Git falls back to the 'askpass' method, which includes a direct ping to the user via the terminal. Even setting the 'core.askPass' config as something like 'echo' will causes Git to fallback to a terminal prompt. It uses git_terminal_prompt(), which finds the terminal from the environment and ignores whether stdin has been redirected. This can also block the process awaiting input. Create a new config option to prevent user interaction, favoring a failure to a blocked process. The chosen name, 'credential.interactive', is taken from the config option used by Git Credential Manager to already avoid user interactivity, so there is already one credential helper that integrates with this option. However, older versions of Git Credential Manager also accepted other string values, including 'auto', 'never', and 'always'. The modern use is to use a boolean value, but we should still be careful that some users could have these non-booleans. Further, we should respect 'never' the same as 'false'. This is respected by the implementation and test, but not mentioned in the documentation. The implementation for the Git interactions takes place within credential_getpass(). The method prototype is modified to return an 'int' instead of 'void'. This allows us to detect that no attempt was made to fill the given credential, changing the single caller slightly. Also, a new trace2 region is added around the interactive portion of the credential request. This provides a way to measure the amount of time spent in that region for commands that _are_ interactive. It also makes a conventient way to test that the config option works with 'test_region'. Signed-off-by: Derrick Stolee <[email protected]>
At the moment, some background jobs are getting blocked on credentials during the 'prefetch' task. This leads to other tasks, such as incremental repacks, getting blocked. Further, if a user manages to fix their credentials, then they still need to cancel the background process before their background maintenance can continue working. Update the background schedules for our four scheduler integrations to include these config options via '-c' options: * 'credential.interactive=false' will stop Git and some credential helpers from prompting in the UI (assuming the '-c' parameters are carried through and respected by GCM). * 'core.askPass=true' will replace the text fallback for a username and password into the 'true' command, which will return a success in its exit code, but Git will treat the empty string returned as an invalid password and move on. We can do some testing that the credentials are passed, at least in the systemd case due to writing the service files. Signed-off-by: Derrick Stolee <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Add test case to demonstrate that `git index-pack -o <idx-path> pack-path` fails if <idx-path> does not end in ".idx" when `--rev-index` is enabled. In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25) we learned to create `.rev` reverse indexes in addition to `.idx` index files. The `.rev` file pathname is constructed by replacing the suffix on the `.idx` file. The code assumes a hard-coded "idx" suffix. In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12) reverse indexes were enabled by default. If the `-o <idx-path>` argument is used, the index file may have a different suffix. This causes an error when it tries to create the reverse index pathname. The test here demonstrates the failure. (The test forces `--rev-index` to avoid interaction with `GIT_TEST_NO_WRITE_REV_INDEX` during CI runs.) Signed-off-by: Jeff Hostetler <[email protected]>
The 'scalar reconfigure' command is intended to update registered repos with the latest settings available. However, up to now we were not reregistering the repos with background maintenance. In particular, this meant that the background maintenance schedule would not be updated if there are improvements between versions. Be sure to register repos for maintenance during the reconfigure step. Signed-off-by: Derrick Stolee <[email protected]>
In ac8acb4 (sparse-index: complete partial expansion, 2022-05-23), 'expand_index()' was updated to expand the index to a given pathspec. However, the 'path_matches_pattern_list()' method used to facilitate this has the side effect of initializing or updating the index hash variables ('name_hash', 'dir_hash', and 'name_hash_initialized'). This operation is performed on 'istate', though, not 'full'; as a result, the initialized hashes are later overwritten when copied from 'full'. To ensure the correct hashes are in 'istate' after the index expansion, change the arg used in 'path_matches_pattern_list()' from 'istate' to 'full'. Note that this does not fully solve the problem. If 'istate' does not have an initialized 'name_hash' when its contents are copied to 'full', initialized hashes will be copied back into 'istate' but 'name_hash_initialized' will be 0. Therefore, we also need to copy 'full->name_hash_initialized' back to 'istate' after the index expansion is complete. Signed-off-by: Victoria Dye <[email protected]>
Teach index-pack to silently omit the reverse index if the index file does not have the standard ".idx" suffix. In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25) we learned to create `.rev` reverse indexes in addition to `.idx` index files. The `.rev` file pathname is constructed by replacing the suffix on the `.idx` file. The code assumes a hard-coded "idx" suffix. In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12) reverse indexes were enabled by default. If the `-o <idx-path>` argument is used, the index file may have a different suffix. This causes an error when it tries to create the reverse index pathname. Since we do not know why the user requested a non-standard suffix for the index, we cannot guess what the proper corresponding suffix should be for the reverse index. So we disable it. The t5300 test has been updated to verify that we no longer error out and that the .rev file is not created. TODO We could warn the user that we skipped it (perhaps only if they TODO explicitly requested `--rev-index` on the command line). TODO TODO Ideally, we should add an `--rev-index-path=<path>` argument TODO or change `--rev-index` to take a pathname. TODO TODO I'll leave these questions for a future series. Signed-off-by: Jeff Hostetler <[email protected]>
In the Office monorepo, we've recently had an uptick in issues with `scalar clone`. These issues didn't make sense at first and seemed like the users weren't using `microsoft/git` but instead the upstream version's `scalar clone`. Instead of using GVFS cache servers, they were attempting to use the Git protocol's partial clone (which times out). It turns out that what's actually happening is that some network issue is causing the connection with Azure DevOps to error out during the `/gvfs/config` request. In the Git traces, we see the following error during this request: (curl:56) Failure when receiving data from the peer [transient] This isn't 100% of the time, but has increased enough to cause problems for a variety of users. The solution being proposed in this pull request is to remove the fall-back mechanism and instead have an explicit choice to use the GVFS protocol. To avoid significant disruption to Azure DevOps customers (the vast majority of `microsoft/git` users who use `scalar clone` based on my understanding), I added some inferring of a default value from the clone URL. This fallback mechanism was first implemented in the C# version of Scalar in microsoft/scalar#339. This was an attempt to make the Scalar client interesting to non-Azure DevOps customers, especially as GitHub was about to launch the availability of partial clones. Now that the `scalar` client is available upstream, users don't need the GVFS-enabled version to get these benefits. In addition, this will resolve #384 since those requests won't happen against non-ADO URLs unless requested. Signed-off-by: Derrick Stolee <[email protected]>
Add a test verifying that sparse-checkout (with and without sparse index enabled) treat untracked files & directories correctly when changing sparse patterns. Specifically, it ensures that 'git sparse-checkout set' * deletes empty directories outside the sparse cone * does _not_ delete untracked files outside the sparse cone Signed-off-by: Victoria Dye <[email protected]>
Cherry-pick rev-index fixes from v2.41.0.vfs.0.5 into v2.42.0.*
When sparse-checkout is enabled, add the sparse-checkout percentage to the Trace2 data stream. This number was already computed and printed on the console in the "You are in a sparse checkout..." message. It would be helpful to log it too for performance monitoring. Signed-off-by: Jeff Hostetler <[email protected]>
Add VFS checkout hydration percentage information to the default `git status` output. When VFS is enable, users will now see a "You are in a partially-hydrated checkout with <percentage> of tracked files present." message. Upstream `git status` normally prints a "You are in a sparse checkout with <percentage> of tracked files present." This message was hidden in `microsoft/git` when `core_virtualfilesystem` is set (because GVFS users are always (and secretly) in a sparse checkout) and it was thought that it would annoy users. However, we now believe that it may be helpful for users to always see the percentage and know when they are over-hyrdated, since over-hyrdation can occur by accident and may greatly impact their Git performance. Knowing this value may help with GVFS support. Helped-by: Johannes Schindelin <[email protected]> Signed-off-by: Jeff Hostetler <[email protected]>
Target `macos-11` is deprecated now and is in scheduled brownouts. Update to `macos-13`. Signed-off-by: Jeff Hostetler <[email protected]>
Use federated authentication with GitHub Actions and Azure Entra ID for the Azure login commands during build-git-installers.yml builds. This will allow us to drop the use of a client secret to authenticate as the signing identity for Trusted Code Signing. Signed-off-by: Matthew John Cheetham <[email protected]>
GVFS users can easily (and accidentally) over-hydrate their enlistments. This causes some commands to be very slow. Create a command to print the current hydration level. This should help our support team investigate the state of their enlistment. This command will print something like: ``` % git virtualization Skipped: 2 Hydrated: 3 Total: 5 Hydration: 60.00% ``` and log those values to Trace2 in a `data_json` record of the form: ``` {"skipped":2,"hydrated":3,"total":5,"hydration":60.00} ```
Prefetch the value of GIT_TRACE2_DST_DEBUG during startup and before we try to open any Trace2 destination pathnames. Normally, Trace2 always silently fails if a destination target cannot be opened so that it doesn't affect the execution of a Git command. The command should run normally, but just not generate any trace data. This can make it difficult to debug a telemetry setup, since the user doesn't know why telemetry isn't being generated. If the environment variable GIT_TRACE2_DST_DEBUG is true, the Trace2 startup will print a warning message with the `errno` to make debugging easier. However, on Windows, looking up the env variable resets `errno` so the warning message always ends with `...tracing: No error` which is not very helpful. Prefetch the env variable at startup. This avoids the need to update each call-site to capture `errno` in the usual `saved-errno` variable. Signed-off-by: Jeff Hostetler <[email protected]>
Use federated authentication with GitHub Actions and Azure Entra ID for the Azure login commands during `build-git-installers.yml` builds. This will allow us to drop the use of a client secret to authenticate as the signing identity for Trusted Code Signing. The `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, and `AZURE_SUBSCRIPTION_ID` secrets have already been added to the `release` environment, and a test of the `azure/login` step using this mechanism and a subsequent `az` command has been successfully demonstrated here: https://github.com/microsoft/git/actions/runs/9652892561/job/26624014573
Prefetch the value of GIT_TRACE2_DST_DEBUG during startup and before we try to open any Trace2 destination pathnames. Normally, Trace2 always silently fails if a destination target cannot be opened so that it doesn't affect the execution of a Git command. The command should run normally, but just not generate any trace data. This can make it difficult to debug a telemetry setup, since the user doesn't know why telemetry isn't being generated. If the environment variable GIT_TRACE2_DST_DEBUG is true, the Trace2 startup will print a warning message with the `errno` to make debugging easier. However, on Windows, looking up the env variable resets `errno` so the warning message always ends with `...tracing: No error` which is not very helpful. Prefetch the env variable at startup. This avoids the need to update each call-site to capture `errno` in the usual `saved-errno` variable.
Construct 2 new unit tests to explicitly verify the use of `--fallback` and `--no-fallback` arguments to `gvfs-helper`. When a cache-server is enabled, `gvfs-helper` will try to fetch objects from it rather than the origin server. If the cache-server fails (and all cache-server retry attempts have been exhausted), `gvfs-helper` can optionally "fallback" and try to fetch the objects from the origin server. (The retry logic is also applied to the origin server, if the origin server fails on the first request.) Add new unit tests to verify that `gvfs-helper` respects both the `--max-retries` and `--[no-]fallback` arguments. We use the "http_503" mayhem feature of the `test_gvfs_protocol` server to force a 503 response on all requests to the cache-server and the origin server end-points. We can then count the number of connection requests that `gvfs-helper` makes to the server and confirm both the per-server retries and whether fallback was attempted. Signed-off-by: Jeff Hostetler <[email protected]>
By default, GVFS Protocol-enabled Scalar clones will fall back to the origin server if there is a network issue with the cache servers. However (and especially for the prefetch endpoint) this may be a very expensive operation for the origin server, leading to the user being throttled. This shows up later in cases such as 'git push' or other web operations. To avoid this, create a new config option, 'gvfs.fallback', which defaults to true. When set to 'false', pass '--no-fallback' from the gvfs-helper client to the child gvfs-helper server process. This will allow users who have hit this problem to avoid it in the future. In case this becomes a more widespread problem, engineering systems can enable the config option more broadly. Enabling the config will of course lead to immediate failures for users, but at least that will help diagnose the problem when it occurs instead of later when the throttling shows up and the server load has already passed, damage done. Signed-off-by: Derrick Stolee <[email protected]>
Create new `cache_http_503` mayhem method where only the cache server sends a 503. The normal `http_503` directs both cache and origin server to send 503s. This will be used to help test fallback. Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
By default, GVFS Protocol-enabled Scalar clones will fall back to the origin server if there is a network issue with the cache servers. However (and especially for the prefetch endpoint) this may be a very expensive operation for the origin server, leading to the user being throttled. This shows up later in cases such as 'git push' or other web operations. To avoid this, create a new config option, 'gvfs.fallback', which defaults to true. When set to 'false', pass '--no-fallback' from the gvfs-helper client to the child gvfs-helper server process. This will allow users who have hit this problem to avoid it in the future. In case this becomes a more widespread problem, engineering systems can enable the config option more broadly. Enabling the config will of course lead to immediate failures for users, but at least that will help diagnose the problem when it occurs instead of later when the throttling shows up and the server load has already passed, damage done. This change only applies to interactions with Azure DevOps and the GVFS Protocol. --- * [x] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Wrong repo? |
if (!core_apply_sparse_checkout || | ||
core_virtualfilesystem || | ||
sparse_expect_files_outside_of_patterns) | ||
return to_restart; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this if
(and the fact that microsoft/git
needs core_virtualfilesystem
checks here) belongs in clear_skip_worktree_from_present_files()
, further down.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, absolutely! Good eyes.
Yep. Again ;-) |
Range-diff relative to clean/vfs-2.45.2
1: 9d4453e (upstream: 9d4453e) < -: ------------ ci: drop mention of BREW_INSTALL_PACKAGES variable
2: 3c295c8 (upstream: 3c295c8) < -: ------------ mingw: drop bogus (and unneeded) declaration of
_pgmptr
3: 9256695 = 1: 9256695 advice: warn when sparse index expands
4: fa9d858 = 2: fa9d858 Revert "midx-write.c: use
--stdin-packs
when repacking"5: 43daded = 3: 43daded t: remove advice from some tests
6: 9fa03ce = 4: d493e7b survey: stub in new experimental
git-survey
command7: 19db94a = 5: 891929e survey: add command line opts to select references
8: a49a5cd ! 6: 0e33f37 survey: collect the set of requested refs
9: 937d231 ! 7: 02c2b1e survey: calculate stats on refs and print results
10: 90588ad ! 8: 46d0691 survey: stub in treewalk of reachable commits and objects
11: 9897239 = 9: 44a41d0 survey: add traverse callback for commits
12: d90ba6a ! 10: 38f5aec survey: add vector of largest objects for various scaling dimensions
13: 617deae = 11: acf22c6 survey: add pathname of blob or tree to large_item_vec
14: 1de055f = 12: 09c93cc survey: add commit-oid to large_item detail
15: 2bc37ef ! 13: 9d24f68 survey: add commit name-rev lookup to each large_item
16: 3713eae ! 14: f4a94db survey: add --json option and setup for pretty output
17: de268cd ! 15: ecef306 survey: add pretty printing of stats
18: c7ab70c = 16: 67a115d t8100: create test for git-survey
19: d06b3ff ! 17: e210d30 survey: add --no-name-rev option
20: ec39557 = 18: 95f6cde survey: started TODO list at bottom of source file
21: 28549b1 = 19: aecf6ec survey: expanded TODO list at the bottom of the source file
22: 3d885d0 = 20: 79f3731 survey: expanded TODO with more notes
23: d81c8ad (upstream: 12c2ee5) < -: ------------ for-each-repo: optionally keep going on an error
24: 2ef315a (upstream: c75662b) < -: ------------ maintenance: running maintenance should not stop on errors
198: 3be470a = 21: d0ae24c survey: clearly note the experimental nature in the output
25: 8b5ff37 = 22: 5f1879c reset --stdin: trim carriage return from the paths
26: de0e08c ! 23: bb17814 Identify microsoft/git via a distinct version suffix
27: 44873f8 = 24: 462b375 gvfs: ensure that the version is based on a GVFS tag
28: 29d46d7 = 25: 13619bc gvfs: add a GVFS-specific header file
29: d4c7891 ! 26: 6c9e6d9 gvfs: add the core.gvfs config setting
@@ environment.h: int get_shared_repository(void); ## gvfs.c (new) ## @@ ++#define USE_THE_REPOSITORY_VARIABLE +#include "git-compat-util.h" +#include "environment.h" +#include "gvfs.h"
30: e2fcd93 = 27: 663af32 gvfs: add the feature to skip writing the index' SHA-1
31: f74478b ! 28: 0de65b9 gvfs: add the feature that blobs may be missing
32: 2750fd4 ! 29: d3efbf0 gvfs: prevent files to be deleted outside the sparse checkout
@@ t/t1090-sparse-checkout-scope.sh: test_expect_success 'in partial clone, sparse ## unpack-trees.c ## @@ + #include "git-compat-util.h" #include "advice.h" +#include "gvfs.h"
33: 56146c3 = 30: b182882 gvfs: optionally skip reachability checks/upload pack during fetch
34: 298844d ! 31: fdf68f8 gvfs: ensure all filters and EOL conversions are blocked
@@ Documentation/config/core.txt: core.gvfs:: ## convert.c ## @@ + #include "git-compat-util.h" #include "advice.h" +#include "gvfs.h"
35: 9290d8c ! 32: 1673b6c gvfs: allow "virtualizing" objects
36: b1d92ce = 33: 818d82e Hydrate missing loose objects in check_and_freshen()
37: 091dc10 = 34: 2d7a9dc sha1_file: when writing objects, skip the read_object_hook
38: 90f1631 ! 35: 4bee1df gvfs: add global command pre and post hook procs
39: 81a39e7 = 36: 585cc63 t0400: verify that the hook is called correctly from a subdirectory
40: 69c187b = 37: 04b3614 Pass PID of git process to hooks.
41: f224aa5 = 38: 6dc796c pre-command: always respect core.hooksPath
42: 899ad7e = 39: 1beb06a sparse-checkout: update files with a modify/delete conflict
43: c32183f = 40: 2e74640 sparse-checkout: avoid writing entries with the skip-worktree bit
44: 326aa6c = 41: 47b22d8 Do not remove files outside the sparse-checkout
45: 54634b1 = 42: 1b457f9 send-pack: do not check for sha1 file when GVFS_MISSING_OK set
46: 0ed17f0 = 43: 7969d78 cache-tree: remove use of strbuf_addf in update_one
47: 116dbce ! 44: f2eae33 gvfs: block unsupported commands when running in a GVFS repo
48: deeb420 = 45: a08a89d worktree: allow in Scalar repositories
49: 7fc02f9 = 46: 7227b52 gvfs: allow overriding core.gvfs
50: 1fb74c0 = 47: 5733f8e BRANCHES.md: Add explanation of branches and using forks
51: 8e5b270 ! 48: bb9f5c7 Add virtual file system settings and hook proc
52: 47fb1b2 = 49: 3fb4e78 virtualfilesystem: don't run the virtual file system hook if the index has been redirected
53: f998e1d = 50: e15d01a virtualfilesystem: check if directory is included
54: 95436e4 = 51: 11f1b68 backwards-compatibility: support the post-indexchanged hook
55: 91e8518 = 52: 4de21a4 gvfs: verify that the built-in FSMonitor is disabled
56: e9609a8 ! 53: 97258c2 status: add status serialization mechanism
57: b3e7505 = 54: e8957a4 Teach ahead-behind and serialized status to play nicely together
58: 8407c36 = 55: ea03d3a status: serialize to path
59: 28d3839 = 56: 8c430b7 status: reject deserialize in V2 and conflicts
60: 1d82aa2 ! 57: 0f59d77 serialize-status: serialize global and repo-local exclude file metadata
61: 3ff191a = 58: 2c5314d status: deserialization wait
62: c5e4ee0 = 59: f4cb020 merge-recursive: avoid confusing logic in was_dirty()
63: 47d58ff = 60: 6939c6c merge-recursive: add some defensive coding to was_dirty()
64: cc07a28 ! 61: aa1f3bc merge-recursive: teach was_dirty() about the virtualfilesystem
65: 72ea807 = 62: 8433d53 status: deserialize with -uno does not print correct hint
66: abb2796 = 63: e3b6c53 fsmonitor: check CE_FSMONITOR_VALID in ce_uptodate
67: aaab24e = 64: 3789b0b fsmonitor: add script for debugging and update script for tests
68: c156980 ! 65: c46d883 status: disable deserialize when verbose output requested.
69: 81f271c = 66: 7cd1302 t7524: add test for verbose status deserialzation
70: 4262315 = 67: cf86aec deserialize-status: silently fallback if we cannot read cache file
71: 9d211c4 = 68: 1ebaa2f gvfs:trace2:data: add trace2 tracing around read_object_process
72: a41a3ca ! 69: cc3bffa gvfs:trace2:data: status deserialization information
73: aa452a0 ! 70: 3d76475 gvfs:trace2:data: status serialization
74: cc0a9c5 ! 71: b626df8 gvfs:trace2:data: add vfs stats
@@ Commit message ## virtualfilesystem.c ## @@ ++#define USE_THE_REPOSITORY_VARIABLE #include "git-compat-util.h" #include "environment.h" #include "gettext.h"
75: dea7ca4 = 72: 0f500d8 trace2: refactor setting process starting time
76: b276ce0 = 73: 5399de4 trace2:gvfs:experiment: clear_ce_flags_1
77: f002535 = 74: 4d61a80 trace2:gvfs:experiment: report_tracking
78: a3b1d7e = 75: 72f952b trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache
79: 3449130 = 76: 7dd7e4a trace2:gvfs:experiment: read-cache: time read/write of cache-tree extension
80: 6eb56cb = 77: 4cf463d trace2:gvfs:experiment: add region to apply_virtualfilesystem()
81: 7bdb055 = 78: f448b0c trace2:gvfs:experiment: add region around unpack_trees()
82: 90a8fbc = 79: 59a24cb trace2:gvfs:experiment: add region to cache_tree_fully_valid()
83: 79acfe9 = 80: abd63d5 trace2:gvfs:experiment: add unpack_entry() counter to unpack_trees() and report_tracking()
84: f7c6c30 = 81: 3fd6062 trace2:gvfs:experiment: increase default event depth for unpack-tree data
85: 5d4fb1a = 82: 5e8ddc6 trace2:gvfs:experiment: add data for check_updates() in unpack_trees()
86: bcaa51d = 83: 1734a7b Trace2:gvfs:experiment: capture more 'tracking' details
87: 8d15da7 = 84: 7b4dae2 credential: set trace2_child_class for credential manager children
88: 42702c9 = 85: ba45ef7 sub-process: do not borrow cmd pointer from caller
89: 82c64b8 = 86: bae34eb sub-process: add subprocess_start_argv()
90: b5111c7 = 87: 1723a4d sha1-file: add function to update existing loose object cache
91: 826c0d6 = 88: fb2734f packfile: add install_packed_git_and_mru()
92: 638ebe9 = 89: b76054a index-pack: avoid immediate object fetch while parsing packfile
93: d8a6be6 ! 90: a757089 gvfs-helper: create tool to fetch objects using the GVFS Protocol
94: cdd7cfe ! 91: 9c031a0 sha1-file: create shared-cache directory if it doesn't exist
@@ environment.h: extern int protect_hfs; ## gvfs-helper-client.c ## @@ + #define USE_THE_REPOSITORY_VARIABLE #include "git-compat-util.h" +#include "environment.h" #include "hex.h"
95: a038c3c = 92: aa92325 gvfs-helper: better handling of network errors
96: 2910aff ! 93: 622d279 gvfs-helper-client: properly update loose cache with fetched OID
97: b97808a = 94: 5ee547a gvfs-helper: V2 robust retry and throttling
98: 2106050 ! 95: fb037ee gvfs-helper: expose gvfs/objects GET and POST semantics
99: 22f0423 = 96: 5b03acd gvfs-helper: dramatically reduce progress noise
100: 501670e = 97: 33cf1d1 gvfs-helper-client.h: define struct object_id
101: faff4f0 = 98: af55253 gvfs-helper: handle pack-file after single POST request
102: 078ef69 ! 99: b256fb9 test-gvfs-prococol, t5799: tests for gvfs-helper
103: 4f6ba78 = 100: eb160f5 gvfs-helper: move result-list construction into install functions
104: a413cb5 = 101: a22056d t5799: add support for POST to return either a loose object or packfile
105: 5b79aa8 = 102: 3349825 t5799: cleanup wc-l and grep-c lines
106: 669ad33 = 103: acef21f gvfs-helper: verify loose objects after write
107: be74d21 = 104: af0a1a3 t7599: create corrupt blob test
108: 96f1bca ! 105: 07fb0c4 gvfs-helper: add prefetch support
109: 3d89f61 = 106: 92a5dcf gvfs-helper: add prefetch .keep file for last packfile
110: 814f0c2 = 107: c7d926f gvfs-helper: do one read in my_copy_fd_len_tail()
111: 5b292e5 = 108: 81c661c gvfs-helper: move content-type warning for prefetch packs
112: b9530af = 109: f7590cd fetch: use gvfs-helper prefetch under config
113: 26017bd = 110: 4a36b51 gvfs-helper: better support for concurrent packfile fetches
114: 2d6625d = 111: 420a047 remote-curl: do not call fetch-pack when using gvfs-helper
115: c4b3dc4 = 112: 0dffb7c fetch: reprepare packs before checking connectivity
116: bf38a3a = 113: 2d8e72e gvfs-helper: retry when creating temp files
117: de22e15 = 114: f2e1101 sparse: avoid warnings about known cURL issues in gvfs-helper.c
119: a3a331f = 115: e6cf816 maintenance: care about gvfs.sharedCache config
120: c356384 = 116: 7896a46 unpack-trees:virtualfilesystem: Improve efficiency of clear_ce_flags
121: fa34db9 = 117: 4d2226c homebrew: add GitHub workflow to release Cask
123: 75e7179 = 118: e8d810b Adding winget workflows
118: ed719c0 = 119: 867c972 gvfs-helper: add --max-retries to prefetch verb
125: 359e3ea ! 120: 357d01c Disable the
monitor-components
workflow in msft-git122: 5118ec2 = 121: 31c056e t5799: add tests to detect corrupt pack/idx files in prefetch
126: 8112927 = 122: c339eb7 .github: enable windows builds on microsoft fork
124: 88e914c = 123: 066a381 gvfs-helper: ignore .idx files in prefetch multi-part responses
127: 0a1b186 = 124: 202f594 release: create initial Windows installer build workflow
128: d3284e4 = 125: 10bdcbb help: special-case HOST_CPU
universal
129: adc0dc6 = 126: aad3316 release: add Mac OSX installer build
130: 20b196b = 127: 9763e2a release: build unsigned Ubuntu .deb package
131: ce98293 = 128: 8cc9e75 release: add signing step for .deb package
132: 2168e21 = 129: ee4dfe9 release: create draft GitHub release with packages & installers
133: 61cbd89 = 130: db038fa build-git-installers: publish gpg public key
134: b748af6 = 131: 9fe62a4 release: continue pestering until user upgrades
139: 81a3a97 = 132: b5a25d0 update-microsoft-git: create barebones builtin
135: 4c2a29f = 133: c7e456b Makefile: allow specifying GIT_BUILT_FROM_COMMIT
140: 34d9e20 = 134: 7115885 update-microsoft-git: Windows implementation
136: 697df23 = 135: 56a8771 dist: archive HEAD instead of HEAD^{tree}
141: 24bfc6f = 136: ba2d385 update-microsoft-git: use brew on macOS
137: 6487ff0 = 137: dc50266 release: include GIT_BUILT_FROM_COMMIT in MacOS build
142: bab57f6 = 138: 3fc5ef5 .github: update ISSUE_TEMPLATE.md for microsoft/git
138: 7f25f3c = 139: 366ebdd release: add installer validation
145: 0cb2146 = 140: 84d0024 git_config_set_multivar_in_file_gently(): add a lock timeout
146: 3d94588 = 141: 5ebaf3f scalar: set the config write-lock timeout to 150ms
143: 533f117 = 142: 28078c0 .github: update PULL_REQUEST_TEMPLATE.md
147: 225409d = 143: 2521c6e scalar: add docs from microsoft/scalar
144: bee22b6 = 144: cc8ed44 Adjust README.md for microsoft/git
148: 6391ce0 = 145: 8474a8c scalar (Windows): use forward slashes as directory separators
149: 4dbcf6f ! 146: fced4b6 scalar: add retry logic to run_git()
@@ scalar.c: static void setup_enlistment_directory(int argc, const char **argv, +static int git_retries = 3; + + LAST_ARG_MUST_BE_NULL static int run_git(const char *arg, ...) { - struct child_process cmd = CHILD_PROCESS_INIT;
150: 7903a9a = 147: 98e3445 scalar: support the
config
command for backwards compatibility151: 16260a6 = 148: cdbb019 scalar: implement a minimal JSON parser
152: 68145ff = 149: b1d4560 scalar clone: support GVFS-enabled remote repositories
153: f34bc0a ! 150: 1543ff8 test-gvfs-protocol: also serve smart protocol
@@ Commit message ## t/helper/test-gvfs-protocol.c ## @@ + #define USE_THE_REPOSITORY_VARIABLE #include "git-compat-util.h" #include "environment.h" +#include "gettext.h"
154: 156e684 ! 151: e04d0a3 gvfs-helper: add the
endpoint
command155: f0c7192 = 152: 0456f91 dir_inside_of(): handle directory separators correctly
156: 37a1679 = 153: ae4d1ed scalar: disable authentication in unattended mode
157: e4c9fb2 = 154: ef16896 scalar: do initialize
gvfs.sharedCache
158: fdfccaa = 155: dc735db scalar diagnose: include shared cache info
159: 1fcf783 = 156: e207719 scalar: only try GVFS protocol on https:// URLs
160: 13db4b2 = 157: ada5bd1 scalar: verify that we can use a GVFS-enabled repository
161: de44bdb ! 158: 3ffa8b2 scalar: add the
cache-server
command162: bc1f931 = 159: ae36d1f scalar: add a test toggle to skip accessing the vsts/info endpoint
163: 82a620f = 160: f774791 scalar: adjust documentation to the microsoft/git fork
164: 8c62847 = 161: 0af034f scalar: enable untracked cache unconditionally
165: dc53c3d = 162: b7cdecd scalar: parse
clone --no-fetch-commits-and-trees
for backwards compatibility166: fd4a81f = 163: 4fdb7b1 scalar diagnose: accommodate Scalar's Functional Tests
167: 48961e3 = 164: d0e35c1 ci: run Scalar's Functional Tests
168: 39ceaeb = 165: 0a1ef09 scalar: upgrade to newest FSMonitor config setting
169: 6a72129 = 166: c3009c4 abspath: make strip_last_path_component() global
170: 265c748 = 167: d6b0350 scalar: .scalarCache should live above enlistment
171: 10dd178 ! 168: 27f15ae add/rm: allow adding sparse entries when virtual
172: e00dddb = 169: 5643619 sparse-checkout: add config to disable deleting dirs
173: e082cb8 = 170: 29da448 diff: ignore sparse paths in diffstat
174: df6a787 = 171: fc656ce repo-settings: enable sparse index by default
175: 26babbf = 172: decb024 diff(sparse-index): verify with partially-sparse
176: 40ea561 = 173: a108f0f stash: expand testing for
git stash -u
177: f21079b = 174: 45d1ce3 sequencer: avoid progress when stderr is redirected
178: 9008be1 = 175: 2ac1994 sparse: add vfs-specific precautions
179: 994a008 ! 176: ee73305 reset: fix mixed reset when using virtual filesystem
180: faab0ce ! 177: 4cde4fd credential: add new interactive config option
181: 238ecde = 178: 65740e8 maintenance: add custom config to background jobs
182: 9b6f619 ! 179: f664318 scalar: configure maintenance during 'reconfigure'
183: f9506f8 (upstream: b64b0df) < -: ------------ scalar: avoid segfault in reconfigure --all
184: 918d0a0 = 180: 5c9c8e1 scalar: make GVFS Protocol a forced choice
185: 1304aa1 = 181: f8ffde0 t5300: confirm failure of git index-pack when non-idx suffix requested
186: ef4f090 = 182: bd791ef index-pack: disable rev-index if index file has non .idx suffix
187: 84ee4f9 = 183: 844a12a sparse-index.c: fix use of index hashes in expand_index
188: aad6134 = 184: 5a293e1 t1092: add test for untracked files and directories
189: fd33009 = 185: ebba2a4 wt-status: add trace2 data for sparse-checkout percentage
190: 4ff4abe = 186: 4953deb wt-status: add VFS hydration percentage to normal
git status
output191: 12b09dd = 187: 4b33e7e scalar-functional-tests.yml: update macos-11 to macos-13
192: 8a7f3f4 = 188: ac292b9 github: use federated auth for Azure login
193: 080a634 = 189: 1fc79a6 trace2: prefetch value of GIT_TRACE2_DST_DEBUG at startup
194: 7fdf949 = 190: 53052f2 t5799: explicitly test gvfs-helper --fallback and --no-fallback
195: 0b0286c ! 191: 6fb1e02 gvfs-helper: don't fallback with new config
196: 138a74b = 192: fbd8164 test-gvfs-protocol: add cache_http_503 to mayhem
197: 3f0abb7 = 193: db9d5e3 t5799: add unit tests for new
gvfs.fallback
config setting