Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase to v2.47.1 #5274

Merged
merged 428 commits into from
Nov 25, 2024
Merged

Conversation

dscho
Copy link
Member

@dscho dscho commented Nov 25, 2024

Range-diff relative to main
  • 1: 82b47a4 = 1: 5c8cfb9 t9350: point out that refs are not updated correctly

  • 3: 84c0be4 = 2: bb78dfd transport-helper: add trailing --

  • 4: 8b51b7a = 3: 40bbb20 remote-helper: check helper status after import/export

  • 5: 5278695 = 4: c1365f1 mingw: demonstrate a problem with certain absolute paths

  • 2: 32c8a61 = 5: e2e116a gitk(Windows): avoid inadvertently calling executables in the worktree

  • 6: 2c843e8 = 6: 0ffc708 Always auto-gc after calling a fast-import transport

  • 9: 30e563a = 7: e357054 clean: do not traverse mount points

  • 8: 298f318 = 8: 28669ef mingw: include the Python parts in the build

  • 7: 4097c1d = 9: a453ea2 mingw: allow absolute paths without drive prefix

  • 11: 71b00f9 = 10: 7d1ae8f clean: remove mount points when possible

  • 10: 0c32432 = 11: d3370a9 win32/pthread: avoid name clashes with winpthread

  • 12: e08a265 = 12: 6bddbec git-compat-util: avoid redeclaring _DEFAULT_SOURCE

  • 13: 8d0c8ba = 13: 4725915 Import the source code of mimalloc v2.1.2

  • 14: f5bbbbb = 14: 9e370d8 mimalloc: adjust for building inside Git

  • 15: c70b879 = 15: a4ec08c mimalloc: offer a build-time option to enable it

  • 16: 7134a1e = 16: 2c9b986 mimalloc: use "weak" random seed when statically linked

  • 17: a7c62e6 = 17: 7ab713d mingw: use mimalloc

  • 18: 894ab10 = 18: a8e5535 transport: optionally disable side-band-64k

  • 19: bee546e = 19: f3916a0 mingw: make sure errno is set correctly when socket operations fail

  • 21: 9d6e390 = 20: 53bd507 mingw: demonstrate a git add issue with NTFS junctions

  • 22: f931810 = 21: a93211c strbuf_realpath(): use platform-dependent API if available

  • 20: 9b9040f = 22: 3b63062 mingw: do resolve symlinks in getcwd()

  • 23: 14f188f = 23: d898328 mingw: fix fatal error working on mapped network drives on Windows

  • 24: 9a3f6c2 = 24: cb062b1 clink.pl: fix MSVC compile script to handle libcurl-d.lib

  • 25: 2d3b95d = 25: 456ac58 mingw: ensure valid CTYPE

  • 26: a25428a = 26: e803c66 mingw: allow git.exe to be used instead of the "Git wrapper"

  • 27: 8bf47b1 = 27: 31c97b6 mingw: ignore HOMEDRIVE/HOMEPATH if it points to Windows' system directory

  • 28: 20ed4ce = 28: 2c50312 http: use new "best effort" strategy for Secure Channel revoke checking

  • 29: 29b8d1e = 29: 05eb5f5 mingw: implement a platform-specific strbuf_realpath()

  • 30: e46b3fe = 30: 23f2d87 vcxproj: unclash project directories with build outputs

  • 31: e58a01e = 31: d55b1b6 t5505/t5516: allow running without .git/branches/ in the templates

  • 32: 7ef36fb = 32: b80c258 t5505/t5516: fix white-space around redirectors

  • 33: 3dbe3b6 = 33: d276393 t3701: verify that we can add lots of files interactively

  • 34: 4096d96 = 34: 8f35920 git add -i: handle CR/LF line endings in the interactive input

  • 35: ec6ab10 = 35: bf7b3b6 commit: accept "scissors" with CR/LF line endings

  • 36: 0297fd3 = 36: a9bc204 t0014: fix indentation

  • 37: eca2cf5 = 37: a5c5c3d git-gui: accommodate for intent-to-add files

  • 38: d19278d = 38: 7f30b78 clink.pl: fix libexpatd.lib link error when using MSVC

  • 39: a1cdb00 = 39: 9dc2022 Makefile: clean up .ilk files when MSVC=1

  • 40: b0782da = 40: 12df0b7 vcbuild: add support for compiling Windows resource files

  • 41: bfc8ec0 = 41: bbc6fe8 config.mak.uname: add git.rc to MSVC builds

  • 42: 7571c7c = 42: 91c70e8 clink.pl: ignore no-stack-protector arg on MSVC=1 builds

  • 43: acd8bcd = 43: 96b8bc9 clink.pl: move default linker options for MSVC=1 builds

  • 44: 530cbc4 = 44: c8c4463 buildsystems: remove duplicate clause

  • 45: 75f4966 = 45: 9303fcb vcxproj: handle resource files, too

  • 46: 314c8cd = 46: b1ad8e8 vcxproj: ignore -fno-stack-protector and -fno-common

  • 47: 4ced2ab = 47: 1919480 vcxproj: handle GUI programs, too

  • 51: 321c096 = 48: 8b62f2e cmake: install headless-git.

  • 48: eb6bbeb = 49: f321ca9 vcpkg_install: detect lack of Git

  • 49: b8faef9 = 50: f315aff vcpkg_install: add comment regarding slow network connections

  • 50: f4aea0b = 51: c3e7756 vcxproj: support building Windows/ARM64 binaries

  • 52: d2e44fd = 52: 19d30f7 vcbuild: install ARM64 dependencies when building ARM64 binaries

  • 53: 984b177 = 53: fe49458 vcbuild: add an option to install individual 'features'

  • 54: d848405 = 54: 7ec0269 cmake: allow building for Windows/ARM64

  • 55: 8efc694 = 55: fea6ee3 ci(vs-build) also build Windows/ARM64 artifacts

  • 56: 54bbcfe = 56: 3b37c55 Add schannel to curl installation

  • 57: cbaedbe = 57: d006d65 cmake(): allow setting HOST_CPU for cross-compilation

  • 59: d96e86f = 58: fd76ce1 mingw: allow for longer paths in parse_interpreter()

  • 60: fd88e77 = 59: 21de89c compat/vcbuild: document preferred way to build in Visual Studio

  • 61: ef787ec = 60: 93d79ee http: optionally send SSL client certificate

  • 58: d1078e1 = 61: c0c98a5 subtree: update contrib/subtree test target

  • 65: 3a5a85d = 62: d537e6c ci: run contrib/subtree tests in CI builds

  • 62: 72c0388 = 63: 583f668 CMake: default Visual Studio generator has changed

  • 63: dbeace9 = 64: 63debc5 .gitignore: add Visual Studio CMakeSetting.json file

  • 64: dc7b364 = 65: 66e8e3e CMakeLists: add default "x64-windows" arch for Visual Studio

  • 66: 8f9ee9a = 66: 2047422 CMake: show Win32 and Generator_platform build-option values

  • 67: 89a13aa = 67: 591478a hash-object: demonstrate a >4GB/LLP64 problem

  • 68: d21a631 = 68: 9623c9d write_object_file_literally(): use size_t

  • 69: fc342ff = 69: 4d26723 object-file.c: use size_t for header lengths

  • 70: af5980d = 70: 8839e60 hash algorithms: use size_t for section lengths

  • 72: 2e16f8d = 71: 8ab656d init: do parse all core.* settings early

  • 71: a3aa580 = 72: eb630eb hash-object --stdin: verify that it works with >4GB/LLP64

  • 73: 75551ae = 73: 5dfb2a2 hash-object: add another >4GB/LLP64 test case

  • 75: 7f3f3e4 = 74: 6634ac1 hash-object: add a >4GB/LLP64 test case using filtered input

  • 74: 3011d30 = 75: 58ab53f setup: properly use "%(prefix)/" when in WSL

  • 76: 2f29de6 = 76: 59250a0 compat/mingw.c: do not warn when failing to get owner

  • 77: 1e34e83 = 77: 7c4ff36 mingw: $env:TERM="xterm-256color" for newer OSes

  • 78: fcd3b4e = 78: dc3d458 Add config option windows.appendAtomically

  • 80: 17fd211 = 79: 1ddc09b winansi: check result and Buffer before using Name

  • 81: 1a36188 = 80: c9315d9 mingw: change core.fsyncObjectFiles = 1 by default

  • 79: 4bb392d = 81: b750a5f vcxproj: allow building with NO_PERL again

  • 82: ae9705d = 82: c68eb37 vcxproj: require C11

  • 83: 5c2f375 = 83: 81e7c62 vcxproj: ignore the -pedantic option

  • 84: 31233e3 = 84: b7ed7c5 vcxproj: include reftable when committing .vcxproj files

  • 85: 956a0d4 = 85: e22fc0b vcxproj: handle libreftable_test, too

  • 86: 4eb3bfc = 86: 1c3a6f9 vcxproj: avoid escaping double quotes in the defines

  • 87: af6fc9a = 87: 0f55e74 ci: adjust Azure Pipeline for runs_on_pool

  • 88: 81c2b63 = 88: c4c7782 ci: stop linking the prove cache

  • 89: 63dd727 = 89: 1c42d58 ci: reinstate Azure Pipelines support

  • 90: 8c8d3ee = 90: 4bf5031 azure-pipeline: drop the GETTEXT_POISON job

  • 91: 082f7b6 = 91: b4c9d13 azure-pipeline: stop hard-coding apt-get calls

  • 92: 915aa04 = 92: 9939a9f azure-pipeline: drop the code to write to/read from a file share

  • 93: fe5de3e = 93: 224e010 azure-pipeline: use partial clone/parallel checkout to initialize minimal-sdk

  • 94: 4572d4d = 94: 6a84be4 azure-pipeline: downcase the job name of the Linux32 job

  • 95: 83cceed = 95: 41f82a8 azure-pipeline: run static-analysis on jammy

  • 96: fe9ded9 = 96: ab7ef94 bswap.h: add support for built-in bswap functions

  • 97: 9dff4e7 = 97: 02be695 config.mak.uname: add support for clangarm64

  • 98: 94f1f27 = 98: 0260c4c MinGW: link as terminal server aware

  • 99: 9685fb6 = 99: f722e2a Fix Windows version resources

  • 100: cafdce8 = 100: c2361f8 ci: create clangarm64-build.yml

  • 101: 3957cee = 101: 74740a3 status: fix for old-style submodules with commondir

  • 102: fd5e65d = 102: a1a82fc windows: skip linking git-<command> for built-ins

  • 103: 74807d1 = 103: d31ff6c windows: fix Repository>Explore Working Copy

  • 104: 98f21f5 = 104: 2ca159e http: optionally load libcurl lazily

  • 105: eec957e = 105: 4bfc6a0 http: support lazy-loading libcurl also on Windows

  • 106: 9d0e622 = 106: 58317a8 http: when loading libcurl lazily, allow for multiple SSL backends

  • 107: 2718450 = 107: 2cfe8fc mingw: do load libcurl dynamically by default

  • 108: 1047e1b = 108: 70a07b0 Add a GitHub workflow to verify that Git/Scalar work in Nano Server

  • 109: 94f5522 = 109: 513b13e mingw: suggest windows.appendAtomically in more cases

  • 110: 56ea536 = 110: 9e28d96 win32: use native ANSI sequence processing, if possible

  • 111: b478fc7 = 111: 729ed48 git.rc: include winuser.h

  • 112: a92f41b = 112: 910d494 common-main.c: fflush stdout buffer upon exit

  • 113: 7179e42 = 113: 5d2bfee t5601/t7406(mingw): do run tests with symlink support

  • 114: 8202b24 = 114: 262bebd win32: ensure that localtime_r() is declared even in i686 builds

  • 115: d2ebe05 = 115: d6606cb Fallback to AppData if XDG_CONFIG_HOME is unset

  • 120: f398f64 = 116: 97819f5 run-command: be helpful with Git LFS fails on Windows 7

  • 116: 776b930 = 117: 37c7e21 ci: work around a problem with HTTP/2 vs libcurl v8.10.0

  • 117: 31382dc = 118: 6489746 pack-objects: add --full-name-hash option

  • 118: e9fe71f = 119: 0400635 repack: test --full-name-hash option

  • 119: 2bd50fb = 120: 32fbc51 pack-objects: add GIT_TEST_FULL_NAME_HASH

  • 121: d3c0826 = 121: 5ec06b8 git-repack: update usage to match docs

  • 122: 871f936 = 122: 93b5259 p5313: add size comparison test

  • 123: 97829b5 = 123: eb9f0d9 test-tool: add helper for name-hash values

  • 124: 5c8182b = 124: f5cf8e5 repack/pack-objects: mark --full-name-hash as experimental

  • 125: 685d130 = 125: 4abe739 path-walk: introduce an object walk by path

  • 126: f10a957 = 126: 483c73b t6601: add helper for testing path-walk API

  • 127: 118b06d = 127: efea267 path-walk: allow consumer to specify object types

  • 128: 731998c = 128: 229cc68 path-walk: allow visiting tags

  • 129: 9e7da31 = 129: 6b02e8c revision: create mark_trees_uninteresting_dense()

  • 130: ec1a5d4 = 130: 2bd12ae path-walk: add prune_all_uninteresting option

  • 131: 4578489 = 131: 7b0b966 pack-objects: extract should_attempt_deltas()

  • 132: 454cfdd = 132: 4ae0c15 pack-objects: add --path-walk option

  • 133: 82abe40 = 133: e9acce7 pack-objects: introduce GIT_TEST_PACK_PATH_WALK

  • 134: f07075e = 134: 4e6ac7c repack: add --path-walk option

  • 135: bf51d09 = 135: 6108ed4 pack-objects: enable --path-walk via config

  • 136: 2d533c9 < -: ------------- scalar: enable path-walk during push via config

  • 280: 8ca850e ! 136: cb1b10c fixup! scalar: enable path-walk during push via config

    @@
      ## Metadata ##
    -Author: Johannes Schindelin <[email protected]>
    +Author: Derrick Stolee <[email protected]>
     
      ## Commit message ##
    -    fixup! scalar: enable path-walk during push via config
    +    scalar: enable path-walk during push via config
     
    -    There was a typo in the configuration Scalar registers, which was missed
    -    by author and reviewer 🤦. No big harm done, let's just fix this in time
    -    for the next Git for Windows version.
    +    Repositories registered with Scalar are expected to be client-only
    +    repositories that are rather large. This means that they are more likely to
    +    be good candidates for using the --path-walk option when running 'git
    +    pack-objects', especially under the hood of 'git push'. Enable this config
    +    in Scalar repositories.
     
    -    Signed-off-by: Johannes Schindelin <[email protected]>
    +    Signed-off-by: Derrick Stolee <[email protected]>
     
      ## scalar.c ##
     @@ scalar.c: static int set_recommended_config(int reconfigure)
      		{ "core.autoCRLF", "false" },
      		{ "core.safeCRLF", "false" },
      		{ "fetch.showForcedUpdates", "false" },
    --		{ "push.usePathWalk", "true" },
     +		{ "pack.usePathWalk", "true" },
      		{ NULL, NULL },
      	};
  • 137: 2f07cd7 = 137: 11969a9 pack-objects: refactor path-walk delta phase

  • 138: cd6e3a6 = 138: db77bb9 pack-objects: thread the path-based compression

  • 139: 8b02eeb ! 139: 1df3d82 path-walk API: avoid adding a root tree more than once

    @@ path-walk.c: int walk_objects_by_path(struct path_walk_info *info)
      				struct tag *tag = lookup_tag(info->revs->repo,
      							     &obj->oid);
     @@ path-walk.c: int walk_objects_by_path(struct path_walk_info *info)
    - 		oid = get_commit_tree_oid(c);
      		t = lookup_tree(info->revs->repo, oid);
      
    -+		if (t->object.flags & SEEN)
    -+			continue;
    -+		t->object.flags |= SEEN;
    -+
      		if (t) {
     -			oidset_insert(&root_tree_set, oid);
     -			oid_array_append(&root_tree_list->oids, oid);
    ++			if (t->object.flags & SEEN)
    ++				continue;
    ++			t->object.flags |= SEEN;
    ++
     +			if (!oidset_insert(&root_tree_set, oid))
     +				oid_array_append(&root_tree_list->oids, oid);
      		} else {
  • 140: a288333 = 140: 2f56089 backfill: add builtin boilerplate

  • 141: e61870b = 141: 976f964 backfill: basic functionality and tests

  • 142: c7df6b4 = 142: d9013cb backfill: add --batch-size= option

  • 143: 938bdf9 = 143: 65ecbcd backfill: add --sparse option

  • 144: b8cebf0 = 144: ad0d10b backfill: assume --sparse when sparse-checkout is enabled

  • 145: 51b8ae8 = 145: f8f86c9 backfill: mark it as experimental

  • 146: 6d2fb8b = 146: b5b22c9 survey: stub in new experimental 'git-survey' command

  • 147: 4e61f93 = 147: 5411a08 survey: add command line opts to select references

  • 148: 3c9a3dd = 148: a2f2b04 survey: start pretty printing data in table form

  • 149: dc4a5f7 = 149: 47b2659 survey: add object count summary

  • 150: 61f5ff5 = 150: 8a45402 survey: summarize total sizes by object type

  • 151: 0714ece = 151: 22f09cd survey: show progress during object walk

  • 152: 76ae6b5 = 152: 0c887c3 survey: add ability to track prioritized lists

  • 153: 1eef810 = 153: 3e1d3a4 survey: add report of "largest" paths

  • 154: 910d865 ! 154: bd2cc14 survey: add --top= option and config

    @@ Commit message
         survey: add --top=<N> option and config
     
         The 'git survey' builtin provides several detail tables, such as "top
    -    files by on-disk size". The size of these tables defaults to 100,
    +    files by on-disk size". The size of these tables defaults to 10,
         currently.
     
         Allow the user to specify this number via a new --top=<N> option or the
         new survey.top config key.
     
         Signed-off-by: Derrick Stolee <[email protected]>
    +    Signed-off-by: Johannes Schindelin <[email protected]>
     
      ## Documentation/config/survey.txt ##
     @@ Documentation/config/survey.txt: survey.*::
    @@ builtin/survey.c: int cmd_survey(int argc, const char **argv, const char *prefix
      		.opts = {
      			.verbose = 0,
      			.show_progress = -1, /* defaults to isatty(2) */
    -+			.top_nr = 100,
    ++			.top_nr = 10,
      
      			.refs.want_all_refs = -1,
      
  • 155: 1b6847f (upstream: fc5589d) < -: ------------- line-log: protect inner strbuf from free

  • 156: a563945 = 155: 2050d48 survey: clearly note the experimental nature in the output

  • 157: eff0ca0 (upstream: 8aeff2c) < -: ------------- line-log: remove output_prefix()

  • 158: db93c66 (upstream: 436728f) < -: ------------- diff: modify output_prefix function pointer

  • 279: 6d3f006 = 156: dee6e10 path-walk: improve path-walk speed with many tags

  • 288: 9abda5e = 157: 2b74f7c Makefile: fix dependency for $(UNIT_TEST_DIR)/clar/clar.o

  • 290: a7a3cea ! 158: ac0b0e8 mingw.c: Fix complier warnings for a 64 bit msvc

    @@ compat/compiler.h: static inline void get_compiler_info(struct strbuf *info)
      	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
     
      ## compat/mingw.c ##
    -@@ compat/mingw.c: int mingw_chmod(const char *filename, int mode)
    +@@ compat/mingw.c: static inline void filetime_to_timespec(const FILETIME *ft, struct timespec *ts)
       */
      static int has_valid_directory_prefix(wchar_t *wfilename)
      {
    @@ compat/mingw.c: int mingw_chmod(const char *filename, int mode)
      		wchar_t c = wfilename[--n];
     @@ compat/mingw.c: static const char *parse_interpreter(const char *cmd)
      {
    - 	static char buf[MAX_PATH];
    + 	static char buf[100];
      	char *p, *opt;
     -	int n, fd;
     +	ssize_t n; /* read() can return negative values */
    @@ compat/mingw.c: int mingw_putenv(const char *namevalue)
      	xutftowcs(wide, namevalue, size);
      	equal = wcschr(wide, L'=');
      	if (!equal)
    -@@ compat/mingw.c: static BOOL WINAPI handle_ctrl_c(DWORD ctrl_type)
    +@@ compat/mingw.c: static void maybe_redirect_std_handles(void)
       */
      int wmain(int argc, const wchar_t **wargv)
      {
  • 159: 2e6b871 = 159: 3b31514 Win32: make FILETIME conversion functions public

  • 160: dcfad57 = 160: 0916d98 Win32: dirent.c: Move opendir down

  • 161: 0077575 = 161: 078ca6f mingw: make the dirent implementation pluggable

  • 162: 95bccd8 = 162: 7a9d04d Win32: make the lstat implementation pluggable

  • 163: 98e220a = 163: b432ad2 mingw: add infrastructure for read-only file system level caches

  • 164: 028efda = 164: fe287c9 mingw: add a cache below mingw's lstat and dirent implementations

  • 165: d4f0444 = 165: 76af47d fscache: load directories only once

  • 166: e45f8ad = 166: fba27d9 fscache: add key for GIT_TRACE_FSCACHE

  • 167: d00800e = 167: 6c3fcc6 fscache: remember not-found directories

  • 168: ade3ba7 = 168: 5334827 fscache: add a test for the dir-not-found optimization

  • 169: 53de33f = 169: 9a2ec4c add: use preload-index and fscache for performance

  • 170: fda16c8 = 170: d376cd2 dir.c: make add_excludes aware of fscache during status

  • 171: 3826f0d = 171: e7fb9d5 fscache: make fscache_enabled() public

  • 172: eeead24 = 172: b648f3d dir.c: regression fix for add_excludes with fscache

  • 173: 7973ab2 = 173: 3ed426a fetch-pack.c: enable fscache for stats under .git/objects

  • 174: bb923bb = 174: 9edc43b checkout.c: enable fscache for checkout again

  • 175: 6f5ebad = 175: ff7b5dc Enable the filesystem cache (fscache) in refresh_index().

  • 176: 5517a9f = 176: 0769498 fscache: use FindFirstFileExW to avoid retrieving the short name

  • 177: 2a5d448 = 177: c03535e status: disable and free fscache at the end of the status command

  • 178: e931185 = 178: 9e9c053 fscache: add GIT_TEST_FSCACHE support

  • 179: 94dd824 = 179: 43f88c4 fscache: add fscache hit statistics

  • 180: 3257dfe = 180: 514aa16 mem_pool: add GIT_TRACE_MEMPOOL support

  • 181: 9e1e228 = 181: 3e2f937 fscache: fscache takes an initial size

  • 182: daf7e83 = 182: c17cce8 fscache: update fscache to be thread specific instead of global

  • 183: b8f8cdc = 183: e19097e fscache: teach fscache to use mempool

  • 184: f34dae8 = 184: d7cf33d fscache: make fscache_enable() thread safe

  • 185: 6330375 = 185: 8b23bf0 fscache: teach fscache to use NtQueryDirectoryFile

  • 186: 3629591 = 186: fb8d206 unpack-trees: enable fscache for sparse-checkout

  • 187: 8cf3d40 = 187: 2e024ab fscache: remember the reparse tag for each entry

  • 189: d5c7454 = 188: 03343ed fscache: implement an FSCache-aware is_mount_point()

  • 191: 75ce329 = 189: d9f7c48 clean: make use of FSCache

  • 188: 8b37641 = 190: 2962efb git-gui: provide question helper for retry fallback on Windows

  • 190: 3158c66 = 191: a4c23dc git gui: set GIT_ASKPASS=git-gui--askpass if not set yet

  • 193: 4ab6eca = 192: b1f11b8 git-gui--askyesno: fix funny text wrapping

  • 192: 76d79d9 = 193: 4685e5f gitk: Unicode file name support

  • 195: dde11f4 = 194: d6f6e9b git-gui--askyesno: allow overriding the window title

  • 194: 0c82cc9 = 195: 528786d gitk: Use an external icon file on Windows

  • 197: e6058c8 = 196: 0d4f495 git-gui--askyesno (mingw): use Git for Windows' icon, if available

  • 196: 28d40b9 = 197: 74c8ff8 gitk: fix arrow keys in input fields with Tcl/Tk >= 8.6

  • 198: 24be74a = 198: 7726fb9 gitk: make the "list references" default window width wider

  • 199: bf28c7f = 199: 0d214ce pack-objects (mingw): demonstrate a segmentation fault with large deltas

  • 200: 0b30bc0 = 200: e6f2707 mingw: support long paths

  • 201: 0a9dc09 = 201: ff963b4 Win32: fix 'lstat("dir/")' with long paths

  • 202: 5ff1ebe = 202: e3ce232 win32(long path support): leave drive-less absolute paths intact

  • 203: e4527a8 = 203: a6f588e compat/fsmonitor/fsm-*-win32: support long paths

  • 208: a36b695 = 204: f418cfd clean: suggest using core.longPaths if paths are too long to remove

  • 204: 67337c8 = 205: 69e60dd mingw: Support git_terminal_prompt with more terminals

  • 205: b5dc26a = 206: 712ea9e compat/terminal.c: only use the Windows console if bash 'read -r' fails

  • 206: 1f391dd = 207: b084059 mingw (git_terminal_prompt): do fall back to CONIN$/CONOUT$ method

  • 207: 2c87e7b = 208: 5c4aa7d strbuf_readlink: don't call readlink twice if hint is the exact link size

  • 209: 5bea09f = 209: 47f46b2 strbuf_readlink: support link targets that exceed PATH_MAX

  • 210: 512490f = 210: c0d6f43 lockfile.c: use is_dir_sep() instead of hardcoded '/' checks

  • 211: bd1d088 = 211: 3cee87d Win32: don't call GetFileAttributes twice in mingw_lstat()

  • 212: 6d21967 = 212: 8c09f0b Win32: implement stat() with symlink support

  • 213: a32c452 = 213: cfbbe6d Win32: remove separate do_lstat() function

  • 214: 04ac44d = 214: 1adc244 Win32: let mingw_lstat() error early upon problems with reparse points

  • 215: 797b60f = 215: 052c593 mingw: teach fscache and dirent about symlinks

  • 216: 147a648 = 216: 16c30c4 Win32: lstat(): return adequate stat.st_size for symlinks

  • 217: 3d9ebe3 = 217: d430fe6 Win32: factor out retry logic

  • 218: 65c4e23 = 218: c153bde Win32: change default of 'core.symlinks' to false

  • 219: d8c0893 = 219: f4d71dd Win32: add symlink-specific error codes

  • 220: 3a56953 = 220: 393164b Win32: mingw_unlink: support symlinks to directories

  • 221: d27bed2 = 221: b140540 Win32: mingw_rename: support renaming symlinks

  • 222: f99b857 = 222: a6b988f Win32: mingw_chdir: change to symlink-resolved directory

  • 223: 86420a1 = 223: beccee0 Win32: implement readlink()

  • 224: dc59fce = 224: 99d1d87 mingw: lstat: compute correct size for symlinks

  • 225: a7ad2f4 = 225: 78ba988 Win32: implement basic symlink() functionality (file symlinks only)

  • 226: 73e5a1e = 226: 6c4ce32 Win32: symlink: add support for symlinks to directories

  • 227: 248e7b4 = 227: 3903bbc mingw: try to create symlinks without elevated permissions

  • 228: ebe8730 = 228: 37d8a03 mingw: emulate stat() a little more faithfully

  • 229: 132c94f = 229: 721b645 mingw: special-case index entries for symlinks with buggy size

  • 230: 4d79b76 = 230: 44d9078 mingw: introduce code to detect whether we're inside a Windows container

  • 231: f3ef296 = 231: 6729032 mingw: when running in a Windows container, try to rename() harder

  • 232: 7d1cc97 = 232: 02332aa mingw: move the file_attr_to_st_mode() function definition

  • 233: 53cb5c1 = 233: 492cfa7 mingw: Windows Docker volumes are not symbolic links

  • 234: ec29d2b = 234: 9847064 Win32: symlink: move phantom symlink creation to a separate function

  • 235: c088807 = 235: 01be787 Introduce helper to create symlinks that knows about index_state

  • 236: 5a30497 = 236: 3704977 mingw: allow to specify the symlink type in .gitattributes

  • 237: a9ed70c = 237: c2381fd Win32: symlink: add test for symlink attribute

  • 238: fc96572 = 238: cf25bbc mingw: work around rename() failing on a read-only file

  • 239: 236b7b7 = 239: 2e1cfb8 mingw: explicitly specify with which cmd to prefix the cmdline

  • 240: 9fd2e84 = 240: 8d1cd93 mingw: when path_lookup() failed, try BusyBox

  • 241: cf2a479 = 241: f13ebfb test-lib: avoid unnecessary Perl invocation

  • 242: 00139f9 = 242: b7f932c test-tool: learn to act as a drop-in replacement for iconv

  • 243: 7774624 = 243: 4d1c8b1 tests(mingw): if iconv is unavailable, use test-helper --iconv

  • 244: 023f17b = 244: 02cbbe7 gitattributes: mark .png files as binary

  • 245: 9ab6b6b = 245: d539e10 tests: move test PNGs into t/lib-diff/

  • 246: 95f6db9 = 246: be5515e tests: only override sort & find if there are usable ones in /usr/bin/

  • 247: 1aab1ad = 247: ac4b91a tests: use the correct path separator with BusyBox

  • 248: e033676 = 248: 7304ba5 mingw: only use Bash-ism builtin pwd -W when available

  • 249: 8e3b66b = 249: 3078bdb tests (mingw): remove Bash-specific pwd option

  • 250: 7ae8372 = 250: c42863a test-lib: add BUSYBOX prerequisite

  • 251: bb1b278 = 251: 37cc59c t5003: use binary file from t/lib-diff/

  • 252: ffe9503 = 252: cfb83ab t5532: workaround for BusyBox on Windows

  • 253: dbb27bd = 253: 113706c t5605: special-case hardlink test for BusyBox-w32

  • 254: 5b36798 = 254: 9afcac0 t5813: allow for $PWD to be a Windows path

  • 255: 918f4ab = 255: 0a64894 t9200: skip tests when $PWD contains a colon

  • 256: b99fde4 = 256: 228d139 mingw: add a Makefile target to copy test artifacts

  • 257: 0f35e77 = 257: 5f5aad9 mingw: optionally enable wsl compability file mode bits

  • 258: 10bb07f = 258: 16b12db mingw: kill child processes in a gentler way

  • 259: 63cc420 = 259: 9d6f549 mingw: do not call xutftowcs_path in mingw_mktemp

  • 260: 26539c4 = 260: fd2663d mingw: really handle SIGINT

  • 261: 8b8b123 = 261: d66d168 Partially un-revert "editor: save and reset terminal after calling EDITOR"

  • 262: 2ce548c = 262: e59bd66 Add a GitHub workflow to monitor component updates

  • 263: c3e1525 = 263: 0665635 reset: reinstate support for the deprecated --stdin option

  • 264: b4fb482 = 264: 5a344f0 fsmonitor: reintroduce core.useBuiltinFSMonitor

  • 265: 7a06b7b = 265: 8299d91 dependabot: help keeping GitHub Actions versions up to date

  • 266: 2ce307a = 266: 8638ef4 Describe Git for Windows' architecture [no ci]

  • 267: 26bccbe = 267: ce425d1 Modify the Code of Conduct for Git for Windows

  • 268: 141b8ab = 268: 969b198 CONTRIBUTING.md: add guide for first-time contributors

  • 269: b134efe = 269: 3946a5f README.md: Add a Windows-specific preamble

  • 270: fa6fc63 = 270: 4a8cf2f Add an issue template

  • 271: 0ce6de6 = 271: c169b69 Modify the GitHub Pull Request template (to reflect Git for Windows)

  • 272: e587604 < -: ------------- .github: Add configuration for the Sentiment Bot

  • 273: 2ff87a4 = 272: 4931518 Document how $HOME is set on Windows

  • 274: 9468391 ! 273: bb1e874 SECURITY.md: document Git for Windows' policies

    @@ Commit message
         This is the recommended way on GitHub to describe policies revolving around
         security issues and about supported versions.
     
    +    Helped-by: Sven Strickroth <[email protected]>
         Signed-off-by: Johannes Schindelin <[email protected]>
     
      ## SECURITY.md ##
    @@ SECURITY.md: Examples for details to include:
     +
     +## Snapshot versions ('nightly builds')
     +
    -+Git for Windows also provides snapshots (these are not releases) of the the current development as per git-for-Windows/git's `master` branch at the [Snapshots](https://wingit.blob.core.windows.net/files/index.html) page. This link is also listed in the footer of the [Git for Windows](https://gitforwindows.org/) home page.
    ++Git for Windows also provides snapshots (these are not releases) of the current development as per git-for-Windows/git's `master` branch at the [Snapshots](https://wingit.blob.core.windows.net/files/index.html) page. This link is also listed in the footer of the [Git for Windows](https://gitforwindows.org/) home page.
     +
     +Note: even if those builds are not exactly "nightly", they are sometimes referred to as "nightly builds" to keep with other projects' nomenclature.
     +
  • 275: d53e464 < -: ------------- fixup! path-walk API: avoid adding a root tree more than once

  • 276: 6138534 (upstream: c95547a) < -: ------------- builtin/gc: fix crash when running git maintenance start

  • 277: becfd0a < -: ------------- fixup! builtin/gc: fix crash when running git maintenance start

  • 278: eeca006 < -: ------------- amend! survey: add --top= option and config

  • 281: c92a039 (upstream: f36b8cb) < -: ------------- git-config.1: remove value from positional args in unset usage

  • 282: 422a4e3 (upstream: 4154ed4) < -: ------------- docs: fix the maintain-git links in technical/platform-support

  • 283: 823223f (upstream: 766fce6) < -: ------------- simple-ipc: split async server initialization and running

  • 284: dacf467 (upstream: 51907f8) < -: ------------- fsmonitor: initialize fs event listener before accepting clients

  • 285: e8922bd (upstream: 9f11959) < -: ------------- cache-tree: refactor verification to return error codes

  • 286: ce5ee8d (upstream: 2be7fc0) < -: ------------- cache-tree: detect mismatching number of index entries

  • 287: 7959bb6 (upstream: ecb5c43) < -: ------------- unpack-trees: detect mismatching number of cache-tree/index entries

  • 289: 6e0b224 (upstream: 0c1a998) < -: ------------- submodule: correct remote name with fetch

  • 291: 1f8a83c < -: ------------- fixup! .github: Add configuration for the Sentiment Bot

  • 292: 087b674 < -: ------------- amend! SECURITY.md: document Git for Windows' policies

The usual stuff: non-trivial parts of the range-diff reflect either upstreamed (or finally-merged, backported) or squashed fixup! commits.

derrickstolee and others added 30 commits November 25, 2024 08:20
Add some tests based on the current behavior, doing interesting checks
for different sets of branches, ranges, and the --boundary option. This
sets a baseline for the behavior and we can extend it as new options are
introduced.

Signed-off-by: Derrick Stolee <[email protected]>
This option is still under discussion on the Git mailing list.

We still would like to have some real-world data, and the best way to
get it is to get a Git for Windows release into users' hands so that
they can test it.

Nevertheless, without the official blessing of the Git maintainer, this
optionis experimental, and we need to be clear about that.

Signed-off-by: Johannes Schindelin <[email protected]>
We add the ability to filter the object types in the path-walk API so
the callback function is called fewer times.

This adds the ability to ask for the commits in a list, as well. Future
changes will add the ability to visit annotated tags.

Signed-off-by: Derrick Stolee <[email protected]>
In anticipation of using the path-walk API to analyze tags or include
them in a pack-file, add the ability to walk the tags that were included
in the revision walk.

Signed-off-by: Derrick Stolee <[email protected]>
The sparse tree walk algorithm was created in d5d2e93 (revision:
implement sparse algorithm, 2019-01-16) and involves using the
mark_trees_uninteresting_sparse() method. This method takes a repository
and an oidset of tree IDs, some of which have the UNINTERESTING flag and
some of which do not.

Create a method that has an equivalent set of preconditions but uses a
"dense" walk (recursively visits all reachable trees, as long as they
have not previously been marked UNINTERESTING). This is an important
difference from mark_tree_uninteresting(), which short-circuits if the
given tree has the UNINTERESTING flag.

A use of this method will be added in a later change, with a condition
set whether the sparse or dense approach should be used.

Signed-off-by: Derrick Stolee <[email protected]>
This option causes the path-walk API to act like the sparse tree-walk
algorithm implemented by mark_trees_uninteresting_sparse() in
list-objects.c.

Starting from the commits marked as UNINTERESTING, their root trees and
all objects reachable from those trees are UNINTERSTING, at least as we
walk path-by-path. When we reach a path where all objects associated
with that path are marked UNINTERESTING, then do no continue walking the
children of that path.

We need to be careful to pass the UNINTERESTING flag in a deep way on
the UNINTERESTING objects before we start the path-walk, or else the
depth-first search for the path-walk API may accidentally report some
objects as interesting.

Signed-off-by: Derrick Stolee <[email protected]>
This will be helpful in a future change.

Signed-off-by: Derrick Stolee <[email protected]>
In order to more easily compute delta bases among objects that appear at the
exact same path, add a --path-walk option to 'git pack-objects'.

This option will use the path-walk API instead of the object walk given by
the revision machinery. Since objects will be provided in batches
representing a common path, those objects can be tested for delta bases
immediately instead of waiting for a sort of the full object list by
name-hash. This has multiple benefits, including avoiding collisions by
name-hash.

The objects marked as UNINTERESTING are included in these batches, so we
are guaranteeing some locality to find good delta bases.

After the individual passes are done on a per-path basis, the default
name-hash is used to find other opportunistic delta bases that did not
match exactly by the full path name.

RFC TODO: It is important to note that this option is inherently
incompatible with using a bitmap index. This walk probably also does not
work with other advanced features, such as delta islands.

Getting ahead of myself, this option compares well with --full-name-hash
when the packfile is large enough, but also performs at least as well as
the default in all cases that I've seen.

RFC TODO: this should probably be recording the batch locations to another
list so they could be processed in a second phase using threads.

RFC TODO: list some examples of how this outperforms previous pack-objects
strategies. (This is coming in later commits that include performance
test changes.)

Signed-off-by: Derrick Stolee <[email protected]>
There are many tests that validate whether 'git pack-objects' works as
expected. Instead of duplicating these tests, add a new test environment
variable, GIT_TEST_PACK_PATH_WALK, that implies --path-walk by default
when specified.

This was useful in testing the implementation of the --path-walk
implementation, especially in conjunction with test such as:

 - t0411-clone-from-partial.sh : One test fetches from a repo that does
   not have the boundary objects. This causes the path-based walk to
   fail. Disable the variable for this test.

 - t5306-pack-nobase.sh : Similar to t0411, one test fetches from a repo
   without a boundary object.

 - t5310-pack-bitmaps.sh : One test compares the case when packing with
   bitmaps to the case when packing without them. Since we disable the
   test variable when writing bitmaps, this causes a difference in the
   object list (the --path-walk option adds an extra object). Specify
   --no-path-walk in both processes for the comparison. Another test
   checks for a specific delta base, but when computing dynamically
   without using bitmaps, the base object it too small to be considered
   in the delta calculations so no base is used.

 - t5316-pack-delta-depth.sh : This script cares about certain delta
   choices and their chain lengths. The --path-walk option changes how
   these chains are selected, and thus changes the results of this test.

 - t5322-pack-objects-sparse.sh : This demonstrates the effectiveness of
   the --sparse option and how it combines with --path-walk.

 - t5332-multi-pack-reuse.sh : This test verifies that the preferred
   pack is used for delta reuse when possible. The --path-walk option is
   not currently aware of the preferred pack at all, so finds a
   different delta base.

 - t7406-submodule-update.sh : When using the variable, the --depth
   option collides with the --path-walk feature, resulting in a warning
   message. Disable the variable so this warning does not appear.

I want to call out one specific test change that is only temporary:

 - t5530-upload-pack-error.sh : One test cares specifically about an
   "unable to read" error message. Since the current implementation
   performs delta calculations within the path-walk API callback, a
   different "unable to get size" error message appears. When this
   is changed in a future refactoring, this test change can be reverted.

Signed-off-by: Derrick Stolee <[email protected]>
Since 'git pack-objects' supports a --path-walk option, allow passing it
through in 'git repack'. This presents interesting testing opportunities for
comparing the different repacking strategies against each other.

Add the --path-walk option to the performance tests in p5313.

For the microsoft/fluentui repo [1] checked out at a specific commit [2],
the results are very interesting:

Test                                           this tree
------------------------------------------------------------------
5313.2: thin pack                              0.40(0.47+0.04)
5313.3: thin pack size                                    1.2M
5313.4: thin pack with --full-name-hash        0.09(0.10+0.04)
5313.5: thin pack size with --full-name-hash             22.8K
5313.6: thin pack with --path-walk             0.08(0.06+0.02)
5313.7: thin pack size with --path-walk                  20.8K
5313.8: big pack                               2.16(8.43+0.23)
5313.9: big pack size                                    17.7M
5313.10: big pack with --full-name-hash        1.42(3.06+0.21)
5313.11: big pack size with --full-name-hash             18.0M
5313.12: big pack with --path-walk             2.21(8.39+0.24)
5313.13: big pack size with --path-walk                  17.8M
5313.14: repack                                98.05(662.37+2.64)
5313.15: repack size                                    449.1K
5313.16: repack with --full-name-hash          33.95(129.44+2.63)
5313.17: repack size with --full-name-hash              182.9K
5313.18: repack with --path-walk               106.21(121.58+0.82)
5313.19: repack size with --path-walk                   159.6K

[1] https://github.com/microsoft/fluentui
[2] e70848ebac1cd720875bccaa3026f4a9ed700e08

This repo suffers from having a lot of paths that collide in the name
hash, so examining them in groups by path leads to better deltas. Also,
in this case, the single-threaded implementation is competitive with the
full repack. This is saving time diffing files that have significant
differences from each other.

A similar, but private, repo has even more extremes in the thin packs:

Test                                           this tree
--------------------------------------------------------------
5313.2: thin pack                              2.39(2.91+0.10)
5313.3: thin pack size                                    4.5M
5313.4: thin pack with --full-name-hash        0.29(0.47+0.12)
5313.5: thin pack size with --full-name-hash             15.5K
5313.6: thin pack with --path-walk             0.35(0.31+0.04)
5313.7: thin pack size with --path-walk                  14.2K

Notice, however, that while the --full-name-hash version is working
quite well in these cases for the thin pack, it does poorly for some
other standard cases, such as this test on the Linux kernel repository:

Test                                           this tree
--------------------------------------------------------------
5313.2: thin pack                              0.01(0.00+0.00)
5313.3: thin pack size                                     310
5313.4: thin pack with --full-name-hash        0.00(0.00+0.00)
5313.5: thin pack size with --full-name-hash              1.4K
5313.6: thin pack with --path-walk             0.00(0.00+0.00)
5313.7: thin pack size with --path-walk                    310

Here, the --full-name-hash option does much worse than the default name
hash, but the path-walk option does exactly as well.

Signed-off-by: Derrick Stolee <[email protected]>
Users may want to enable the --path-walk option for 'git pack-objects' by
default, especially underneath commands like 'git push' or 'git repack'.

This should be limited to client repositories, since the --path-walk option
disables bitmap walks, so would be bad to include in Git servers when
serving fetches and clones. There is potential that it may be helpful to
consider when repacking the repository, to take advantage of improved deltas
across historical versions of the same files.

Much like how "pack.useSparse" was introduced and included in
"feature.experimental" before being enabled by default, use the repository
settings infrastructure to make the new "pack.usePathWalk" config enabled by
"feature.experimental" and "feature.manyFiles".

Signed-off-by: Derrick Stolee <[email protected]>
Repositories registered with Scalar are expected to be client-only
repositories that are rather large. This means that they are more likely to
be good candidates for using the --path-walk option when running 'git
pack-objects', especially under the hood of 'git push'. Enable this config
in Scalar repositories.

Signed-off-by: Derrick Stolee <[email protected]>
Previously, the --path-walk option to 'git pack-objects' would compute
deltas inline with the path-walk logic. This would make the progress
indicator look like it is taking a long time to enumerate objects, and
then very quickly computed deltas.

Instead of computing deltas on each region of objects organized by tree,
store a list of regions corresponding to these groups. These can later
be pulled from the list for delta compression before doing the "global"
delta search.

This presents a new progress indicator that can be used in tests to
verify that this stage is happening.

The current implementation is not integrated with threads, but could be
done in a future update.

Since we do not attempt to sort objects by size until after exploring
all trees, we can remove the previous change to t5530 due to a different
error message appearing first.

Signed-off-by: Derrick Stolee <[email protected]>
In anticipation of implementing 'git backfill', populate the necessary files
with the boilerplate of a new builtin.

RFC TODO: When preparing this for a full implementation, make sure it is
based on the newest standards introduced by [1].

[1] https://lore.kernel.org/git/[email protected]/T/#m606036ea2e75a6d6819d6b5c90e729643b0ff7f7
    [PATCH 1/3] builtin: add a repository parameter for builtin functions

Signed-off-by: Derrick Stolee <[email protected]>
Adapting the implementation of ll_find_deltas(), create a threaded
version of the --path-walk compression step in 'git pack-objects'.

This involves adding a 'regions' member to the thread_params struct,
allowing each thread to own a section of paths. We can simplify the way
jobs are split because there is no value in extending the batch based on
name-hash the way sections of the object entry array are attempted to be
grouped. We re-use the 'list_size' and 'remaining' items for the purpose
of borrowing work in progress from other "victim" threads when a thread
has finished its batch of work more quickly.

Using the Git repository as a test repo, the p5313 performance test
shows that the resulting size of the repo is the same, but the threaded
implementation gives gains of varying degrees depending on the number of
objects being packed. (This was tested on a 16-core machine.)

Test                                    HEAD~1    HEAD
-------------------------------------------------------------
5313.6: thin pack with --path-walk        0.01    0.01  +0.0%
5313.7: thin pack size with --path-walk    475     475  +0.0%
5313.12: big pack with --path-walk        1.99    1.87  -6.0%
5313.13: big pack size with --path-walk  14.4M   14.3M  -0.4%
5313.18: repack with --path-walk         98.14   41.46 -57.8%
5313.19: repack size with --path-walk   197.2M  197.3M  +0.0%

Signed-off-by: Derrick Stolee <[email protected]>
When adding tree objects, we are very careful to avoid adding the same
tree object more than once. There was one small gap in that logic,
though: when adding a root tree object. Two refs can easily share the
same root tree object, and we should still not add it more than once.

Signed-off-by: Johannes Schindelin <[email protected]>
The default behavior of 'git backfill' is to fetch all missing blobs that
are reachable from HEAD. Document and test this behavior.

The implementation is a very simple use of the path-walk API, initializing
the revision walk at HEAD to start the path-walk from all commits reachable
from HEAD. Ignore the object arrays that correspond to tree entries,
assuming that they are all present already.

Signed-off-by: Derrick Stolee <[email protected]>
Users may want to specify a minimum batch size for their needs. This is only
a minimum: the path-walk API provides a list of OIDs that correspond to the
same path, and thus it is optimal to allow delta compression across those
objects in a single server request.

We could consider limiting the request to have a maximum batch size in the
future.

Signed-off-by: Derrick Stolee <[email protected]>
One way to significantly reduce the cost of a Git clone and later fetches is
to use a blobless partial clone and combine that with a sparse-checkout that
reduces the paths that need to be populated in the working directory. Not
only does this reduce the cost of clones and fetches, the sparse-checkout
reduces the number of objects needed to download from a promisor remote.

However, history investigations can be expensie as computing blob diffs will
trigger promisor remote requests for one object at a time. This can be
avoided by downloading the blobs needed for the given sparse-checkout using
'git backfill' and its new '--sparse' mode, at a time that the user is
willing to pay that extra cost.

Note that this is distinctly different from the '--filter=sparse:<oid>'
option, as this assumes that the partial clone has all reachable trees and
we are using client-side logic to avoid downloading blobs outside of the
sparse-checkout cone. This avoids the server-side cost of walking trees
while also achieving a similar goal. It also downloads in batches based on
similar path names, presenting a resumable download if things are
interrupted.

This augments the path-walk API to have a possibly-NULL 'pl' member that may
point to a 'struct pattern_list'. This could be more general than the
sparse-checkout definition at HEAD, but 'git backfill --sparse' is currently
the only consumer.

Be sure to test this in both cone mode and not cone mode. Cone mode has the
benefit that the path-walk can skip certain paths once they would expand
beyond the sparse-checkout.

Signed-off-by: Derrick Stolee <[email protected]>
The previous change introduced the '--[no-]sparse' option for the 'git
backfill' command, but did not assume it as enabled by default. However,
this is likely the behavior that users will most often want to happen.
Without this default, users with a small sparse-checkout may be confused
when 'git backfill' downloads every version of every object in the full
history.

However, this is left as a separate change so this decision can be reviewed
independently of the value of the '--[no-]sparse' option.

Add a test of adding the '--sparse' option to a repo without sparse-checkout
to make it clear that supplying it without a sparse-checkout is an error.

Signed-off-by: Derrick Stolee <[email protected]>
This is a highly useful command, and we want it to get some testing "in
the wild". However, the patches have not yet been reviewed on the Git
mailing list, and are therefore subject to change. By marking the
command as experimental, users will be warned to pay attention to those
changes.

Signed-off-by: Johannes Schindelin <[email protected]>
Start work on a new 'git survey' command to scan the repository
for monorepo performance and scaling problems.  The goal is to
measure the various known "dimensions of scale" and serve as a
foundation for adding additional measurements as we learn more
about Git monorepo scaling problems.

The initial goal is to complement the scanning and analysis performed
by the GO-based 'git-sizer' (https://github.com/github/git-sizer) tool.
It is hoped that by creating a builtin command, we may be able to take
advantage of internal Git data structures and code that is not
accessible from GO to gain further insight into potential scaling
problems.

Co-authored-by: Derrick Stolee <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Derrick Stolee <[email protected]>
By default we will scan all references in "refs/heads/", "refs/tags/"
and "refs/remotes/".

Add command line opts let the use ask for all refs or a subset of them
and to include a detached HEAD.

Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Derrick Stolee <[email protected]>
When 'git survey' provides information to the user, this will be presented
in one of two formats: plaintext and JSON. The JSON implementation will be
delayed until the functionality is complete for the plaintext format.

The most important parts of the plaintext format are headers specifying the
different sections of the report and tables providing concreted data.

Create a custom table data structure that allows specifying a list of
strings for the row values. When printing the table, check each column for
the maximum width so we can create a table of the correct size from the
start.

The table structure is designed to be flexible to the different kinds of
output that will be implemented in future changes.

Signed-off-by: Derrick Stolee <[email protected]>
At the moment, nothing is obvious about the reason for the use of the
path-walk API, but this will become more prevelant in future iterations. For
now, use the path-walk API to sum up the counts of each kind of object.

For example, this is the reachable object summary output for my local repo:

REACHABLE OBJECT SUMMARY
========================
Object Type |  Count
------------+-------
       Tags |   1343
    Commits | 179344
      Trees | 314350
      Blobs | 184030

Signed-off-by: Derrick Stolee <[email protected]>
Now that we have explored objects by count, we can expand that a bit more to
summarize the data for the on-disk and inflated size of those objects. This
information is helpful for diagnosing both why disk space (and perhaps
clone or fetch times) is growing but also why certain operations are slow
because the inflated size of the abstract objects that must be processed is
so large.

Signed-off-by: Derrick Stolee <[email protected]>
In future changes, we will make use of these methods. The intention is to
keep track of the top contributors according to some metric. We don't want
to store all of the entries and do a sort at the end, so track a
constant-size table and remove rows that get pushed out depending on the
chosen sorting algorithm.

Co-authored-by: Jeff Hostetler <[email protected]>
Signed-off-by; Jeff Hostetler <[email protected]>
Signed-off-by: Derrick Stolee <[email protected]>
Since we are already walking our reachable objects using the path-walk API,
let's now collect lists of the paths that contribute most to different
metrics. Specifically, we care about

 * Number of versions.
 * Total size on disk.
 * Total inflated size (no delta or zlib compression).

This information can be critical to discovering which parts of the
repository are causing the most growth, especially on-disk size. Different
packing strategies might help compress data more efficiently, but the toal
inflated size is a representation of the raw size of all snapshots of those
paths. Even when stored efficiently on disk, that size represents how much
information must be processed to complete a command such as 'git blame'.

Since the on-disk size is likely to be fragile, stop testing the exact
output of 'git survey' and check that the correct set of headers is
output.

Signed-off-by: Derrick Stolee <[email protected]>
The 'git survey' builtin provides several detail tables, such as "top
files by on-disk size". The size of these tables defaults to 10,
currently.

Allow the user to specify this number via a new --top=<N> option or the
new survey.top config key.

Signed-off-by: Derrick Stolee <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
shiftkey and others added 22 commits November 25, 2024 09:23
With improvements by Clive Chan, Adric Norris, Ben Bodenmiller and
Philip Oakley.

Helped-by: Clive Chan <[email protected]>
Helped-by: Adric Norris <[email protected]>
Helped-by: Ben Bodenmiller <[email protected]>
Helped-by: Philip Oakley <[email protected]>
Signed-off-by: Brendan Forster <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Git for Windows accepts pull requests; Core Git does not. Therefore we
need to adjust the template (because it only matches core Git's
project management style, not ours).

Also: direct Git for Windows enhancements to their contributions page,
space out the text for easy reading, and clarify that the mailing list
is plain text, not HTML.

Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Git documentation refers to $HOME and $XDG_CONFIG_HOME often, but does not specify how or where these values come from on Windows where neither is set by default. The new documentation reflects the behavior of setup_windows_environment() in compat/mingw.c.

Signed-off-by: Alejandro Barreto <[email protected]>
This is the recommended way on GitHub to describe policies revolving around
security issues and about supported versions.

Helped-by: Sven Strickroth <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
These are Git for Windows' Git GUI and gitk patches. We will have to
decide at some point what to do about them, but that's a little lower
priority (as Git GUI seems to be unmaintained for the time being, and
the gitk maintainer keeps a very low profile on the Git mailing list,
too).

Signed-off-by: Johannes Schindelin <[email protected]>
…dvice

clean: suggest using `core.longPaths` if paths are too long to remove
Signed-off-by: Johannes Schindelin <[email protected]>
This was pull request git-for-windows#1645 from ZCube/master

Support windows container.

Signed-off-by: Johannes Schindelin <[email protected]>
…ws#4527)

With this patch, Git for Windows works as intended on mounted APFS
volumes (where renaming read-only files would fail).

Signed-off-by: Johannes Schindelin <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
This patch introduces support to set special NTFS attributes that are
interpreted by the Windows Subsystem for Linux as file mode bits, UID
and GID.

Signed-off-by: Johannes Schindelin <[email protected]>
Handle Ctrl+C in Git Bash nicely

Signed-off-by: Johannes Schindelin <[email protected]>
A fix for calling `vim` in Windows Terminal caused a regression and was
reverted. We partially un-revert this, to get the fix again.

Signed-off-by: Johannes Schindelin <[email protected]>
This topic branch re-adds the deprecated --stdin/-z options to `git
reset`. Those patches were overridden by a different set of options in
the upstream Git project before we could propose `--stdin`.

We offered this in MinGit to applications that wanted a safer way to
pass lots of pathspecs to Git, and these applications will need to be
adjusted.

Instead of `--stdin`, `--pathspec-from-file=-` should be used, and
instead of `-z`, `--pathspec-file-nul`.

Signed-off-by: Johannes Schindelin <[email protected]>
Originally introduced as `core.useBuiltinFSMonitor` in Git for Windows
and developed, improved and stabilized there, the built-in FSMonitor
only made it into upstream Git (after unnecessarily long hemming and
hawing and throwing overly perfectionist style review sticks into the
spokes) as `core.fsmonitor = true`.

In Git for Windows, with this topic branch, we re-introduce the
now-obsolete config setting, with warnings suggesting to existing users
how to switch to the new config setting, with the intention to
ultimately drop the patch at some stage.

Signed-off-by: Johannes Schindelin <[email protected]>
…updates

Start monitoring updates of Git for Windows' component in the open
Add a README.md for GitHub goodness.

Signed-off-by: Johannes Schindelin <[email protected]>
@dscho dscho requested a review from mjcheetham November 25, 2024 08:27
@dscho dscho self-assigned this Nov 25, 2024
@dscho dscho linked an issue Nov 25, 2024 that may be closed by this pull request
@dscho dscho added this to the Next release milestone Nov 25, 2024
@dscho
Copy link
Member Author

dscho commented Nov 25, 2024

/git-artifacts

The tag-git workflow run was started

The git-artifacts-x86_64 workflow run was started.
The git-artifacts-i686 workflow run was started.
The git-artifacts-aarch64 workflow run was started.

@dscho
Copy link
Member Author

dscho commented Nov 25, 2024

/release

The release-git workflow run was started

@gitforwindowshelper gitforwindowshelper bot merged commit 2cd2243 into git-for-windows:main Nov 25, 2024
67 of 68 checks passed
@dscho dscho deleted the rebase-to-v2.47.1 branch November 25, 2024 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[New git version] v2.47.1