Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE YET] Rebase to v2.45.0 #4923

Closed
wants to merge 203 commits into from

Conversation

dscho
Copy link
Member

@dscho dscho commented Apr 23, 2024

Range-diff relative to vfs-2.44.0
  • 189: abd7968 = 1: dfb63a6 for-each-repo: optionally keep going on an error

  • 190: 1ae1155 = 2: d70b66b maintenance: running maintenance should not stop on errors

  • 1: 08acea8 = 3: 291baa1 reset --stdin: trim carriage return from the paths

  • 2: a75168a < -: ----------- gvfs: start by adding the -gvfs suffix to the version

  • -: ----------- > 4: 7171e5c Identify microsoft/git via a distinct version suffix

  • 3: 1db481c = 5: 45448ec gvfs: ensure that the version is based on a GVFS tag

  • 4: 7cc11e5 = 6: 3f492fa gvfs: add a GVFS-specific header file

  • 5: 656515f = 7: 75ddd14 gvfs: add the core.gvfs config setting

  • 6: 2ad35f3 ! 8: 0dec18a gvfs: add the feature to skip writing the index' SHA-1

    @@ repo-settings.c: void prepare_repo_settings(struct repository *r)
      	 * The GIT_TEST_MULTI_PACK_INDEX variable is special in that
      	 * either it *or* the config sets
     
    - ## t/t1016-read-tree-skip-sha-on-read.sh (new) ##
    + ## t/t1017-read-tree-skip-sha-on-read.sh (new) ##
     @@
     +#!/bin/sh
     +
  • 7: 69804bb = 9: 8b5f8b6 gvfs: add the feature that blobs may be missing

  • 8: 08e1394 = 10: d2756a0 gvfs: prevent files to be deleted outside the sparse checkout

  • 9: 8f94806 = 11: d2d9185 gvfs: optionally skip reachability checks/upload pack during fetch

  • 10: 425f60c = 12: cc89974 gvfs: ensure all filters and EOL conversions are blocked

  • 11: 1809338 ! 13: 01ec987 gvfs: allow "virtualizing" objects

    @@ Commit message
         to "lazy clone" and work more on this.
     
         Signed-off-by: Ben Peart <[email protected]>
    +    Signed-off-by: Johannes Schindelin <[email protected]>
     
      ## config.c ##
     @@ config.c: int git_default_core_config(const char *var, const char *value,
    @@ environment.c: int core_gvfs;
     
      ## environment.h ##
     @@ environment.h: struct strvec;
    - extern char comment_line_char;
    + extern const char *comment_line_str;
      extern int auto_comment_line_char;
      
     +extern int core_virtualize_objects;
    @@ environment.h: struct strvec;
     
      ## object-file.c ##
     @@
    - #include "setup.h"
    - #include "submodule.h"
      #include "fsck.h"
    + #include "loose.h"
    + #include "object-file-convert.h"
     +#include "trace.h"
     +#include "hook.h"
      
  • 12: 6829117 ! 14: 77ececa Hydrate missing loose objects in check_and_freshen()

    @@ contrib/long-running-read-object/example.pl (new)
     
      ## object-file.c ##
     @@
    - #include "fsck.h"
    + #include "object-file-convert.h"
      #include "trace.h"
      #include "hook.h"
     +#include "sigchain.h"
  • 13: 41d24c3 ! 15: 1ebd6a6 sha1_file: when writing objects, skip the read_object_hook

    @@ object-file.c: int stream_loose_object(struct input_stream *in_stream, size_t le
      		goto cleanup;
      	}
     @@ object-file.c: int write_object_file_flags(const void *buf, size_t len,
    + 	 * it out into .git/objects/??/?{38} file.
      	 */
    - 	write_object_file_prepare(the_hash_algo, buf, len, type, oid, hdr,
    - 				  &hdrlen);
    + 	write_object_file_prepare(algo, buf, len, type, oid, hdr, &hdrlen);
     -	if (freshen_packed_object(oid) || freshen_loose_object(oid))
     +	if (freshen_packed_object(oid) || freshen_loose_object(oid, 1))
      		return 0;
    - 	return write_loose_object(oid, hdr, hdrlen, buf, len, 0, flags);
    - }
    + 	if (write_loose_object(oid, hdr, hdrlen, buf, len, 0, flags))
    + 		return -1;
     @@ object-file.c: int write_object_file_literally(const void *buf, size_t len,
      
      	if (!(flags & HASH_WRITE_OBJECT))
    @@ object-file.c: int write_object_file_literally(const void *buf, size_t len,
     +	if (freshen_packed_object(oid) || freshen_loose_object(oid, 1))
      		goto cleanup;
      	status = write_loose_object(oid, header, hdrlen, buf, len, 0, 0);
    - 
    + 	if (compat_type != -1)
     
      ## t/t0410/read-object ##
     @@ t/t0410/read-object: while (1) {
  • 14: c10e77d ! 16: 8daad9f gvfs: add global command pre and post hook procs

    @@ git.c: static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
     +
      	trace_argv_printf(argv, "trace: built-in: git");
      	trace2_cmd_name(p->cmd);
    - 	trace2_cmd_list_config();
    - 	trace2_cmd_list_env_vars();
      
      	validate_cache_entries(the_repository->index);
     -	status = p->fn(argc, argv, prefix);
  • 15: 5a78b0c = 17: 3cf79a7 t0400: verify that the hook is called correctly from a subdirectory

  • 16: bcbcbb1 = 18: da90d91 Pass PID of git process to hooks.

  • 17: 2135705 = 19: 5567a3e pre-command: always respect core.hooksPath

  • 18: 8173d74 = 20: 2fa2c0c sparse-checkout: update files with a modify/delete conflict

  • 19: 5cf5793 = 21: e94543d sparse-checkout: avoid writing entries with the skip-worktree bit

  • 20: 9fea45a = 22: 43b4268 Do not remove files outside the sparse-checkout

  • 21: 203814b = 23: 7804aff send-pack: do not check for sha1 file when GVFS_MISSING_OK set

  • 22: 9f0e5f9 = 24: 26b3cbc cache-tree: remove use of strbuf_addf in update_one

  • 23: d724e71 ! 25: 6b2941f gvfs: block unsupported commands when running in a GVFS repo

    @@ Commit message
         11) worktree
     
         Signed-off-by: Ben Peart <[email protected]>
    +    Signed-off-by: Johannes Schindelin <[email protected]>
     
      ## builtin/gc.c ##
     @@
    @@ builtin/gc.c: int cmd_gc(int argc, const char **argv, const char *prefix)
      	if (quiet)
      		strvec_push(&repack, "-q");
      
    -+	if ((!auto_gc || (auto_gc && gc_auto_threshold > 0)) && gvfs_config_is_set(GVFS_BLOCK_COMMANDS))
    ++	if ((!opts.auto_flag || (opts.auto_flag && gc_auto_threshold > 0)) && gvfs_config_is_set(GVFS_BLOCK_COMMANDS))
     +		die(_("'git gc' is not supported on a GVFS repo"));
     +
    - 	if (auto_gc) {
    + 	if (opts.auto_flag) {
      		/*
      		 * Auto-gc should be least intrusive as possible.
     
  • 24: 40bf6bd = 26: eb1a110 worktree: allow in Scalar repositories

  • 25: a202667 = 27: e56fd83 gvfs: allow overriding core.gvfs

  • 26: 12b6663 = 28: 143f6c8 BRANCHES.md: Add explanation of branches and using forks

  • 27: 81dc570 ! 29: 8530efe Add virtual file system settings and hook proc

    @@ dir.c: enum pattern_match_result path_matches_pattern_list(
      	int result = NOT_MATCHED;
      	size_t slash_pos;
      
    -+	/*
    -+	 * The virtual file system data is used to prevent git from traversing
    -+	 * any part of the tree that is not in the virtual file system.  Return
    -+	 * 1 to exclude the entry if it is not found in the virtual file system,
    -+	 * else fall through to the regular excludes logic as it may further exclude.
    -+	 */
    -+	if (*dtype == DT_UNKNOWN)
    -+		*dtype = resolve_dtype(DT_UNKNOWN, istate, pathname, pathlen);
    -+	if (is_excluded_from_virtualfilesystem(pathname, pathlen, *dtype) > 0)
    -+		return 1;
    ++	if (core_virtualfilesystem) {
    ++		/*
    ++		* The virtual file system data is used to prevent git from traversing
    ++		* any part of the tree that is not in the virtual file system.  Return
    ++		* 1 to exclude the entry if it is not found in the virtual file system,
    ++		* else fall through to the regular excludes logic as it may further exclude.
    ++		*/
    ++		if (*dtype == DT_UNKNOWN)
    ++			*dtype = resolve_dtype(DT_UNKNOWN, istate, pathname, pathlen);
    ++		if (is_excluded_from_virtualfilesystem(pathname, pathlen, *dtype) > 0)
    ++			return 1;
    ++	}
     +
      	if (!pl->use_cone_patterns) {
      		pattern = last_matching_pattern_from_list(pathname, pathlen, basename,
    @@ dir.c: struct path_pattern *last_matching_pattern(struct dir_struct *dir,
     -		last_matching_pattern(dir, istate, pathname, dtype_p);
     +	struct path_pattern *pattern;
     +
    -+	/*
    -+	 * The virtual file system data is used to prevent git from traversing
    -+	 * any part of the tree that is not in the virtual file system.  Return
    -+	 * 1 to exclude the entry if it is not found in the virtual file system,
    -+	 * else fall through to the regular excludes logic as it may further exclude.
    -+	 */
    -+	if (*dtype_p == DT_UNKNOWN)
    -+		*dtype_p = resolve_dtype(DT_UNKNOWN, istate, pathname, strlen(pathname));
    -+	if (is_excluded_from_virtualfilesystem(pathname, strlen(pathname), *dtype_p) > 0)
    -+		return 1;
    ++	if (core_virtualfilesystem) {
    ++		/*
    ++		* The virtual file system data is used to prevent git from traversing
    ++		* any part of the tree that is not in the virtual file system.  Return
    ++		* 1 to exclude the entry if it is not found in the virtual file system,
    ++		* else fall through to the regular excludes logic as it may further exclude.
    ++		*/
    ++		if (*dtype_p == DT_UNKNOWN)
    ++			*dtype_p = resolve_dtype(DT_UNKNOWN, istate, pathname, strlen(pathname));
    ++		if (is_excluded_from_virtualfilesystem(pathname, strlen(pathname), *dtype_p) > 0)
    ++			return 1;
    ++	}
     +
     +	pattern = last_matching_pattern(dir, istate, pathname, dtype_p);
      	if (pattern)
  • 28: 8c1e2c8 = 30: d2859b2 virtualfilesystem: don't run the virtual file system hook if the index has been redirected

  • 29: 456cdb5 = 31: 69e43b2 virtualfilesystem: check if directory is included

  • 30: 3e677ad = 32: 6707a3b backwards-compatibility: support the post-indexchanged hook

  • 31: 9e26884 = 33: 0da573b gvfs: verify that the built-in FSMonitor is disabled

  • 32: a3f9c23 ! 34: cdad578 status: add status serialization mechanism

    @@ builtin/commit.c: static int opt_parse_porcelain(const struct option *opt, const
      static int opt_parse_m(const struct option *opt, const char *arg, int unset)
      {
      	struct strbuf *buf = opt->value;
    -@@ builtin/commit.c: static void handle_untracked_files_arg(struct wt_status *s)
    - 		s->show_untracked_files = SHOW_NORMAL_UNTRACKED_FILES;
    - 	else if (!strcmp(untracked_files_arg, "all"))
    - 		s->show_untracked_files = SHOW_ALL_UNTRACKED_FILES;
    -+	else if (!strcmp(untracked_files_arg,"complete"))
    -+		s->show_untracked_files = SHOW_COMPLETE_UNTRACKED_FILES;
    - 	/*
    - 	 * Please update $__git_untracked_file_modes in
    - 	 * git-completion.bash when you add new options
    +@@ builtin/commit.c: static enum untracked_status_type parse_untracked_setting_name(const char *u)
    + 		return SHOW_NORMAL_UNTRACKED_FILES;
    + 	else if (!strcmp(u, "all"))
    + 		return SHOW_ALL_UNTRACKED_FILES;
    ++	else if (!strcmp(u,"complete"))
    ++		return SHOW_COMPLETE_UNTRACKED_FILES;
    + 	else
    + 		return SHOW_UNTRACKED_FILES_ERROR;
    + }
     @@ builtin/commit.c: static int git_status_config(const char *k, const char *v,
      		s->relative_paths = git_config_bool(k, v);
      		return 0;
    @@ builtin/commit.c: static int git_status_config(const char *k, const char *v,
     +		return 0;
     +	}
      	if (!strcmp(k, "status.showuntrackedfiles")) {
    - 		if (!v)
    - 			return config_error_nonbool(k);
    + 		enum untracked_status_type u;
    + 
     @@ builtin/commit.c: int cmd_status(int argc, const char **argv, const char *prefix)
      	static const char *rename_score_arg = (const char *)-1;
      	static struct wt_status s;
    @@ wt-status.h
      #include "remote.h"
      
      struct repository;
    -@@ wt-status.h: enum color_wt_status {
    - enum untracked_status_type {
    - 	SHOW_NO_UNTRACKED_FILES,
    +@@ wt-status.h: enum untracked_status_type {
    + 	SHOW_UNTRACKED_FILES_ERROR = -1,
    + 	SHOW_NO_UNTRACKED_FILES = 0,
      	SHOW_NORMAL_UNTRACKED_FILES,
     -	SHOW_ALL_UNTRACKED_FILES
     +	SHOW_ALL_UNTRACKED_FILES,
  • 33: afa8244 = 35: 21ab719 Teach ahead-behind and serialized status to play nicely together

  • 34: b9a54bf = 36: 01dd8aa status: serialize to path

  • 35: 9adbd52 = 37: 1ca8d3d status: reject deserialize in V2 and conflicts

  • 36: 3f92ef4 = 38: 0d42119 serialize-status: serialize global and repo-local exclude file metadata

  • 37: 3533ff3 ! 39: 2110517 status: deserialization wait

    @@ builtin/commit.c: static int git_status_config(const char *k, const char *v,
     +		return 0;
     +	}
      	if (!strcmp(k, "status.showuntrackedfiles")) {
    - 		if (!v)
    - 			return config_error_nonbool(k);
    + 		enum untracked_status_type u;
    + 
     @@ builtin/commit.c: int cmd_status(int argc, const char **argv, const char *prefix)
      		{ OPTION_CALLBACK, 0, "deserialize", NULL,
      		  N_("path"), N_("deserialize raw status data from file"),
  • 38: 4f0dc95 = 40: cf81231 merge-recursive: avoid confusing logic in was_dirty()

  • 39: a6513c4 = 41: 6393c9e merge-recursive: add some defensive coding to was_dirty()

  • 40: 6207355 = 42: 2ea6a15 merge-recursive: teach was_dirty() about the virtualfilesystem

  • 41: f37bdd5 = 43: 0164495 status: deserialize with -uno does not print correct hint

  • 42: 25c3e0b = 44: 7f27df8 fsmonitor: check CE_FSMONITOR_VALID in ce_uptodate

  • 43: d23c6d3 = 45: b2289c3 fsmonitor: add script for debugging and update script for tests

  • 44: 7788fa9 = 46: 178f355 status: disable deserialize when verbose output requested.

  • 45: da3a063 = 47: 645bcd7 t7524: add test for verbose status deserialzation

  • 46: bc53d52 = 48: 81c16ea deserialize-status: silently fallback if we cannot read cache file

  • 47: cbd5040 ! 49: 67c0345 gvfs:trace2:data: add trace2 tracing around read_object_process

    @@ Commit message
     
      ## object-file.c ##
     @@
    - #include "submodule.h"
    - #include "fsck.h"
    + #include "loose.h"
    + #include "object-file-convert.h"
      #include "trace.h"
     +#include "trace2.h"
      #include "hook.h"
  • 48: e77727e = 50: 99d7bb0 gvfs:trace2:data: status deserialization information

  • 49: 69bc46a = 51: 4066c3a gvfs:trace2:data: status serialization

  • 50: 375bcec = 52: fea657c gvfs:trace2:data: add vfs stats

  • 51: 6c017b4 = 53: 2b6aec5 trace2: refactor setting process starting time

  • 52: 50541e7 = 54: 648829b trace2:gvfs:experiment: clear_ce_flags_1

  • 53: e84e9bb ! 55: 27d8220 trace2:gvfs:experiment: report_tracking

    @@ Commit message
     
      ## builtin/checkout.c ##
     @@ builtin/checkout.c: static void update_refs_for_switch(const struct checkout_opts *opts,
    - 	remove_branch_state(the_repository, !opts->quiet);
      	strbuf_release(&msg);
      	if (!opts->quiet &&
    --	    (new_branch_info->path || (!opts->force_detach && !strcmp(new_branch_info->name, "HEAD"))))
    -+	    (new_branch_info->path || (!opts->force_detach && !strcmp(new_branch_info->name, "HEAD")))) {
    + 	    !opts->force_detach &&
    +-	    (new_branch_info->path || !strcmp(new_branch_info->name, "HEAD")))
    ++	    (new_branch_info->path || !strcmp(new_branch_info->name, "HEAD"))) {
     +		trace2_region_enter("exp", "report_tracking", the_repository);
      		report_tracking(new_branch_info);
     +		trace2_region_leave("exp", "report_tracking", the_repository);
  • 54: 7d83388 = 56: 0965bf1 trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache

  • 55: 2e92c18 = 57: 36ae2c4 trace2:gvfs:experiment: read-cache: time read/write of cache-tree extension

  • 56: f9a546e = 58: a4aa7f8 trace2:gvfs:experiment: add region to apply_virtualfilesystem()

  • 57: 8a09e08 = 59: b5e088c trace2:gvfs:experiment: add region around unpack_trees()

  • 58: 2424bb0 = 60: faaefd9 trace2:gvfs:experiment: add region to cache_tree_fully_valid()

  • 59: dfa422c ! 61: b96ee3c trace2:gvfs:experiment: add unpack_entry() counter to unpack_trees() and report_tracking()

    @@ builtin/checkout.c
      #include "path.h"
      #include "preload-index.h"
     @@ builtin/checkout.c: static void update_refs_for_switch(const struct checkout_opts *opts,
    - 	strbuf_release(&msg);
      	if (!opts->quiet &&
    - 	    (new_branch_info->path || (!opts->force_detach && !strcmp(new_branch_info->name, "HEAD")))) {
    + 	    !opts->force_detach &&
    + 	    (new_branch_info->path || !strcmp(new_branch_info->name, "HEAD"))) {
     +		unsigned long nr_unpack_entry_at_start;
     +
      		trace2_region_enter("exp", "report_tracking", the_repository);
  • 60: d9fbd93 = 62: 34bed83 trace2:gvfs:experiment: increase default event depth for unpack-tree data

  • 61: ee1ef57 = 63: da2922f trace2:gvfs:experiment: add data for check_updates() in unpack_trees()

  • 62: 41c5b12 ! 64: 6e7cb09 Trace2:gvfs:experiment: capture more 'tracking' details

    @@ Commit message
     
      ## builtin/checkout.c ##
     @@ builtin/checkout.c: static void update_refs_for_switch(const struct checkout_opts *opts,
    - 	    (new_branch_info->path || (!opts->force_detach && !strcmp(new_branch_info->name, "HEAD")))) {
    + 	    (new_branch_info->path || !strcmp(new_branch_info->name, "HEAD"))) {
      		unsigned long nr_unpack_entry_at_start;
      
     -		trace2_region_enter("exp", "report_tracking", the_repository);
  • 63: 49870a5 = 65: f7ea389 credential: set trace2_child_class for credential manager children

  • 64: f6fca70 = 66: 4607e7a sub-process: do not borrow cmd pointer from caller

  • 65: 327886f = 67: 39242d3 sub-process: add subprocess_start_argv()

  • 66: 154bddb = 68: e7e2aa1 sha1-file: add function to update existing loose object cache

  • 67: 99bb46e = 69: 67419c8 packfile: add install_packed_git_and_mru()

  • 68: 1095cd4 = 70: 00ebc76 index-pack: avoid immediate object fetch while parsing packfile

  • 69: 5c54630 ! 71: af90c33 gvfs-helper: create tool to fetch objects using the GVFS Protocol

    @@ Makefile: LIB_OBJS += gpg-interface.o
      LIB_OBJS += hashmap.o
      LIB_OBJS += help.o
     @@ Makefile: endif
    - 	endif
    +         endif
      	BASIC_CFLAGS += $(CURL_CFLAGS)
      
     +	PROGRAM_OBJS += gvfs-helper.o
  • 70: f3dc228 = 72: 83b2721 sha1-file: create shared-cache directory if it doesn't exist

  • 71: 73a6cc3 = 73: 2339e36 gvfs-helper: better handling of network errors

  • 72: ca24fc3 = 74: c1ae8f0 gvfs-helper-client: properly update loose cache with fetched OID

  • 73: 0f79deb = 75: 8606223 gvfs-helper: V2 robust retry and throttling

  • 74: a5e5dee = 76: 5840ca3 gvfs-helper: expose gvfs/objects GET and POST semantics

  • 75: 08fc2ee = 77: dcc1811 gvfs-helper: dramatically reduce progress noise

  • 76: c503dd5 = 78: 6b467dd gvfs-helper-client.h: define struct object_id

  • 77: 531ffc5 = 79: 623b1ab gvfs-helper: handle pack-file after single POST request

  • 78: ca192b7 = 80: 77907e3 test-gvfs-prococol, t5799: tests for gvfs-helper

  • 79: 1f34846 = 81: 86f38f4 gvfs-helper: move result-list construction into install functions

  • 80: 1528d92 = 82: d42183f t5799: add support for POST to return either a loose object or packfile

  • 81: 03298f9 = 83: b876a22 t5799: cleanup wc-l and grep-c lines

  • 82: 08188f9 = 84: 877b631 gvfs-helper: verify loose objects after write

  • 83: 6a55d77 = 85: 26d0ff9 t7599: create corrupt blob test

  • 84: 05c4105 = 86: 4bbe382 gvfs-helper: add prefetch support

  • 85: 328ac49 = 87: 85fd4a2 gvfs-helper: add prefetch .keep file for last packfile

  • 86: 3bd3ff1 = 88: 8cff89a gvfs-helper: do one read in my_copy_fd_len_tail()

  • 87: 85b6d44 = 89: ff060f1 gvfs-helper: move content-type warning for prefetch packs

  • 88: f71d334 = 90: 5d6087a fetch: use gvfs-helper prefetch under config

  • 89: f752421 = 91: 818f1ad gvfs-helper: better support for concurrent packfile fetches

  • 90: 0f68cb6 = 92: 2c8bec6 remote-curl: do not call fetch-pack when using gvfs-helper

  • 91: d356ff2 = 93: 64264fb fetch: reprepare packs before checking connectivity

  • 92: 0dac5bb = 94: 68873c4 gvfs-helper: retry when creating temp files

  • 93: 2129426 = 95: b7becf0 sparse: avoid warnings about known cURL issues in gvfs-helper.c

  • 94: 14738b6 = 96: 874f2f3 gvfs-helper: add --max-retries to prefetch verb

  • 97: ac61687 = 97: 38a71bf maintenance: care about gvfs.sharedCache config

  • 95: 1865729 = 98: 4a919c3 t5799: add tests to detect corrupt pack/idx files in prefetch

  • 98: 76cd2f0 ! 99: 6d0231b unpack-trees:virtualfilesystem: Improve efficiency of clear_ce_flags

    @@ name-hash.c: struct cache_entry *index_file_exists(struct index_state *istate, c
      	if (!istate->name_hash_initialized)
     
      ## name-hash.h ##
    -@@ name-hash.h: struct index_state;
    - int index_dir_exists(struct index_state *istate, const char *name, int namelen);
    +@@ name-hash.h: int index_dir_find(struct index_state *istate, const char *name, int namelen,
    + 
      void adjust_dirname_case(struct index_state *istate, char *name);
      struct cache_entry *index_file_exists(struct index_state *istate, const char *name, int namelen, int igncase);
     +struct cache_entry *index_file_next_match(struct index_state *istate, struct cache_entry *ce, int igncase);
  • 96: 4243dfc = 100: ad66fd9 gvfs-helper: ignore .idx files in prefetch multi-part responses

  • 99: d3cc952 = 101: 1fca3d4 homebrew: add GitHub workflow to release Cask

  • 100: baa69bf = 102: b379436 Adding winget workflows

  • 101: 1445711 = 103: 0ff11b0 Disable the monitor-components workflow in msft-git

  • 102: 0c9f22e = 104: 78396e5 .github: enable windows builds on microsoft fork

  • 103: 5a3a699 = 105: 54ee216 release: create initial Windows installer build workflow

  • 104: a751ad0 = 106: c69bc4c help: special-case HOST_CPU universal

  • 105: 55fe1c4 = 107: 89ccded release: add Mac OSX installer build

  • 106: 1015e36 = 108: f067789 release: build unsigned Ubuntu .deb package

  • 107: 5c21313 = 109: 1620b43 release: add signing step for .deb package

  • 149: 3d95968 ! 110: 995377c release: create draft GitHub release with packages & installers

    @@ .github/workflows/build-git-installers.yml: jobs:
     +      - create-linux-artifacts
     +      - create-macos-artifacts
     +      - windows_artifacts
    ++      - prereqs
     +    if: |
     +      success() ||
     +        (needs.create-linux-artifacts.result == 'skipped' &&
  • 150: 7acf5f5 ! 111: 1e4a6a3 build-git-installers: publish gpg public key

    @@ Commit message
     
      ## .github/workflows/build-git-installers.yml ##
     @@ .github/workflows/build-git-installers.yml: jobs:
    -       - create-linux-artifacts
            - create-macos-artifacts
            - windows_artifacts
    +       - prereqs
     +    env:
     +      AZURE_VAULT: ${{ secrets.AZURE_VAULT }}
     +      GPG_PUBLIC_KEY_SECRET_NAME: ${{ secrets.GPG_PUBLIC_KEY_SECRET_NAME }}
  • 151: 6cd4ad4 = 112: 459814c release: continue pestering until user upgrades

  • 152: 0035b9f = 113: 4fad2e2 Makefile: allow specifying GIT_BUILT_FROM_COMMIT

  • 153: 7db2e99 = 114: 9afa61e dist: archive HEAD instead of HEAD^{tree}

  • 154: 9e989ee = 115: 169aad4 release: include GIT_BUILT_FROM_COMMIT in MacOS build

  • 155: 23dd028 ! 116: 78a2c46 release: add installer validation

    @@ .github/workflows/build-git-installers.yml: jobs:
     +          - os: macos-latest
     +            artifact: macos-artifacts
     +            command: git
    -+          - os: macos-latest-xl-arm64
    -+            artifact: macos-artifacts
    -+            command: git
     +          - os: windows-latest
     +            artifact: win-installer-x86_64
     +            command: $PROGRAMFILES\Git\cmd\git.exe
    @@ .github/workflows/build-git-installers.yml: jobs:
          runs-on: ubuntu-latest
          permissions:
            contents: write
    --    needs:
    --      - create-linux-artifacts
    --      - create-macos-artifacts
    --      - windows_artifacts
    -+    needs: [validate-installers]
    -     env:
    -       AZURE_VAULT: ${{ secrets.AZURE_VAULT }}
    -       GPG_PUBLIC_KEY_SECRET_NAME: ${{ secrets.GPG_PUBLIC_KEY_SECRET_NAME }}
    +     needs:
    ++      - validate-installers
    +       - create-linux-artifacts
    +       - create-macos-artifacts
    +       - windows_artifacts
  • 108: 6ceeebe = 117: b8d0691 update-microsoft-git: create barebones builtin

  • 109: 9fa2c36 = 118: b6b748a update-microsoft-git: Windows implementation

  • 110: eeb8fb8 = 119: f9de633 update-microsoft-git: use brew on macOS

  • 114: 6895430 ! 120: a111d41 git_config_set_multivar_in_file_gently(): add a lock timeout

    @@ Documentation/config/core.txt: core.WSLCompat::
     
      ## config.c ##
     @@ config.c: int git_config_set_multivar_in_file_gently(const char *config_filename,
    - 					   const char *value_pattern,
    + 					   const char *comment,
      					   unsigned flags)
      {
     +	static unsigned long timeout_ms = ULONG_MAX;
  • 111: 6f05ba9 = 121: e75356e .github: update ISSUE_TEMPLATE.md for microsoft/git

  • 115: 955384e = 122: 53002f5 scalar: set the config write-lock timeout to 150ms

  • 112: c7a8331 = 123: ab2819b .github: update PULL_REQUEST_TEMPLATE.md

  • 116: 50b32c9 = 124: 9c869b3 scalar: add docs from microsoft/scalar

  • 113: 61747ce = 125: 63936d7 Adjust README.md for microsoft/git

  • 117: 93ca048 = 126: 28554ba scalar (Windows): use forward slashes as directory separators

  • 118: ddd637c = 127: fcb838e scalar: add retry logic to run_git()

  • 119: 9904172 = 128: 437450c scalar: support the config command for backwards compatibility

  • 120: 453d4eb = 129: 5fa2bf3 scalar: implement a minimal JSON parser

  • 121: a56de78 = 130: bd0998d scalar clone: support GVFS-enabled remote repositories

  • 122: 9cc3bf9 = 131: 6cd3889 test-gvfs-protocol: also serve smart protocol

  • 123: 107a31f = 132: acaa396 gvfs-helper: add the endpoint command

  • 124: 8708a4b = 133: 5d74ee7 dir_inside_of(): handle directory separators correctly

  • 125: bc239e7 = 134: 8f9bb4c scalar: disable authentication in unattended mode

  • 126: 20105da = 135: 2fcce2f scalar: do initialize gvfs.sharedCache

  • 127: 6ec6b87 = 136: b0113a7 scalar diagnose: include shared cache info

  • 128: 80f9ccf = 137: 724da89 scalar: only try GVFS protocol on https:// URLs

  • 140: ab0c7de = 138: 123dd36 add/rm: allow adding sparse entries when virtual

  • 141: 8577cb1 = 139: ff7dfe3 sparse-checkout: add config to disable deleting dirs

  • 142: ee5b0c3 = 140: 6e4a1cb diff: ignore sparse paths in diffstat

  • 143: e4ec64e = 141: bd7fa4a repo-settings: enable sparse index by default

  • 144: 12f176c = 142: 506b545 diff(sparse-index): verify with partially-sparse

  • 145: 914a993 = 143: 1712592 stash: expand testing for git stash -u

  • 146: 5f9ba58 ! 144: dafb3e6 sequencer: avoid progress when stderr is redirected

    @@ sequencer.c: static int do_recursive_merge(struct repository *r,
     +	o.show_rename_progress = isatty(2);
      
      	head_tree = parse_tree_indirect(head);
    - 	next_tree = next ? repo_get_commit_tree(r, next) : empty_tree(r);
    + 	if (!head_tree)
  • 147: ffd7346 = 145: 82cf49c sparse: add vfs-specific precautions

  • 148: a74960e = 146: 6319faf reset: fix mixed reset when using virtual filesystem

  • 129: 894fd6f ! 147: 77bbd9c scalar: verify that we can use a GVFS-enabled repository

    @@ t/t9210-scalar.sh: test_expect_success UNZIP 'scalar diagnose' '
     +		test_path_is_missing 1/2 &&
     +		GIT_TRACE=$PWD/trace.txt git cat-file blob $second >actual &&
     +		: verify that the gvfs-helper was invoked to fetch it &&
    -+		test_i18ngrep gvfs-helper trace.txt &&
    ++		test_grep gvfs-helper trace.txt &&
     +		echo "second" >expect &&
     +		test_cmp expect actual
     +	)
  • 130: de5319c = 148: aa0ae53 scalar: add the cache-server command

  • 131: 7aaefa9 = 149: d6181ec scalar: add a test toggle to skip accessing the vsts/info endpoint

  • 132: 2e09e47 = 150: dea3ce0 scalar: adjust documentation to the microsoft/git fork

  • 133: 728bf67 = 151: d631f2c scalar: enable untracked cache unconditionally

  • 134: 7386dd1 = 152: 4650132 scalar: parse clone --no-fetch-commits-and-trees for backwards compatibility

  • 135: 03accb7 = 153: 8808cd5 scalar diagnose: accommodate Scalar's Functional Tests

  • 136: e960ac0 = 154: 7f48808 ci: run Scalar's Functional Tests

  • 137: 99b9b65 = 155: fdc75ed scalar: upgrade to newest FSMonitor config setting

  • 138: f80613f = 156: 7fffee2 abspath: make strip_last_path_component() global

  • 139: e0cbc01 = 157: 1e254a2 scalar: .scalarCache should live above enlistment

  • 156: c8fe603 = 158: ab9507e credential: add new interactive config option

  • 157: a7642ea ! 159: 74b97c7 maintenance: add custom config to background jobs

    @@ Commit message
         systemd case due to writing the service files.
     
         Signed-off-by: Derrick Stolee <[email protected]>
    +    Signed-off-by: Johannes Schindelin <[email protected]>
     
      ## builtin/gc.c ##
     @@ builtin/gc.c: static const char *get_frequency(enum schedule_priority schedule)
    @@ builtin/gc.c: static int launchctl_schedule_plist(const char *exec_path, enum sc
      		   "<string>--exec-path=%s</string>\n"
     +		   "%s" /* For extra config parameters. */
      		   "<string>for-each-repo</string>\n"
    + 		   "<string>--keep-going</string>\n"
      		   "<string>--config=maintenance.repo</string>\n"
    - 		   "<string>maintenance</string>\n"
     @@ builtin/gc.c: static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
      		   "</array>\n"
      		   "<key>StartCalendarInterval</key>\n"
    @@ builtin/gc.c: static int schtasks_schedule_task(const char *exec_path, enum sche
      	      "<Actions Context=\"Author\">\n"
      	      "<Exec>\n"
      	      "<Command>\"%s\\headless-git.exe\"</Command>\n"
    --	      "<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
    -+	      "<Arguments>--exec-path=\"%s\" %s for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
    +-	      "<Arguments>--exec-path=\"%s\" for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
    ++	      "<Arguments>--exec-path=\"%s\" %s for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
      	      "</Exec>\n"
      	      "</Actions>\n"
      	      "</Task>\n";
    @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd)
      			"# replaced in the future by a Git command.\n\n");
      
      		strbuf_addf(&line_format,
    --			    "%%d %%s * * %%s \"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%s\n",
    +-			    "%%d %%s * * %%s \"%s/git\" --exec-path=\"%s\" for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%%s\n",
     -			    exec_path, exec_path);
    -+			    "%%d %%s * * %%s \"%s/git\" --exec-path=\"%s\" %s for-each-repo --config=maintenance.repo maintenance run --schedule=%%s\n",
    ++			    "%%d %%s * * %%s \"%s/git\" --exec-path=\"%s\" %s for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%%s\n",
     +			    exec_path, exec_path, get_extra_config_parameters());
      		fprintf(cron_in, line_format.buf, minute, "1-23", "*", "hourly");
      		fprintf(cron_in, line_format.buf, minute, "0", "1-6", "daily");
    @@ builtin/gc.c: static int systemd_timer_write_service_template(const char *exec_p
      	       "\n"
      	       "[Service]\n"
      	       "Type=oneshot\n"
    --	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
    -+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" %s for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
    +-	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%%i\n"
    ++	       "ExecStart=\"%s/git\" --exec-path=\"%s\" %s for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%%i\n"
      	       "LockPersonality=yes\n"
      	       "MemoryDenyWriteExecute=yes\n"
      	       "NoNewPrivileges=yes\n"
  • 158: f1896c9 = 160: b47f839 scalar: configure maintenance during 'reconfigure'

  • 159: 3fba5cb = 161: 781cdae t5300: confirm failure of git index-pack when non-idx suffix requested

  • 160: ea013ed = 162: c3607e7 index-pack: disable rev-index if index file has non .idx suffix

  • 161: 57a9598 = 163: cf5c079 sparse-index.c: fix use of index hashes in expand_index

  • 162: 187c9b0 = 164: 34275c7 t1092: add test for untracked files and directories

  • 163: e35106f (upstream: b316552) < -: ----------- name-hash: add index_dir_find()

  • 164: 15e25b5 (upstream: 32ca706) < -: ----------- t7527: add case-insensitve test for FSMonitor

  • 165: ac5f796 (upstream: e5da3dd) < -: ----------- fsmonitor: refactor refresh callback on directory events

  • 166: 64c9b50 (upstream: 7a15a62) < -: ----------- fsmonitor: clarify handling of directory events in callback helper

  • 167: 7c8609c (upstream: 8687c2b) < -: ----------- fsmonitor: refactor refresh callback for non-directory events

  • 168: f4e4019 (upstream: 3e4ffda) < -: ----------- dir: create untracked_cache_invalidate_trimmed_path()

  • 169: 5fdd6fb (upstream: 48f4cd7) < -: ----------- fsmonitor: refactor untracked-cache invalidation

  • 170: 9628e45 (upstream: 7c97174) < -: ----------- fsmonitor: move untracked-cache invalidation into helper functions

  • 171: c3c49b4 (upstream: a524820) < -: ----------- fsmonitor: return invalidated cache-entry count on directory event

  • 172: 8e9dfb8 (upstream: 558d146) < -: ----------- fsmonitor: remove custom loop from non-directory path handler

  • 173: 405d568 (upstream: 9e34e56) < -: ----------- fsmonitor: return invalidated cache-entry count on non-directory event

  • 174: 428ed9d (upstream: 84d441f) < -: ----------- fsmonitor: trace the new invalidated cache-entry count

  • 175: 3357ac3 (upstream: b0dba50) < -: ----------- fsmonitor: refactor bit invalidation in refresh callback

  • 176: b1b11c1 (upstream: 29c139c) < -: ----------- fsmonitor: support case-insensitive events

  • 177: 4e9f006 (upstream: 0c1c3c8) < -: ----------- t0211: demonstrate missing 'def_param' events for certain commands

  • 178: cc026b2 (upstream: 520cf66) < -: ----------- trace2: avoid emitting 'def_param' set more than once

  • 179: 910c853 (upstream: 6111252) < -: ----------- trace2: emit 'def_param' set with 'cmd_name' event

  • 180: e7269ad < -: ----------- fixup! release: add installer validation

  • 181: 00bb8b3 < -: ----------- fixup! release: create draft GitHub release with packages & installers

  • 182: 94a7088 = 165: 451a563 macos-installer/Makefile: debug dump linkage of build targets

  • 183: 1e44a0f = 166: c4c91c5 workflow/build-git-installer: use OS supplied libcurl

  • 184: 3242311 (upstream: 3242311) < -: ----------- http: reset POSTFIELDSIZE when clearing curl handle

  • 185: c28ee09 (upstream: c28ee09) < -: ----------- INSTALL: bump libcurl version to 7.21.3

  • 186: 92a209b (upstream: 92a209b) < -: ----------- remote-curl: add Transfer-Encoding header only for older curl

  • 187: 6aabf8c < -: ----------- fixup! Add virtual file system settings and hook proc

  • 188: fd1d5e1 (upstream: 199f44c) < -: ----------- builtin/clone: allow remote helpers to detect repo

The big non-obvious change is this:

  • 157: a7642ea ! 159: 74b97c7 maintenance: add custom config to background jobs

    @@ Commit message
         systemd case due to writing the service files.
     
         Signed-off-by: Derrick Stolee <[email protected]>
    +    Signed-off-by: Johannes Schindelin <[email protected]>
     
      ## builtin/gc.c ##
     @@ builtin/gc.c: static const char *get_frequency(enum schedule_priority schedule)
    @@ builtin/gc.c: static int launchctl_schedule_plist(const char *exec_path, enum sc
      		   "<string>--exec-path=%s</string>\n"
     +		   "%s" /* For extra config parameters. */
      		   "<string>for-each-repo</string>\n"
    + 		   "<string>--keep-going</string>\n"
      		   "<string>--config=maintenance.repo</string>\n"
    - 		   "<string>maintenance</string>\n"
     @@ builtin/gc.c: static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
      		   "</array>\n"
      		   "<key>StartCalendarInterval</key>\n"
    @@ builtin/gc.c: static int schtasks_schedule_task(const char *exec_path, enum sche
      	      "<Actions Context=\"Author\">\n"
      	      "<Exec>\n"
      	      "<Command>\"%s\\headless-git.exe\"</Command>\n"
    --	      "<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
    -+	      "<Arguments>--exec-path=\"%s\" %s for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
    +-	      "<Arguments>--exec-path=\"%s\" for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
    ++	      "<Arguments>--exec-path=\"%s\" %s for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
      	      "</Exec>\n"
      	      "</Actions>\n"
      	      "</Task>\n";
    @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd)
      			"# replaced in the future by a Git command.\n\n");
      
      		strbuf_addf(&line_format,
    --			    "%%d %%s * * %%s \"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%s\n",
    +-			    "%%d %%s * * %%s \"%s/git\" --exec-path=\"%s\" for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%%s\n",
     -			    exec_path, exec_path);
    -+			    "%%d %%s * * %%s \"%s/git\" --exec-path=\"%s\" %s for-each-repo --config=maintenance.repo maintenance run --schedule=%%s\n",
    ++			    "%%d %%s * * %%s \"%s/git\" --exec-path=\"%s\" %s for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%%s\n",
     +			    exec_path, exec_path, get_extra_config_parameters());
      		fprintf(cron_in, line_format.buf, minute, "1-23", "*", "hourly");
      		fprintf(cron_in, line_format.buf, minute, "0", "1-6", "daily");
    @@ builtin/gc.c: static int systemd_timer_write_service_template(const char *exec_p
      	       "\n"
      	       "[Service]\n"
      	       "Type=oneshot\n"
    --	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
    -+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" %s for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
    +-	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%%i\n"
    ++	       "ExecStart=\"%s/git\" --exec-path=\"%s\" %s for-each-repo --keep-going --config=maintenance.repo maintenance run --schedule=%%i\n"
      	       "LockPersonality=yes\n"
      	       "MemoryDenyWriteExecute=yes\n"
      	       "NoNewPrivileges=yes\n"

That's because I moved the for-each-repo fix to the root of the branch thicket, and what merge conflicts were formerly resolved in 86c0809 are now folded into the actual conflicting commit.

There's also this, which is due to test_i18ngrep having retired:

  • 129: 894fd6f ! 147: 77bbd9c scalar: verify that we can use a GVFS-enabled repository

    @@ t/t9210-scalar.sh: test_expect_success UNZIP 'scalar diagnose' '
     +		test_path_is_missing 1/2 &&
     +		GIT_TRACE=$PWD/trace.txt git cat-file blob $second >actual &&
     +		: verify that the gvfs-helper was invoked to fetch it &&
    -+		test_i18ngrep gvfs-helper trace.txt &&
    ++		test_grep gvfs-helper trace.txt &&
     +		echo "second" >expect &&
     +		test_cmp expect actual
     +	)

@dscho dscho self-assigned this Apr 23, 2024
dscho and others added 29 commits April 23, 2024 13:17
In #623, it was reported that
the regularly scheduled maintenance stops if one repo in the middle of
the list was found to be missing.

This is undesirable, and points out a gap in the design of `git
for-each-repo`: We need a mode where that command does not stop on an
error, but continues to try running the specified command with the other
repositories.

Imitating the `--keep-going` option of GNU make, this commit teaches
`for-each-repo` the same trick: to continue with the operation on all
the remaining repositories in case there was a problem with one
repository, still setting the exit code to indicate an error occurred.

Helped-by: Eric Sunshine <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
In #623, it was reported that
maintenance stops on a missing repository, omitting the remaining
repositories that were scheduled for maintenance.

This is undesirable, as it should be a best effort type of operation.

It should still fail due to the missing repository, of course, but not
leave the non-missing repositories in unmaintained shapes.

Let's use `for-each-repo`'s shiny new `--keep-going` option that we just
introduced for that very purpose.

This change will be picked up when running `git maintenance start`,
which is run implicitly by `scalar reconfigure`.

Signed-off-by: Johannes Schindelin <[email protected]>
While using the reset --stdin feature on windows path added may have a
\r at the end of the path that wasn't getting removed so didn't match
the path in the index and wasn't reset.

Signed-off-by: Kevin Willford <[email protected]>
It has been a long-standing practice in Git for Windows to append
`.windows.<n>`, and in microsoft/git to append `.vfs.0.0`. Let's keep
doing that.

Signed-off-by: Johannes Schindelin <[email protected]>
Since we really want to be based on a `.vfs.*` tag, let's make sure that
there was a new-enough one, i.e. one that agrees with the first three
version numbers of the recorded default version.

This prevents e.g. v2.22.0.vfs.0.<some-huge-number>.<commit> from being
used when the current release train was not yet tagged.

It is important to get the first three numbers of the version right
because e.g. Scalar makes decisions depending on those (such as assuming
that the `git maintenance` built-in is not available, even though it
actually _is_ available).

Signed-off-by: Johannes Schindelin <[email protected]>
This header file will accumulate GVFS-specific definitions.

Signed-off-by: Kevin Willford <[email protected]>
This does not do anything yet. The next patches will add various values
for that config setting that correspond to the various features
offered/required by GVFS.

Signed-off-by: Kevin Willford <[email protected]>

gvfs: refactor loading the core.gvfs config value

This code change makes sure that the config value for core_gvfs
is always loaded before checking it.

Signed-off-by: Kevin Willford <[email protected]>
This takes a substantial amount of time, and if the user is reasonably
sure that the files' integrity is not compromised, that time can be saved.

Git no longer verifies the SHA-1 by default, anyway.

Signed-off-by: Kevin Willford <[email protected]>

Update for 2023-02-27: This feature was upstreamed as the index.skipHash
config option. This resulted in some changes to the struct and some of
the setup code. In particular, the config reading was moved to
prepare_repo_settings(), so the core.gvfs bit check was moved there,
too.

Signed-off-by: Derrick Stolee <[email protected]>
Prevent the sparse checkout to delete files that were marked with
skip-worktree bit and are not in the sparse-checkout file.

This is because everything with the skip-worktree bit turned on is being
virtualized and will be removed with the change of HEAD.

There was only one failing test when running with these changes that was
checking to make sure the worktree narrows on checkout which was
expected since we would no longer be narrowing the worktree.

Update 2022-04-05: temporarily set 'sparse.expectfilesoutsideofpatterns' in
test (until we start disabling the "remove present-despite-SKIP_WORKTREE"
behavior with 'core.virtualfilesystem' in a later commit).

Signed-off-by: Kevin Willford <[email protected]>
While performing a fetch with a virtual file system we know that there
will be missing objects and we don't want to download them just because
of the reachability of the commits.  We also don't want to download a
pack file with commits, trees, and blobs since these will be downloaded
on demand.

This flag will skip the first connectivity check and by returning zero
will skip the upload pack. It will also skip the second connectivity
check but continue to update the branches to the latest commit ids.

Signed-off-by: Kevin Willford <[email protected]>
Ensure all filters and EOL conversions are blocked when running under
GVFS so that our projected file sizes will match the actual file size
when it is hydrated on the local machine.

Signed-off-by: Ben Peart <[email protected]>
The idea is to allow blob objects to be missing from the local repository,
and to load them lazily on demand.

After discussing this idea on the mailing list, we will rename the feature
to "lazy clone" and work more on this.

Signed-off-by: Ben Peart <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Hydrate missing loose objects in check_and_freshen() when running
virtualized. Add test cases to verify read-object hook works when
running virtualized.

This hook is called in check_and_freshen() rather than
check_and_freshen_local() to make the hook work also with alternates.

Helped-by: Kevin Willford <[email protected]>
Signed-off-by: Ben Peart <[email protected]>
This is an early version of patches I am about to send upstream:
gitgitgadget#1719.

This addresses #623.
This adds hard-coded call to GVFS.hooks.exe before and after each Git
command runs.

To make sure that this is only called on repositories cloned with GVFS, we
test for the tell-tale .gvfs.

2021-10-30: Recent movement of find_hook() to hook.c required moving these
changes out of run-command.c to hook.c.

Signed-off-by: Ben Peart <[email protected]>
Suggested by Ben Peart.

Signed-off-by: Johannes Schindelin <[email protected]>
We need to respect that config setting even if we already know that we
have a repository, but have not yet read the config.

The regression test was written by Alejandro Pauly.

2021-10-30: Recent movement of find_hook() into hook.c required moving this
change from run-command.c.

Signed-off-by: Johannes Schindelin <[email protected]>
When using the sparse-checkout feature, the file might not be on disk
because the skip-worktree bit is on.

Signed-off-by: Kevin Willford <[email protected]>
When using the sparse-checkout feature git should not write to the working
directory for files with the skip-worktree bit on.  With the skip-worktree
bit on the file may or may not be in the working directory and if it is
not we don't want or need to create it by calling checkout_entry.

There are two callers of checkout_target.  Both of which check that the
file does not exist before calling checkout_target.  load_current which
make a call to lstat right before calling checkout_target and
check_preimage which will only run checkout_taret it stat_ret is less than
zero.  It sets stat_ret to zero and only if !stat->cached will it lstat
the file and set stat_ret to something other than zero.

This patch checks if skip-worktree bit is on in checkout_target and just
returns so that the entry doesn't not end up in the working directory.
This is so that apply will not create a file in the working directory,
then update the index but not keep the working directory up to date with
the changes that happened in the index.

Signed-off-by: Kevin Willford <[email protected]>
String formatting can be a performance issue when there are
hundreds of thousands of trees.

Change to stop using the strbuf_addf and just add the strings
or characters individually.

There are a limited number of modes so added a switch for the
known ones and a default case if something comes through that
are not a known one for git.

In one scenario regarding a huge worktree, this reduces the
time required for a `git checkout <branch>` from 44 seconds
to 38 seconds, i.e. it is a non-negligible performance
improvement.

Signed-off-by: Kevin Willford <[email protected]>
The following commands and options are not currently supported when working
in a GVFS repo.  Add code to detect and block these commands from executing.

1) fsck
2) gc
4) prune
5) repack
6) submodule
8) update-index --split-index
9) update-index --index-version (other than 4)
10) update-index --[no-]skip-worktree
11) worktree

Signed-off-by: Ben Peart <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
The 'git worktree' command was marked as BLOCK_ON_GVFS_REPO because it
does not interact well with the virtual filesystem of VFS for Git. When
a Scalar clone uses the GVFS protocol, it enables the
GVFS_BLOCK_COMMANDS flag, since commands like 'git gc' do not work well
with the GVFS protocol.

However, 'git worktree' works just fine with the GVFS protocol since it
isn't doing anything special. It copies the sparse-checkout from the
current worktree, so it does not have performance issues.

This is a highly requested option.

The solution is to stop using the BLOCK_ON_GVFS_REPO option and instead
add a special-case check in cmd_worktree() specifically for a particular
bit of the 'core_gvfs' global variable (loaded by very early config
reading) that corresponds to the virtual filesystem. The bit that most
closely resembled this behavior was non-obviously named, but does
provide a signal that we are in a Scalar clone and not a VFS for Git
clone. The error message is copied from git.c, so it will have the same
output as before if a user runs this in a VFS for Git clone.

Signed-off-by: Derrick Stolee <[email protected]>
We found a user who had set "core.gvfs = false" in their global
config. This should not have been necessary, but it also should not
have caused a problem. However, it did.

The reason is that gvfs_load_config_value() is called from config.c
when reading config key/value pairs from all the config files. The
local config should override the global config, and this is done by
config.c reading the global config first then reading the local
config. However, our logic only allowed writing the core_gvfs
variable once.

Put the guards against multiple assignments of core_gvfs into
gvfs_config_is_set() instead, because that will fix the problem
_and_ keep multiple calls to gvfs_config_is_set() from slowing down.

Signed-off-by: Derrick Stolee <[email protected]>
If we are going to write an object there is no use in calling
the read object hook to get an object from a potentially remote
source.  We would rather just write out the object and avoid the
potential round trip for an object that doesn't exist.

This change adds a flag to the check_and_freshen() and
freshen_loose_object() functions' signatures so that the hook
is bypassed when the functions are called before writing loose
objects. The check for a local object is still performed so we
don't overwrite something that has already been written to one
of the objects directories.

Based on a patch by Kevin Willford.

Signed-off-by: Johannes Schindelin <[email protected]>
dscho and others added 26 commits April 23, 2024 13:35
Scalar in Microsoft's Git fork can do a little more than Scalar in
upstream Git: in Microsoft's Git, it supports the GVFS protocol so that
Scalar can clone from Azure DevOps.

Signed-off-by: Johannes Schindelin <[email protected]>
Those tests specifically verify that the `.zip` file path is shown on
`stdout`. Let's do that again, under the assumption that there are
scripts out there that rely on this behavior.

Signed-off-by: Johannes Schindelin <[email protected]>
…atibility

This option does not do anything anymore, though.

Signed-off-by: Johannes Schindelin <[email protected]>
Scalar's Functional Test suite is pretty comprehensive, and caught more
than just one bug in the built-in FSMonitor that was missed by Git's own
test suite.

To benefit from this test suite, automatically run it on the `vfs-*`
and `features/*` branches.

Note: for simplicity, we're building Git from scratch in all matrix
jobs.

Also note: for speed, we are using `git-sdk-64-minimal`, even if it
lacks the `/bin/install` that we need to install Git's files; We're
providing a minimal shell script shim instead. Also, we do not need to
bother with the Tcl/Tk parts, therefore we're skipping them, too.

Finally, we use GIT_FORCE_UNTRACKED_CACHE in the functional tests, to
give the untracked cache a thorough work-out.

Signed-off-by: Johannes Schindelin <[email protected]>
Prepare `scalar` to use the GVFS protocol instead of partial clone
(required to support Azure Repos).

Signed-off-by: Johannes Schindelin <[email protected]>
The strip_last_component() method is helpful for finding the parent
directory of a path stored in a strbuf. Extract it to a global method
advertised in abspath.h. With that additional visibility, it is helpful to
rename it to be more specific to paths.

Signed-off-by: Derrick Stolee <[email protected]>
When FSMonitor was upstreamed, the 'core.useBuiltinFSMonitor' config was
deprecated and replaced with an overload of the 'core.fsmonitor' config
(i.e., if a boolean value was specified in 'core.fsmonitor', it is treated
the way 'core.useBuiltinFSMonitor' originally was). Because 'scalar
register' actively sets that config, use it to upgrade the deprecated config
setting.

Co-authored-by: Johannes Schindelin <[email protected]>
Signed-off-by: Victoria Dye <[email protected]>
Run the comprehensive Scalar Functional Tests as part of CI and PR
builds.

Signed-off-by: Johannes Schindelin <[email protected]>
We should not be putting the .scalarCache inside the enlistment as a
sibling to the 'src' directory. This only happens in "unattended" mode,
but it also negates any benefit of a shared object cache because each
enlistment absolutely does not share any objects with others.

Move the shared object cache in this case to a level above the
enlistment, so at least there is some hope that it can be reused. This
is also critical to the upcoming --no-src option, since the shared
object cache cannot be located within the Git repository.

Signed-off-by: Derrick Stolee <[email protected]>
This allows fixing settings after a Scalar upgrade, or after botching
the enlistments configuration.

Signed-off-by: Johannes Schindelin <[email protected]>
…e after clones

These are two highly-requested items from an internal team considering a
move to Scalar using Azure Repos.

1. Remove the requirement that we create a `src` directory at clone time.

2. Allow `git worktree` even when using the GVFS protocol.

These are not difficult to implement. The `--no-src` option could even
be submitted upstream (though the commit will need to drop one bit about
an interaction with the local cache path).
…x-built-in-fsmonitor

Fix the built-in FSMonitor, and run Scalar's Functional Tests as part of the automated builds
This is random stuff that probably all got upstream in the meantime.
When scripts or background maintenance wish to perform HTTP(S) requests,
there is a risk that our stored credentials might be invalid. At the
moment, this causes the credential helper to ping the user and block the
process. Even if the credential helper does not ping the user, Git falls
back to the 'askpass' method, which includes a direct ping to the user
via the terminal.

Even setting the 'core.askPass' config as something like 'echo' will
causes Git to fallback to a terminal prompt. It uses
git_terminal_prompt(), which finds the terminal from the environment and
ignores whether stdin has been redirected. This can also block the
process awaiting input.

Create a new config option to prevent user interaction, favoring a
failure to a blocked process.

The chosen name, 'credential.interactive', is taken from the config
option used by Git Credential Manager to already avoid user
interactivity, so there is already one credential helper that integrates
with this option. However, older versions of Git Credential Manager also
accepted other string values, including 'auto', 'never', and 'always'.
The modern use is to use a boolean value, but we should still be
careful that some users could have these non-booleans. Further, we
should respect 'never' the same as 'false'. This is respected by the
implementation and test, but not mentioned in the documentation.

The implementation for the Git interactions takes place within
credential_getpass(). The method prototype is modified to return an
'int' instead of 'void'. This allows us to detect that no attempt was
made to fill the given credential, changing the single caller slightly.

Also, a new trace2 region is added around the interactive portion of the
credential request. This provides a way to measure the amount of time
spent in that region for commands that _are_ interactive. It also makes
a conventient way to test that the config option works with
'test_region'.

Signed-off-by: Derrick Stolee <[email protected]>
Add test case to demonstrate that `git index-pack -o <idx-path> pack-path`
fails if <idx-path> does not end in ".idx" when `--rev-index` is
enabled.

In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25)
we learned to create `.rev` reverse indexes in addition to `.idx` index
files.  The `.rev` file pathname is constructed by replacing the suffix
on the `.idx` file.  The code assumes a hard-coded "idx" suffix.

In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12)
reverse indexes were enabled by default.

If the `-o <idx-path>` argument is used, the index file may have a
different suffix.  This causes an error when it tries to create the
reverse index pathname.

The test here demonstrates the failure.  (The test forces `--rev-index`
to avoid interaction with `GIT_TEST_NO_WRITE_REV_INDEX` during CI runs.)

Signed-off-by: Jeff Hostetler <[email protected]>
At the moment, some background jobs are getting blocked on credentials
during the 'prefetch' task. This leads to other tasks, such as
incremental repacks, getting blocked. Further, if a user manages to fix
their credentials, then they still need to cancel the background process
before their background maintenance can continue working.

Update the background schedules for our four scheduler integrations to
include these config options via '-c' options:

 * 'credential.interactive=false' will stop Git and some credential
   helpers from prompting in the UI (assuming the '-c' parameters are
   carried through and respected by GCM).

 * 'core.askPass=true' will replace the text fallback for a username
   and password into the 'true' command, which will return a success in
   its exit code, but Git will treat the empty string returned as an
   invalid password and move on.

We can do some testing that the credentials are passed, at least in the
systemd case due to writing the service files.

Signed-off-by: Derrick Stolee <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
In ac8acb4 (sparse-index: complete partial expansion, 2022-05-23),
'expand_index()' was updated to expand the index to a given pathspec.
However, the 'path_matches_pattern_list()' method used to facilitate this
has the side effect of initializing or updating the index hash variables
('name_hash', 'dir_hash', and 'name_hash_initialized'). This operation is
performed on 'istate', though, not 'full'; as a result, the initialized
hashes are later overwritten when copied from 'full'. To ensure the correct
hashes are in 'istate' after the index expansion, change the arg used in
'path_matches_pattern_list()' from 'istate' to 'full'.

Note that this does not fully solve the problem. If 'istate' does not have
an initialized 'name_hash' when its contents are copied to 'full',
initialized hashes will be copied back into 'istate' but
'name_hash_initialized' will be 0. Therefore, we also need to copy
'full->name_hash_initialized' back to 'istate' after the index expansion is
complete.

Signed-off-by: Victoria Dye <[email protected]>
Teach index-pack to silently omit the reverse index if the
index file does not have the standard ".idx" suffix.

In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25)
we learned to create `.rev` reverse indexes in addition to `.idx` index
files.  The `.rev` file pathname is constructed by replacing the suffix
on the `.idx` file.  The code assumes a hard-coded "idx" suffix.

In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12)
reverse indexes were enabled by default.

If the `-o <idx-path>` argument is used, the index file may have a
different suffix.  This causes an error when it tries to create the
reverse index pathname.

Since we do not know why the user requested a non-standard suffix for
the index, we cannot guess what the proper corresponding suffix should
be for the reverse index.  So we disable it.

The t5300 test has been updated to verify that we no longer error
out and that the .rev file is not created.

TODO We could warn the user that we skipped it (perhaps only if they
TODO explicitly requested `--rev-index` on the command line).
TODO
TODO Ideally, we should add an `--rev-index-path=<path>` argument
TODO or change `--rev-index` to take a pathname.
TODO
TODO I'll leave these questions for a future series.

Signed-off-by: Jeff Hostetler <[email protected]>
The 'scalar reconfigure' command is intended to update registered repos
with the latest settings available. However, up to now we were not
reregistering the repos with background maintenance.

In particular, this meant that the background maintenance schedule would
not be updated if there are improvements between versions.

Be sure to register repos for maintenance during the reconfigure step.

Signed-off-by: Derrick Stolee <[email protected]>
Run `otool -L` on each of the major executables and print the name of
the CPU architecture(s) and the name and version of the referenced
dynamic libraries.

In a later commit, we might want to verify that certain minimim
versions were linked or that there are no unexpected third-party
libraries in the linkage (such as an unexpected homebrew).

Signed-off-by: Jeff Hostetler <[email protected]>
Add a test verifying that sparse-checkout (with and without sparse index
enabled) treat untracked files & directories correctly when changing sparse
patterns. Specifically, it ensures that 'git sparse-checkout set'

* deletes empty directories outside the sparse cone
* does _not_ delete untracked files outside the sparse cone

Signed-off-by: Victoria Dye <[email protected]>
Cherry-pick rev-index fixes from v2.41.0.vfs.0.5 into v2.42.0.*
Stop building custom universal libcurl dylib and let Git binaries
use the version installed on the OS, like we do for other libraries.

Signed-off-by: Jeff Hostetler <[email protected]>
Use the OS provided version of `libcurl` when building release packages.
This is in contrast to using Homebrew on ARM64 in our universal
binaries.

Also add diagnostics during the release build process to display the
DYLIBs
that are referenced by the key executables.
@dscho dscho force-pushed the tentative/vfs-2.45.0 branch from d76bb7e to 4b1e399 Compare April 23, 2024 11:37
@rimrul
Copy link
Member

rimrul commented Apr 23, 2024

Wrong repo? This seems like a PR for microsoft/git.

@dscho
Copy link
Member Author

dscho commented Apr 23, 2024

Wrong repo? This seems like a PR for microsoft/git.

Oops.

@dscho dscho closed this Apr 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.