Skip to content

Rebase to v2.45.1 #652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 206 commits into from
May 31, 2024
Merged

Rebase to v2.45.1 #652

merged 206 commits into from
May 31, 2024

Conversation

dscho
Copy link
Member

@dscho dscho commented May 14, 2024

Range-diff relative to clean/vfs-2.45.0
  • 146: 9d4453e = 1: 9d4453e ci: drop mention of BREW_INSTALL_PACKAGES variable

  • 147: 11c7001 = 2: 11c7001 ci: avoid bare "gcc" for osx-gcc job

  • 148: 7df2405 = 3: 7df2405 ci: stop installing "gcc-13" for osx-gcc

  • 1: c5eb22f = 4: 22c9468 for-each-repo: optionally keep going on an error

  • 2: 39cf372 = 5: 2e13c37 maintenance: running maintenance should not stop on errors

  • 3: 8fc81fb = 6: 13fd78b reset --stdin: trim carriage return from the paths

  • 4: 11a339e ! 7: 257787b Identify microsoft/git via a distinct version suffix

    @@ Commit message
      ## GIT-VERSION-GEN ##
     @@
      GVF=GIT-VERSION-FILE
    - DEF_VER=v2.45.0
    + DEF_VER=v2.45.1
      
     +# Identify microsoft/git via a distinct version suffix
     +DEF_VER=$DEF_VER.vfs.0.0
  • 5: 2d5d918 = 8: 6c6a882 gvfs: ensure that the version is based on a GVFS tag

  • 6: 00d0db0 = 9: 8642241 gvfs: add a GVFS-specific header file

  • 7: 1630ca4 = 10: 4d672df gvfs: add the core.gvfs config setting

  • 8: add4a59 = 11: a4b9a71 gvfs: add the feature to skip writing the index' SHA-1

  • 9: 912f104 = 12: b9f275b gvfs: add the feature that blobs may be missing

  • 10: cc76156 = 13: c4f195b gvfs: prevent files to be deleted outside the sparse checkout

  • 11: 0b42dcd = 14: 526552c gvfs: optionally skip reachability checks/upload pack during fetch

  • 12: 05f8c12 = 15: 1c196d7 gvfs: ensure all filters and EOL conversions are blocked

  • 13: fa0e159 = 16: 22b726b gvfs: allow "virtualizing" objects

  • 109: bad0bc3 = 17: af7f27f git_config_set_multivar_in_file_gently(): add a lock timeout

  • 111: 4b9cb8a = 18: 9bfd2ed scalar: set the config write-lock timeout to 150ms

  • 113: dfd1877 = 19: de1765e scalar: add docs from microsoft/scalar

  • 114: e4139c5 = 20: e64cc9c scalar (Windows): use forward slashes as directory separators

  • 115: 32eb925 = 21: ad5658f scalar: add retry logic to run_git()

  • 116: c65f878 = 22: 9fdf70d scalar: support the config command for backwards compatibility

  • 143: 1b65e5f = 23: b944f0a sequencer: avoid progress when stderr is redirected

  • 14: d6ee336 ! 24: 2e30eb6 Hydrate missing loose objects in check_and_freshen()

    @@ t/t0410/read-object (new)
     +	}
     +}
     
    - ## t/t0411-read-object.sh (new) ##
    + ## t/t0499-read-object.sh (new) ##
     @@
     +#!/bin/sh
     +
  • 15: 156a40f ! 25: 24bcae8 sha1_file: when writing objects, skip the read_object_hook

    @@ t/t0410/read-object: while (1) {
      		die "bad command '$command'";
      	}
     
    - ## t/t0411-read-object.sh ##
    -@@ t/t0411-read-object.sh: test_expect_success 'invalid blobs generate errors' '
    + ## t/t0499-read-object.sh ##
    +@@ t/t0499-read-object.sh: test_expect_success 'invalid blobs generate errors' '
      	 test_must_fail git cat-file blob "invalid")
      '
      
  • 16: 7402811 ! 26: 51870be gvfs: add global command pre and post hook procs

    @@ hook.c
      #include "advice.h"
      #include "gettext.h"
      #include "hook.h"
    -@@
    - #include "run-command.h"
    - #include "config.h"
    - #include "strbuf.h"
    -+#include "setup.h"
    -+
    +@@ hook.c: static int identical_to_template_hook(const char *name, const char *path)
    + 	return ret;
    + }
    + 
     +static int early_hooks_path_config(const char *var, const char *value,
     +				   const struct config_context *ctx, void *cb)
     +{
    @@ hook.c
     +	strbuf_addf(result, "%s%s", hooks_dir.buf, name);
     +	return result->buf;
     +}
    - 
    ++
      const char *find_hook(const char *name)
      {
      	static struct strbuf path = STRBUF_INIT;
    +@@ hook.c: const char *find_hook(const char *name)
    + 	int found_hook;
      
      	strbuf_reset(&path);
     -	strbuf_git_path(&path, "hooks/%s", name);
    @@ hook.c
     +	else if (!hook_path_early(name, &path))
     +		return NULL;
     +
    - 	if (access(path.buf, X_OK) < 0) {
    - 		int err = errno;
    - 
    + 	found_hook = access(path.buf, X_OK) >= 0;
    + #ifdef STRIP_EXTENSION
    + 	if (!found_hook) {
     
      ## t/t0400-pre-command-hook.sh (new) ##
     @@
  • 17: e7bf03f = 27: e6ff17f t0400: verify that the hook is called correctly from a subdirectory

  • 18: 7f7b1ae = 28: 42b51e2 Pass PID of git process to hooks.

  • 19: daba665 ! 29: 31ed9bc pre-command: always respect core.hooksPath

    @@ Commit message
     
      ## hook.c ##
     @@ hook.c: const char *find_hook(const char *name)
    - 	static struct strbuf path = STRBUF_INIT;
    + 	int found_hook;
      
      	strbuf_reset(&path);
     -	if (have_git_dir())
    @@ hook.c: const char *find_hook(const char *name)
     +	} else if (!hook_path_early(name, &path))
      		return NULL;
      
    - 	if (access(path.buf, X_OK) < 0) {
    + 	found_hook = access(path.buf, X_OK) >= 0;
     
      ## t/t0400-pre-command-hook.sh ##
     @@ t/t0400-pre-command-hook.sh: test_expect_success 'in a subdirectory, using an alias' '
  • 20: ea46c9b = 30: 56a24e4 sparse-checkout: update files with a modify/delete conflict

  • 21: 9bb3d30 = 31: 019a683 sparse-checkout: avoid writing entries with the skip-worktree bit

  • 22: 64b8e55 = 32: 4f7cfe3 Do not remove files outside the sparse-checkout

  • 23: df0ee3a = 33: 6fc4113 send-pack: do not check for sha1 file when GVFS_MISSING_OK set

  • 24: a2aad6e = 34: 118a527 cache-tree: remove use of strbuf_addf in update_one

  • 25: 379037d = 35: 1777a92 gvfs: block unsupported commands when running in a GVFS repo

  • 26: f5f8667 = 36: db31b45 worktree: allow in Scalar repositories

  • 27: 559d22d = 37: d38541d gvfs: allow overriding core.gvfs

  • 28: 915dd0e = 38: 180746b BRANCHES.md: Add explanation of branches and using forks

  • 29: fb3896b = 39: d29907e Add virtual file system settings and hook proc

  • 30: 3a23415 = 40: 06228cd virtualfilesystem: don't run the virtual file system hook if the index has been redirected

  • 31: fd88f1d = 41: 3445aa0 virtualfilesystem: check if directory is included

  • 32: a235f3e = 42: b56d9d4 backwards-compatibility: support the post-indexchanged hook

  • 33: 2d54645 = 43: ec31fd5 gvfs: verify that the built-in FSMonitor is disabled

  • 34: 463c44a = 44: 91546cb status: add status serialization mechanism

  • 35: 053e352 = 45: 5883f07 Teach ahead-behind and serialized status to play nicely together

  • 36: 4b646b4 = 46: e6ee448 status: serialize to path

  • 37: 58c9b0b = 47: 8ccf9fb status: reject deserialize in V2 and conflicts

  • 38: b46bedb = 48: 82efde9 serialize-status: serialize global and repo-local exclude file metadata

  • 39: d917783 = 49: 4830a2e status: deserialization wait

  • 40: 801300d = 50: a9c962c merge-recursive: avoid confusing logic in was_dirty()

  • 41: 5a58776 = 51: 9802bfd merge-recursive: add some defensive coding to was_dirty()

  • 42: 7a43c9a = 52: 31f1d9c merge-recursive: teach was_dirty() about the virtualfilesystem

  • 43: 006fb39 = 53: 705e175 status: deserialize with -uno does not print correct hint

  • 44: e383198 = 54: 45e4ed8 fsmonitor: check CE_FSMONITOR_VALID in ce_uptodate

  • 45: 7002ec2 = 55: daed3ee fsmonitor: add script for debugging and update script for tests

  • 46: ab232b3 = 56: be8e145 status: disable deserialize when verbose output requested.

  • 47: dd16e4f = 57: 16993e8 t7524: add test for verbose status deserialzation

  • 48: c51d610 = 58: 297161e deserialize-status: silently fallback if we cannot read cache file

  • 49: d1b84be = 59: af290cd gvfs:trace2:data: add trace2 tracing around read_object_process

  • 50: 323ae62 = 60: a64d037 gvfs:trace2:data: status deserialization information

  • 51: 0983325 = 61: d9948c9 gvfs:trace2:data: status serialization

  • 52: 8ff9081 = 62: 3d9f4cd gvfs:trace2:data: add vfs stats

  • 53: 316985c = 63: 5fefbad trace2: refactor setting process starting time

  • 54: ffa6a7a = 64: 207a15e trace2:gvfs:experiment: clear_ce_flags_1

  • 55: 6ed0189 = 65: 83055a8 trace2:gvfs:experiment: report_tracking

  • 56: 61f3777 = 66: f42f31e trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache

  • 57: 7e259e1 = 67: fe738aa trace2:gvfs:experiment: read-cache: time read/write of cache-tree extension

  • 58: 485d47e = 68: 3a8eaf0 trace2:gvfs:experiment: add region to apply_virtualfilesystem()

  • 59: 63eadcf = 69: 960cd8a trace2:gvfs:experiment: add region around unpack_trees()

  • 60: 3f6652a = 70: 5b2520f trace2:gvfs:experiment: add region to cache_tree_fully_valid()

  • 61: cb0d9f6 = 71: 23f9c10 trace2:gvfs:experiment: add unpack_entry() counter to unpack_trees() and report_tracking()

  • 62: 3431711 = 72: 3a70cd1 trace2:gvfs:experiment: increase default event depth for unpack-tree data

  • 63: 62d510e = 73: 27a974b trace2:gvfs:experiment: add data for check_updates() in unpack_trees()

  • 64: 244c418 = 74: c494bda Trace2:gvfs:experiment: capture more 'tracking' details

  • 65: 5223a1f = 75: 2924e38 credential: set trace2_child_class for credential manager children

  • 66: fe1cf67 = 76: bbff5ff sub-process: do not borrow cmd pointer from caller

  • 67: f277df4 = 77: 344654c sub-process: add subprocess_start_argv()

  • 68: db65d92 = 78: 216562b sha1-file: add function to update existing loose object cache

  • 69: 2e22b51 = 79: c0bdffc packfile: add install_packed_git_and_mru()

  • 70: 582dec7 = 80: f7068d0 index-pack: avoid immediate object fetch while parsing packfile

  • 71: 9f20de3 = 81: e6f6455 gvfs-helper: create tool to fetch objects using the GVFS Protocol

  • 72: 229b5be = 82: 336904a sha1-file: create shared-cache directory if it doesn't exist

  • 73: 7b059df = 83: ff9c896 gvfs-helper: better handling of network errors

  • 74: 091bf68 = 84: 799562e gvfs-helper-client: properly update loose cache with fetched OID

  • 75: aaa2c89 = 85: b53a311 gvfs-helper: V2 robust retry and throttling

  • 76: c869fa7 = 86: da85141 gvfs-helper: expose gvfs/objects GET and POST semantics

  • 77: 2da2d65 = 87: 0b64a68 gvfs-helper: dramatically reduce progress noise

  • 78: d83f757 = 88: 57dea35 gvfs-helper-client.h: define struct object_id

  • 79: e49eee5 = 89: ea95531 gvfs-helper: handle pack-file after single POST request

  • 80: c128fc0 = 90: b3ec480 test-gvfs-prococol, t5799: tests for gvfs-helper

  • 81: 9a9fe2b = 91: 7c82e54 gvfs-helper: move result-list construction into install functions

  • 82: a5ce642 = 92: ad217d4 t5799: add support for POST to return either a loose object or packfile

  • 83: 51d2920 = 93: e4cba83 t5799: cleanup wc-l and grep-c lines

  • 84: f11ed18 = 94: 1ce899d gvfs-helper: verify loose objects after write

  • 85: c484229 = 95: 41ad104 t7599: create corrupt blob test

  • 86: f5af304 = 96: 6243647 gvfs-helper: add prefetch support

  • 87: af3935a = 97: 651d8cb gvfs-helper: add prefetch .keep file for last packfile

  • 88: 45c6634 = 98: ff6c545 gvfs-helper: do one read in my_copy_fd_len_tail()

  • 89: a3ee045 = 99: c6515af gvfs-helper: move content-type warning for prefetch packs

  • 90: 00d4feb = 100: 5305659 fetch: use gvfs-helper prefetch under config

  • 91: 04ee5a1 = 101: 3a933f4 gvfs-helper: better support for concurrent packfile fetches

  • 92: 733f9ad = 102: 968175d remote-curl: do not call fetch-pack when using gvfs-helper

  • 93: f980c67 = 103: d314f2d fetch: reprepare packs before checking connectivity

  • 94: dbc55cc = 104: 545e939 gvfs-helper: retry when creating temp files

  • 95: a788951 = 105: ee2b8b4 sparse: avoid warnings about known cURL issues in gvfs-helper.c

  • 97: 87e7e61 = 106: 5e1af6c gvfs-helper: add --max-retries to prefetch verb

  • 99: 77475d4 = 107: d6d818b t5799: add tests to detect corrupt pack/idx files in prefetch

  • 96: 4cb8f09 = 108: bf9234b maintenance: care about gvfs.sharedCache config

  • 98: eac30a5 = 109: 820079f unpack-trees:virtualfilesystem: Improve efficiency of clear_ce_flags

  • 100: 528a8b1 = 110: 3492d82 homebrew: add GitHub workflow to release Cask

  • 102: c620a9f = 111: 3dfe79a Adding winget workflows

  • 103: ba54b36 = 112: f3fecaa Disable the monitor-components workflow in msft-git

  • 104: e59be1f = 113: e0bd85c .github: enable windows builds on microsoft fork

  • 149: e18dc8b = 114: 25f21e3 release: create initial Windows installer build workflow

  • 150: 60c99e9 = 115: 48dc640 help: special-case HOST_CPU universal

  • 151: 91fe926 = 116: af15e3b release: add Mac OSX installer build

  • 101: d480477 = 117: 5a08847 gvfs-helper: ignore .idx files in prefetch multi-part responses

  • 152: 22da5f2 = 118: f2a141d release: build unsigned Ubuntu .deb package

  • 153: 728f371 = 119: cb824c4 release: add signing step for .deb package

  • 154: 22d0532 = 120: 4cab821 release: create draft GitHub release with packages & installers

  • 155: 88e7185 = 121: 91c9159 build-git-installers: publish gpg public key

  • 156: a2ee54e = 122: 9c5d54a release: continue pestering until user upgrades

  • 105: 586cda0 = 123: b347098 update-microsoft-git: create barebones builtin

  • 157: 20d7ab4 = 124: f8b462c Makefile: allow specifying GIT_BUILT_FROM_COMMIT

  • 106: 3a3c47b = 125: f3ef0b8 update-microsoft-git: Windows implementation

  • 158: 31aef20 = 126: 8bfcd80 dist: archive HEAD instead of HEAD^{tree}

  • 107: 08d7d2f = 127: 810b7d9 update-microsoft-git: use brew on macOS

  • 159: 210db51 = 128: 368e4c1 release: include GIT_BUILT_FROM_COMMIT in MacOS build

  • 108: 12ac17a = 129: a05be3e .github: update ISSUE_TEMPLATE.md for microsoft/git

  • 160: afcc1ed = 130: 3e282e7 release: add installer validation

  • 110: 3b43886 = 131: a7cbc7c .github: update PULL_REQUEST_TEMPLATE.md

  • 112: a5b0281 = 132: 0ab5831 Adjust README.md for microsoft/git

  • 117: 306a13a = 133: fb70d6b scalar: implement a minimal JSON parser

  • 118: 321a4a5 = 134: a2b0d56 scalar clone: support GVFS-enabled remote repositories

  • 119: 11c9190 = 135: ce509f6 test-gvfs-protocol: also serve smart protocol

  • 120: 8538f2c = 136: 2d3cff3 gvfs-helper: add the endpoint command

  • 121: 8f7412b = 137: 0be0f3f dir_inside_of(): handle directory separators correctly

  • 122: 418b97a = 138: 4543f54 scalar: disable authentication in unattended mode

  • 123: bdb620d = 139: bb54b06 scalar: do initialize gvfs.sharedCache

  • 124: 7685c29 = 140: af5bbd9 scalar diagnose: include shared cache info

  • 125: a4fc1df = 141: 67ed368 scalar: only try GVFS protocol on https:// URLs

  • 126: fd51fa5 = 142: 5192e70 scalar: verify that we can use a GVFS-enabled repository

  • 127: 9f18d97 = 143: af7bf17 scalar: add the cache-server command

  • 128: f6efa1c = 144: 081e43f scalar: add a test toggle to skip accessing the vsts/info endpoint

  • 129: 1df44f6 = 145: e3cafb9 scalar: adjust documentation to the microsoft/git fork

  • 130: ce372e8 = 146: a40a2b6 scalar: enable untracked cache unconditionally

  • 131: e8374f9 = 147: a4725c2 scalar: parse clone --no-fetch-commits-and-trees for backwards compatibility

  • 132: 267f68c = 148: 55efd42 scalar diagnose: accommodate Scalar's Functional Tests

  • 133: 9eb2a07 = 149: 7baf5a4 ci: run Scalar's Functional Tests

  • 134: 5854c30 = 150: 015e51f scalar: upgrade to newest FSMonitor config setting

  • 135: 3940c78 = 151: 5bf7774 abspath: make strip_last_path_component() global

  • 136: f505c40 = 152: d2fb8ac scalar: .scalarCache should live above enlistment

  • 137: 052c63e = 153: 42149f3 add/rm: allow adding sparse entries when virtual

  • 138: 518476a = 154: 837faa6 sparse-checkout: add config to disable deleting dirs

  • 139: c168c25 = 155: fc4ffa9 diff: ignore sparse paths in diffstat

  • 140: c2ba772 = 156: fdc1eaa repo-settings: enable sparse index by default

  • 141: c33429b = 157: b20e275 diff(sparse-index): verify with partially-sparse

  • 142: 8a7808a = 158: efb2ce1 stash: expand testing for git stash -u

  • 144: 49adf44 = 159: 3a70279 sparse: add vfs-specific precautions

  • 145: 72afab7 = 160: 7e0d519 reset: fix mixed reset when using virtual filesystem

  • 161: a4c57c1 = 161: f8f589a credential: add new interactive config option

  • 162: 691377e = 162: 237dbe7 maintenance: add custom config to background jobs

  • 163: 6e22ced = 163: 6f73d25 scalar: configure maintenance during 'reconfigure'

  • 164: c784114 = 164: 674791e scalar: avoid segfault in reconfigure --all

  • 165: 6d67155 = 165: 122960d scalar: make GVFS Protocol a forced choice

  • 166: 4a13a74 = 166: cef4221 t5300: confirm failure of git index-pack when non-idx suffix requested

  • 167: af3dcf2 = 167: 4fc49d3 index-pack: disable rev-index if index file has non .idx suffix

  • 168: 50a96c9 = 168: 7851229 sparse-index.c: fix use of index hashes in expand_index

  • 169: 324e77b = 169: f1a4822 t1092: add test for untracked files and directories

peff and others added 30 commits May 9, 2024 09:57
The last user of this variable went away in 4a6e4b9 (CI: remove
Travis CI support, 2021-11-23), so it's doing nothing except making it
more confusing to find out which packages _are_ installed.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On macOS, a bare "gcc" (without a version) will invoke a wrapper for
clang, not actual gcc. Even when gcc is installed via homebrew, that
only provides version-specific links in /usr/local/bin (like "gcc-13"),
and never a version-agnostic "gcc" wrapper.

As far as I can tell, this has been the case for a long time, and this
osx-gcc job has largely been doing nothing. We can point it at "gcc-13",
which will pick up the homebrew-installed version.

The fix here is specific to the github workflow file, as the gitlab one
does not have a matching job.

It's a little unfortunate that we cannot just ask for the latest version
of gcc which homebrew provides, but as far as I can tell there is no
easy alias (you'd have to find the highest number gcc-* in
/usr/local/bin yourself).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Our osx-gcc job explicitly asks to install gcc-13. But since the GitHub
runner image already comes with gcc-13 installed, this is mostly doing
nothing (or in some cases it may install an incremental update over the
runner image). But worse, it recently started causing errors like:

    ==> Fetching gcc@13
    ==> Downloading https://ghcr.io/v2/homebrew/core/gcc/13/blobs/sha256:fb2403d97e2ce67eb441b54557cfb61980830f3ba26d4c5a1fe5ecd0c9730d1a
    ==> Pouring gcc@13--13.2.0.ventura.bottle.tar.gz
    Error: The `brew link` step did not complete successfully
    The formula built, but is not symlinked into /usr/local
    Could not symlink bin/c++-13
    Target /usr/local/bin/c++-13
    is a symlink belonging to gcc. You can unlink it:
      brew unlink gcc

which cause the whole CI job to bail.

I didn't track down the root cause, but I suspect it may be related to
homebrew recently switching the "gcc" default to gcc-14. And it may even
be fixed when a new runner image is released. But if we don't need to
run brew at all, it's one less thing for us to worry about.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In #623, it was reported that
the regularly scheduled maintenance stops if one repo in the middle of
the list was found to be missing.

This is undesirable, and points out a gap in the design of `git
for-each-repo`: We need a mode where that command does not stop on an
error, but continues to try running the specified command with the other
repositories.

Imitating the `--keep-going` option of GNU make, this commit teaches
`for-each-repo` the same trick: to continue with the operation on all
the remaining repositories in case there was a problem with one
repository, still setting the exit code to indicate an error occurred.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In #623, it was reported that
maintenance stops on a missing repository, omitting the remaining
repositories that were scheduled for maintenance.

This is undesirable, as it should be a best effort type of operation.

It should still fail due to the missing repository, of course, but not
leave the non-missing repositories in unmaintained shapes.

Let's use `for-each-repo`'s shiny new `--keep-going` option that we just
introduced for that very purpose.

This change will be picked up when running `git maintenance start`,
which is run implicitly by `scalar reconfigure`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
While using the reset --stdin feature on windows path added may have a
\r at the end of the path that wasn't getting removed so didn't match
the path in the index and wasn't reset.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
It has been a long-standing practice in Git for Windows to append
`.windows.<n>`, and in microsoft/git to append `.vfs.0.0`. Let's keep
doing that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Since we really want to be based on a `.vfs.*` tag, let's make sure that
there was a new-enough one, i.e. one that agrees with the first three
version numbers of the recorded default version.

This prevents e.g. v2.22.0.vfs.0.<some-huge-number>.<commit> from being
used when the current release train was not yet tagged.

It is important to get the first three numbers of the version right
because e.g. Scalar makes decisions depending on those (such as assuming
that the `git maintenance` built-in is not available, even though it
actually _is_ available).

Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
This header file will accumulate GVFS-specific definitions.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
This does not do anything yet. The next patches will add various values
for that config setting that correspond to the various features
offered/required by GVFS.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>

gvfs: refactor loading the core.gvfs config value

This code change makes sure that the config value for core_gvfs
is always loaded before checking it.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
This takes a substantial amount of time, and if the user is reasonably
sure that the files' integrity is not compromised, that time can be saved.

Git no longer verifies the SHA-1 by default, anyway.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>

Update for 2023-02-27: This feature was upstreamed as the index.skipHash
config option. This resulted in some changes to the struct and some of
the setup code. In particular, the config reading was moved to
prepare_repo_settings(), so the core.gvfs bit check was moved there,
too.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Prevent the sparse checkout to delete files that were marked with
skip-worktree bit and are not in the sparse-checkout file.

This is because everything with the skip-worktree bit turned on is being
virtualized and will be removed with the change of HEAD.

There was only one failing test when running with these changes that was
checking to make sure the worktree narrows on checkout which was
expected since we would no longer be narrowing the worktree.

Update 2022-04-05: temporarily set 'sparse.expectfilesoutsideofpatterns' in
test (until we start disabling the "remove present-despite-SKIP_WORKTREE"
behavior with 'core.virtualfilesystem' in a later commit).

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
While performing a fetch with a virtual file system we know that there
will be missing objects and we don't want to download them just because
of the reachability of the commits.  We also don't want to download a
pack file with commits, trees, and blobs since these will be downloaded
on demand.

This flag will skip the first connectivity check and by returning zero
will skip the upload pack. It will also skip the second connectivity
check but continue to update the branches to the latest commit ids.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
This is an early version of patches I am about to send upstream:
gitgitgadget#1719.

This addresses #623.
Ensure all filters and EOL conversions are blocked when running under
GVFS so that our projected file sizes will match the actual file size
when it is hydrated on the local machine.

Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
The idea is to allow blob objects to be missing from the local repository,
and to load them lazily on demand.

After discussing this idea on the mailing list, we will rename the feature
to "lazy clone" and work more on this.

Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
In particular when multiple processes want to write to the config
simultaneously, it would come in handy to not fail immediately when
another process locked the config, but to gently try again.

This will help with Scalar's functional test suite which wants to
register multiple repositories for maintenance semi-simultaneously.

As not all code paths calling this function read the config (e.g. `git
config`), we have to read the config setting via
`git_config_get_ulong()`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
By default, Git fails immediately when locking a config file for writing
fails due to an existing lock. With this change, Scalar-registered
repositories will fall back to trying a couple times within a 150ms
timeout.

Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
These docs have been altered to fit the version implemented in C within
microsoft/git. This means in particular that the advanced.md file no
longer applied at all. Some other areas were removed or significantly
edited.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Allow concurrent `scalar register` and `scalar unregister` calls to be
more collaborative when trying to lock the global Git config at the very
same time.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Adding the extra documentation from the Scalar project.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Git traditionally uses those, not backslashes, ever.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Use a fixed 3 tries total to see how that increases our chances of
success for subcommands such as 'git fetch'.

We special-case the `diagnose` command here: When 672196a
(scalar-diagnose: use 'git diagnose --mode=all', 2022-08-12) updated
'scalar diagnose' to run 'git diagnose' as a subprocess, it was passed
through the run_git() caller. We need to avoid repeating the call when
the underlying 'git diagnose' command fails.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
The .NET version supported running `scalar config` to reconfigure the
current enlistment, and now the C port does, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
During a run of the Scalar functional tests, we hit a case where the
inexact rename detection of a 'git cherry-pick' command slowed to the
point of writing its delayed progress, failing the test because stderr
differed from the control case. Showing progress like this when stderr
is not a terminal is non-standard for Git, so inject an isatty(2) when
initializing the progress option in sequencer.c.

Unfortunately, there is no '--quiet' option in 'git cherry-pick'
currently wired up. This could be considered in the future, and the
isatty(2) could be moved to that position. This would also be needed for
commands like 'git rebase', so we leave that for another time.

Reported-by: Victoria Dye <vdye@github.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Hydrate missing loose objects in check_and_freshen() when running
virtualized. Add test cases to verify read-object hook works when
running virtualized.

This hook is called in check_and_freshen() rather than
check_and_freshen_local() to make the hook work also with alternates.

Helped-by: Kevin Willford <kewillf@microsoft.com>
Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
This adds hard-coded call to GVFS.hooks.exe before and after each Git
command runs.

To make sure that this is only called on repositories cloned with GVFS, we
test for the tell-tale .gvfs.

2021-10-30: Recent movement of find_hook() to hook.c required moving these
changes out of run-command.c to hook.c.

Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Suggested by Ben Peart.

Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
Signed-off-by: Alejandro Pauly <alpauly@microsoft.com>
dscho and others added 19 commits May 14, 2024 20:10
This disables the `monitor-components` workflow in msft-git, and
re-enables the `win+VS` tests in this repository, too.
Implement workflow to create GitHub release with attached `git` installers
Fixes for MacOS release build & build options
This adds a new builtin, `git update-microsoft-git`, that executes the platform-specific upgrade steps to get the latest version of `microsoft-git`.

On Windows, this means running `git update-git-for-windows` which was updated to use the `microsoft/git` releases page, when appropriate. See #321 for details.

On macOS, this means running a sequence of `brew` commands. These are adapted from the `UpgradeVerb` in `microsoft/scalar`, with an important simplification: we don't need to differentiate between the `scalar` and `scalar-azrepos` cask.
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
…x-built-in-fsmonitor

Fix the built-in FSMonitor, and run Scalar's Functional Tests as part of the automated builds
This is random stuff that probably all got upstream in the meantime.
When scripts or background maintenance wish to perform HTTP(S) requests,
there is a risk that our stored credentials might be invalid. At the
moment, this causes the credential helper to ping the user and block the
process. Even if the credential helper does not ping the user, Git falls
back to the 'askpass' method, which includes a direct ping to the user
via the terminal.

Even setting the 'core.askPass' config as something like 'echo' will
causes Git to fallback to a terminal prompt. It uses
git_terminal_prompt(), which finds the terminal from the environment and
ignores whether stdin has been redirected. This can also block the
process awaiting input.

Create a new config option to prevent user interaction, favoring a
failure to a blocked process.

The chosen name, 'credential.interactive', is taken from the config
option used by Git Credential Manager to already avoid user
interactivity, so there is already one credential helper that integrates
with this option. However, older versions of Git Credential Manager also
accepted other string values, including 'auto', 'never', and 'always'.
The modern use is to use a boolean value, but we should still be
careful that some users could have these non-booleans. Further, we
should respect 'never' the same as 'false'. This is respected by the
implementation and test, but not mentioned in the documentation.

The implementation for the Git interactions takes place within
credential_getpass(). The method prototype is modified to return an
'int' instead of 'void'. This allows us to detect that no attempt was
made to fill the given credential, changing the single caller slightly.

Also, a new trace2 region is added around the interactive portion of the
credential request. This provides a way to measure the amount of time
spent in that region for commands that _are_ interactive. It also makes
a conventient way to test that the config option works with
'test_region'.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
At the moment, some background jobs are getting blocked on credentials
during the 'prefetch' task. This leads to other tasks, such as
incremental repacks, getting blocked. Further, if a user manages to fix
their credentials, then they still need to cancel the background process
before their background maintenance can continue working.

Update the background schedules for our four scheduler integrations to
include these config options via '-c' options:

 * 'credential.interactive=false' will stop Git and some credential
   helpers from prompting in the UI (assuming the '-c' parameters are
   carried through and respected by GCM).

 * 'core.askPass=true' will replace the text fallback for a username
   and password into the 'true' command, which will return a success in
   its exit code, but Git will treat the empty string returned as an
   invalid password and move on.

We can do some testing that the credentials are passed, at least in the
systemd case due to writing the service files.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
The 'scalar reconfigure' command is intended to update registered repos
with the latest settings available. However, up to now we were not
reregistering the repos with background maintenance.

In particular, this meant that the background maintenance schedule would
not be updated if there are improvements between versions.

Be sure to register repos for maintenance during the reconfigure step.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Add test case to demonstrate that `git index-pack -o <idx-path> pack-path`
fails if <idx-path> does not end in ".idx" when `--rev-index` is
enabled.

In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25)
we learned to create `.rev` reverse indexes in addition to `.idx` index
files.  The `.rev` file pathname is constructed by replacing the suffix
on the `.idx` file.  The code assumes a hard-coded "idx" suffix.

In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12)
reverse indexes were enabled by default.

If the `-o <idx-path>` argument is used, the index file may have a
different suffix.  This causes an error when it tries to create the
reverse index pathname.

The test here demonstrates the failure.  (The test forces `--rev-index`
to avoid interaction with `GIT_TEST_NO_WRITE_REV_INDEX` during CI runs.)

Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
During the latest v2.45.0 update, 'scalar reconfigure --all' started to
segfault on my machine. Breaking it down via the debugger, it was
faulting on a NULL reference to the_hash_algo, which is a macro pointing
to the_repository->hash_algo.

In my case, this is due to one of my repositories having a detached HEAD,
which requires get_oid_hex() to parse that the HEAD reference is valid.
Another way to cause a failure is to use the "includeIf.onbranch" config
key, which will lead to a BUG() statement.

My first inclination was to try to refactor cmd_reconfigure() to execute
'git for-each-repo' instead of this loop. In addition to the difficulty
of executing 'scalar reconfigure' within 'git for-each-repo', it would
be difficult to perform the clean-up logic for non-existent repos if we
relied on that child process.

Instead, I chose to move the temporary repo to be within the loop and
reinstate the_repository to its old value after we are done performing
logic on the current array item.

Add tests to t9210-scalar.sh to test 'scalar reconfigure --all' with
multiple registered repos. There are two different ways that the old
use of the_repository could trigger bugs. These issues are being solved
independently to be more careful about the_repository being
uninitialized, but the change in this patch around the use of
the_repository is still a good safety precaution.

Co-authored-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In ac8acb4 (sparse-index: complete partial expansion, 2022-05-23),
'expand_index()' was updated to expand the index to a given pathspec.
However, the 'path_matches_pattern_list()' method used to facilitate this
has the side effect of initializing or updating the index hash variables
('name_hash', 'dir_hash', and 'name_hash_initialized'). This operation is
performed on 'istate', though, not 'full'; as a result, the initialized
hashes are later overwritten when copied from 'full'. To ensure the correct
hashes are in 'istate' after the index expansion, change the arg used in
'path_matches_pattern_list()' from 'istate' to 'full'.

Note that this does not fully solve the problem. If 'istate' does not have
an initialized 'name_hash' when its contents are copied to 'full',
initialized hashes will be copied back into 'istate' but
'name_hash_initialized' will be 0. Therefore, we also need to copy
'full->name_hash_initialized' back to 'istate' after the index expansion is
complete.

Signed-off-by: Victoria Dye <vdye@github.com>
Teach index-pack to silently omit the reverse index if the
index file does not have the standard ".idx" suffix.

In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25)
we learned to create `.rev` reverse indexes in addition to `.idx` index
files.  The `.rev` file pathname is constructed by replacing the suffix
on the `.idx` file.  The code assumes a hard-coded "idx" suffix.

In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12)
reverse indexes were enabled by default.

If the `-o <idx-path>` argument is used, the index file may have a
different suffix.  This causes an error when it tries to create the
reverse index pathname.

Since we do not know why the user requested a non-standard suffix for
the index, we cannot guess what the proper corresponding suffix should
be for the reverse index.  So we disable it.

The t5300 test has been updated to verify that we no longer error
out and that the .rev file is not created.

TODO We could warn the user that we skipped it (perhaps only if they
TODO explicitly requested `--rev-index` on the command line).
TODO
TODO Ideally, we should add an `--rev-index-path=<path>` argument
TODO or change `--rev-index` to take a pathname.
TODO
TODO I'll leave these questions for a future series.

Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
In the Office monorepo, we've recently had an uptick in issues with
`scalar clone`. These issues didn't make sense at first and seemed like
the users weren't using `microsoft/git` but instead the upstream
version's `scalar clone`. Instead of using GVFS cache servers, they were
attempting to use the Git protocol's partial clone (which times out).

It turns out that what's actually happening is that some network issue
is causing the connection with Azure DevOps to error out during the
`/gvfs/config` request. In the Git traces, we see the following error
during this request:

  (curl:56) Failure when receiving data from the peer [transient]

This isn't 100% of the time, but has increased enough to cause problems
for a variety of users.

The solution being proposed in this pull request is to remove the
fall-back mechanism and instead have an explicit choice to use the GVFS
protocol. To avoid significant disruption to Azure DevOps customers (the
vast majority of `microsoft/git` users who use `scalar clone` based on
my understanding), I added some inferring of a default value from the
clone URL.

This fallback mechanism was first implemented in the C# version of
Scalar in microsoft/scalar#339. This was an attempt to make the Scalar
client interesting to non-Azure DevOps customers, especially as GitHub
was about to launch the availability of partial clones. Now that the
`scalar` client is available upstream, users don't need the GVFS-enabled
version to get these benefits.

In addition, this will resolve #384 since those requests won't happen
against non-ADO URLs unless requested.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
Add a test verifying that sparse-checkout (with and without sparse index
enabled) treat untracked files & directories correctly when changing sparse
patterns. Specifically, it ensures that 'git sparse-checkout set'

* deletes empty directories outside the sparse cone
* does _not_ delete untracked files outside the sparse cone

Signed-off-by: Victoria Dye <vdye@github.com>
Cherry-pick rev-index fixes from v2.41.0.vfs.0.5 into v2.42.0.*
I noticed the `osx-gcc` job failing in #647 so I found this upstream fix
from @peff. Merging now to unblock PR builds in `microsoft/git`.
@dscho dscho force-pushed the tentative/vfs-2.45.1 branch from c28c6af to d9ecdb8 Compare May 14, 2024 18:11
@derrickstolee derrickstolee merged commit d9ecdb8 into vfs-2.45.1 May 31, 2024
118 of 120 checks passed
@derrickstolee derrickstolee deleted the tentative/vfs-2.45.1 branch May 31, 2024 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.