Skip to content

Commit

Permalink
perf: faster blobless git clones
Browse files Browse the repository at this point in the history
Cloning a repo with git can be really slow if the repo is big.

Adding `--filter=blob:none` to the `git clone` command will make it really faster and slimmer because it will lazy-download blobs on checkout, while only getting commit metadata for whatever is not checked out.

See "Blobless Clones" in https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/ for a good explanation.

Also interesting to see that pip 21.3 itself landed this enhancement: pypa/pip#9086.

@moduon MT-83
  • Loading branch information
yajo committed Apr 4, 2022
1 parent 6b3ed4a commit ffe2a37
Showing 1 changed file with 21 additions and 3 deletions.
24 changes: 21 additions & 3 deletions src/poetry/core/vcs/git.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from collections import namedtuple
from pathlib import Path
from typing import Any
from typing import Any, Tuple

from poetry.core.utils._compat import WINDOWS

Expand Down Expand Up @@ -255,10 +255,28 @@ def normalize_url(cls, url: str) -> GitUrl:
def config(self) -> GitConfig:
return self._config

@property
def version(self) -> Tuple[int, int, int]:
output = self.run("version")
version = re.search(r"(\d+)\.(\d+)\.(\d+)", output)
if not version:
return (0, 0, 0)
return int(version.group(1)), int(version.group(2)), int(version.group(3))

def clone(self, repository: str, dest: Path) -> str:
self._check_parameter(repository)

return self.run("clone", "--recurse-submodules", "--", repository, str(dest))
cmd = [
"clone",
"--filter=blob:none",
"--recurse-submodules",
"--",
repository,
str(dest),
]
# Blobless clones introduced in Git 2.17
if self.version < (2, 17):
cmd.remove("--filter=blob:none")
return self.run(*cmd)

def checkout(self, rev: str, folder: Path | None = None) -> str:
args = []
Expand Down

0 comments on commit ffe2a37

Please sign in to comment.