Precommit CI in an (entirely) broken state

[AaronBallman]

The precommit CI bot appears to be broken again:

$ trap 'kill -- $$' INT TERM QUIT; pip install -q -r scripts/requirements.txt
scripts/apply_patch.sh
INFO    repository exist, will reuse
INFO    working dir /var/lib/buildkite-agent/builds/llvm-project-fork
INFO    Syncing local, origin and upstream...
INFO    Backing off reset_repository(...) for 2.6s (git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git reset --hard
  stderr: 'fatal: Unable to create '/var/lib/buildkite-agent/builds/llvm-project-fork/.git/index.lock': File exists.
 
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.')
INFO    Syncing local, origin and upstream...
INFO    Backing off reset_repository(...) for 3.5s (git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git reset --hard
  stderr: 'fatal: Unable to create '/var/lib/buildkite-agent/builds/llvm-project-fork/.git/index.lock': File exists.
 
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.')
INFO    Syncing local, origin and upstream...
INFO    Backing off reset_repository(...) for 11.6s (git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git reset --hard
  stderr: 'fatal: Unable to create '/var/lib/buildkite-agent/builds/llvm-project-fork/.git/index.lock': File exists.
 
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.')
INFO    Syncing local, origin and upstream...
INFO    Backing off reset_repository(...) for 13.7s (git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git reset --hard
  stderr: 'fatal: Unable to create '/var/lib/buildkite-agent/builds/llvm-project-fork/.git/index.lock': File exists.
 
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.')
INFO    Syncing local, origin and upstream...
ERROR   Giving up reset_repository(...) after 5 tries (git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git reset --hard
  stderr: 'fatal: Unable to create '/var/lib/buildkite-agent/builds/llvm-project-fork/.git/index.lock': File exists.
 
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.')
ERROR   exception: Cmd('git') failed due to: exit code(128)
  cmdline: git reset --hard
  stderr: 'fatal: Unable to create '/var/lib/buildkite-agent/builds/llvm-project-fork/.git/index.lock': File exists.
 
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.'
Reporting results to Phabricator build https://reviews.llvm.org/harbormaster/build/256926
 
[report issue](https://github.com/google/llvm-premerge-checks/issues/new?assignees=&labels=bug&template=bug_report.md&title=buildkite%20build%20diff-checks%20111361)

As an example: https://buildkite.com/llvm-project/diff-checks/builds/111361 but I've been seeing it in most of my reviews today.

[llvmbot]

@llvm/issue-subscribers-infrastructure

[asl]

Tagging @ChristianKuehnel @metaflow

[dwblaikie]

@ChristianKuehnel

[dwblaikie]

Might be google/llvm-premerge-checks#404

[metaflow]

I have stopped the offending bot.

[AaronBallman]

Thanks, I can confirm that the Precommit CI seems to be fixed now, so I'm closing this issue.