`EDITOR=sed` related error when running `make_rebase_i_repo.sh`

[calam1]

These seem to be different errors though, maybe the shell on MacOS has trouble?

MacOS

2022-04-14T09:48:59.9196440Z stderr: /Users/runner/work/gitoxide/gitoxide/git-repository/tests/fixtures/make_rebase_i_repo.sh: line 33: [: 2 35 1: integer expression expected
2022-04-14T09:48:59.9196950Z fatal: --preserve-merges was replaced by --rebase-merges
2022-04-14T09:48:59.9197450Z ', tests/tools/src/lib.rs:115:9
2022-04-14T09:48:59.9197760Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2022-04-14T09:48:59.9197950Z 
2022-04-14T09:48:59.9197960Z 
2022-04-14T09:48:59.9198020Z failures:

Windows

2022-04-14T09:40:43.0699609Z stderr: which: no gsed in (/mingw64/bin:/usr/bin:/c/Users/runneradmin/bin:/d/a/gitoxide/gitoxide/target/debug/build/curl-sys-7e85c957548a17f0/out/build:/d/a/gitoxide/gitoxide/target/debug/build/libgit2-sys-a405fc67d8aee456/out/build:/d/a/gitoxide/gitoxide/target/debug/build/libssh2-sys-0acf82cde4d4f0d1/out/build:/d/a/gitoxide/gitoxide/target/debug/build/libz-sys-a02af73321b6f02b/out/lib:/d/a/gitoxide/gitoxide/target/debug/build/wepoll-ffi-169df33785c3c0f8/out:/d/a/gitoxide/gitoxide/target/debug/deps:/d/a/gitoxide/gitoxide/target/debug:/c/Users/runneradmin/.rustup/toolchains/stable-x86_64-pc-windows-msvc/lib/rustlib/x86_64-pc-windows-msvc/lib:/c/Users/runneradmin/.cargo/bin:/c/Users/runneradmin/.rustup/toolchains/stable-x86_64-pc-windows-msvc/bin:/c/Program Files/MongoDB/Server/5.0/bin:/c/aliyun-cli:/c/vcpkg:/c/cf-cli:/c/Program Files (x86)/NSIS:/c/tools/zstd:/c/Program Files/Mercurial:/c/hostedtoolcache/windows/stack/2.7.5/x64:/c/cabal/bin:/c/ghcup/bin:/c/tools/ghc-9.2.2/bin:/c/Program Files/dotnet:/c/mysql/bin:/c/Program Files/R/R-4.1.3/bin/x64:/c/SeleniumWebDrivers/GeckoDriver:/c/Program Files (x86)/sbt/bin:/c/Program Files (x86)/GitHub CLI:/bin:/c/Program Files (x86)/pipx_bin:/c/hostedtoolcache/windows/go/1.17.8/x64/bin:/c/hostedtoolcache/windows/Python/3.7.9/x64/Scripts:/c/hostedtoolcache/windows/Python/3.7.9/x64:/c/hostedtoolcache/windows/Ruby/2.5.9/x64/bin:/c/tools/kotlinc/bin:/c/hostedtoolcache/windows/Java_Temurin-Hotspot_jdk/8.0.322-6/x64/bin:/c/npm/prefix:/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/c/ProgramData/kind:/c/Program Files/Eclipse Foundation/jdk-8.0.302.8-hotspot/bin:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/ProgramData/Chocolatey/bin:/c/Program Files/Docker:/c/Program Files/PowerShell/7:/c/Program Files/Microsoft/Web Platform Installer:/c/Program Files/dotnet:/c/Program Files/Microsoft SQL Server/130/Tools/Binn:/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/c/Program Files (x86)/Microsoft SQL Server/110/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/120/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/130/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/140/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/150/DTS/Binn:/c/Program Files/nodejs:/c/Program Files/OpenSSL/bin:/c/Strawberry/c/bin:/c/Strawberry/perl/site/bin:/c/Strawberry/perl/bin:/c/ProgramData/chocolatey/lib/pulumi/tools/Pulumi/bin:/c/Program Files/TortoiseSVN/bin:/c/Program Files/CMake/bin:/c/ProgramData/chocolatey/lib/maven/apache-maven-3.8.5/bin:/c/Program Files/Microsoft Service Fabric/bin/Fabric/Fabric.Code:/c/Program Files/Microsoft SDKs/Service Fabric/Tools/ServiceFabricLocalClusterManager:/cmd:/mingw64/bin:/usr/bin:/c/Program Files/GitHub CLI:/c/tools/php:/c/Program Files (x86)/sbt/bin:/c/SeleniumWebDrivers/ChromeDriver:/c/SeleniumWebDrivers/EdgeDriver:/c/Program Files/Amazon/AWSCLIV2:/c/Program Files/Amazon/SessionManagerPlugin/bin:/c/Program Files/Amazon/AWSSAMCLI/bin:/c/Program Files (x86)/Google/Cloud SDK/google-cloud-sdk/bin:/c/Program Files (x86)/Microsoft BizTalk Server:/c/Program Files/LLVM/bin:/c/Users/runneradmin/.dotnet/tools:/c/Users/runneradmin/.cargo/bin:/c/Users/runneradmin/AppData/Local/Microsoft/WindowsApps)
2022-04-14T09:40:43.0705473Z D:\a\gitoxide\gitoxide\git-repository\tests\fixtures\make_rebase_i_repo.sh: line 33: bc: command not found
2022-04-14T09:40:43.0706030Z fatal: --preserve-merges was replaced by --rebase-merges
2022-04-14T09:40:43.0706440Z ', tests\tools\src\lib.rs:115:9
2022-04-14T09:40:43.0706836Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Originally posted by @Byron in #382 (comment)

[calam1]

I am on a Mac with an intel chip, running rust 1.60.0, using Nvim v0.7.0

So I am having issues with running the make tests command. I fail on the same place as this above error, but with a different error. I think it has something to do with this line in make_rebase_i_repo.sh EDITOR="sed -i.bak 's/pick/edit/g'" git rebase --rebase-merges --interactive HEAD~2

When i comment out that above line, I get an assertion error, which makes sense. I am wondering if this has something to do with nvim as the error message alludes to that.

  Finished test [unoptimized + debuginfo] target(s) in 0.87s
     Running tests/git.rs (target/debug/deps/git-c1ac60e3f9ad2185)

running 1 test
Archive at 'tests/fixtures/generated-archives/make_rebase_i_repo.tar.xz' not found, creating fixture using script 'make_rebase_i_repo.sh'
thread 'repository::state::rebase_interactive' panicked at 'repo script failed: stdout: [main (root-commit) 98e8fd1] 1
 Author: author <author@example.com>
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 1
[main 3b79d4c] 2
 Author: author <author@example.com>
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 2
[main 4637843] 3
 Author: author <author@example.com>
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 3

stderr: Vim: Error reading input, exiting...

Vim: Finished.
error: There was a problem with the editor '/usr/local/bin/nvim'.
', tests/tools/src/lib.rs:178:17
stack backtrace:
^[]11;rgb:2828/2828/2828^[\^[[O   0: rust_begin_unwind
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14
   2: git_testtools::scripted_fixture_repo_read_only_with_args
             at /Users/xcxl200/git/development/gitoxide/tests/tools/src/lib.rs:178:17
   3: git_testtools::scripted_fixture_repo_read_only
             at /Users/xcxl200/git/development/gitoxide/tests/tools/src/lib.rs:51:5
   4: git::named_repo
             at ./tests/git.rs:11:21
   5: git::repository::state::rebase_interactive
             at ./tests/repository/state.rs:55:16
   6: git::repository::state::rebase_interactive::{{closure}}
             at ./tests/repository/state.rs:54:1
   7: core::ops::function::FnOnce::call_once
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:227:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
test repository::state::rebase_interactive ... FAILED

failures:

failures:
    repository::state::rebase_interactive

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 40 filtered out; finished in 0.73s

error: test failed, to rerun pass '-p git-repository --test git'
The terminal process "cargo 'test', '--package', 'git-repository', '--test', 'git', '--', 'repository::state::rebase_interactive', '--exact', '--nocapture'" terminated with exit code: 101.

[Byron]

So I am having issues with running the make tests command. I fail on the same place as this above error, but with a different error.

Is there a way to see this different error?

[calam1]

Unless I am mistaken the error I get is in the code block in my second comment. I created the issue off of the commit of this unit test, and in the comments of that PR. So that was probably confusing. My bad

[Byron]

Unless I am mistaken the error I get is in the code block in my second comment.

The reason I ignored it is that it appears it was related to removing the EDITOR=sed part of the invocation, which would invoke the default editor of the system which is unrelated.
Please also be sure that you run the test on the latest main. It's likely something with the sed installed on the system, but since it works on CI my current stance is that it should be fixable on the host. The binary it should pick up is /usr/bin/sed, at least that's what's happening on MacOS for me.

[calam1]

Oh, the error that shows, does NOT have that EDITOR=sed removed. I removed it and ran the tests again, and got an assertion error, which makes sense, since the EDITOR part was not there, thus not ran.

Hmm, my sed is also in /usr/bin/sed but if it is working for you on your Mac, then it must be something weird on my terminal setup.

I'll close this and look further. Thanks!

[Byron]

Alright, please keep me posted if you discover the root cause of the issue. Maybe there is something that can be done to make this test more resilient (and it already came a long way).

[calam1]

Thus far, I have traced it down to this line in the tests/tools/src/lib.rs file the

       std::io::copy(&mut decoder, &mut buf)?;

this copy is crapping out and getting an failed to extract 'tests/fixtures/generated-archives/make_rebase_i_repo.tar.xz': lzma data error

For kicks I ran the cargo tests with GITOXIDE_TEST_IGNORE_ARCHIVES=1 although it bypassed the data error, it didn't work overall there were just other failures, and I think this setting was for linux or what-not, and I am running Mac OSX. Still looking though. I ran this with various versions of rust also, 1.59, 1.60, and 1.60-nightly same error

[calam1]

reopening for now, for visibility, feel free to close once reading and either clarifying, etc

so maybe this is the issue this file https://github.com/Byron/gitoxide/blob/main/git-repository/tests/fixtures/generated-archives/make_rebase_i_repo.tar.xz is a binary file. Makes sense it's a compressed file.

When I clone this repo and look at this file on my laptop it looks like this when i cat it.

 cat  gitoxide/git-repository/tests/fixtures/generated-archives/make_rebase_i_repo.tar.xz

version https://git-lfs.github.com/spec/v1
oid sha256:116d62faab198c39a3d11248a17f8efe1d3f78d879905fe6b5570587edc01b2c
size 11576

so the above lzma data error makes sense because it isn't a binary file since you are using xz2 to Decode.

       # test/tools/src/lib.rs
       ...
        let buffered_file = std::io::BufReader::new(input_archive);
        let mut decoder = xz2::bufread::XzDecoder::new(buffered_file);
        std::io::copy(&mut decoder, &mut buf)?;

[Byron]

I am afraid you were led down the wrong path: Archives are purely optional and errors loading them - for whichever reason - are non-fatal and lead to the original shell script to be executed.

Once it fails, errors are printed. It's possible to execute it yourself in any empty directory to reproduce the issue in the terminal which should help narrowing it down. The script is the only place that matters for determining the root cause of the issue, even though fixing it might involve other changes either to the script itself to handle some cases better, or to the host system.

Additional information will be required in order to resolve this - thanks for your persistence.

Edit: It's clear that it uses nvim instead of sed, so that's the issue. It's the question why that happens.

[calam1]

Thanks for the response. I’ll try your suggestions and keep digging. I will see if I can get further and will update.