Major relx release 4.0 will no longer be an escript

[tsloughter]

Opening this to let you know relx won't be usable the same way by erlang.mk in the next major release.

relx 4.0 won't build to an escript anymore but it should still be usable simply by calling into it like erlang.mk already does for some other tasks with erl.

Another option would be creating a small escript wrapper.

Or I could see about adding a bare release task to rebar3 that doesn't do any of the other rebar3 shit and just calls relx with the args/config needed for erlang.mk to work.

[essen]

Thanks for the heads up, I'll experiment when time allows. Probably just call with erl indeed.

[artman41]

Hi @essen, any update on this?

We were looking at moving to OTP24 but we've encountered an issue with Erlang.mk where we can't build because of relx and since relx isn't an escript anymore, I'm not quite sure how to proceed.

Is the solution to move away from erlang.mk and to rebar3 for now?

[essen]

Was not aware there was an issue with 24, can you be more specific? I will try/fix tomorrow.

[artman41]

Versions:

git version 2.27.0
Relx 3.27.0
Erlang.mk Commit d80984c
Erlang/OTP 24 [erts-12.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [jit]

Steps:

• run make rel after doing make deps app

Issue:

It seems to fail at line 7505 in erlang.mk when it executes this stage, specifically on the first call to $(RELX)

relx-rel: $(RELX) rel-deps app
	$(verbose) $(RELX) $(if $(filter 1,$V),-V 3) -c $(RELX_CONFIG) $(RELX_OPTS) release
	$(verbose) $(MAKE) relx-post-rel

Running make rel V=1, I can see the following for a handful of deps but I'm pretty sure it's a Relx issue (could be erlang.mk but I'm unsure)

        folsom-Unknown option: -C
usage: git [--version] [--help] [-c name=value]
           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
           [-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
           <command> [<args>]: /<PROJECTPATH>/deps/folsom
               Symlink: false
               Active Dependencies:
                    kernel,
                    stdlib,
               Library Dependencies:

My git version supports the -C flag though

$ git -h
unknown option: -h
usage: git [--version] [--help] [-C <path>] [-c <name>=<value>]
           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]
           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
           <command> [<args>]

Relx Error:

=ERROR REPORT==== 7-Oct-2021::10:35:51.884204 ===
Error in process <0.10404.0> with exit value:
{{badmatch,{error,enametoolong}},
 [{rlx_prv_assembler,copy_dir,5,
                     [{file,"/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl"},
                      {line,327}]},
  {rlx_prv_assembler,'-copy_directory/5-lc$^0/1-0-',5,
                     [{file,"/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl"},
                      {line,309}]},
  {rlx_prv_assembler,copy_app_,5,
                     [{file,"/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl"},
                      {line,228}]},
  {lists,map,2,[{file,"lists.erl"},{line,1243}]},
  {ec_plists,'-local_runmany/3-fun-0-',3,
             [{file,"/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/_build/default/lib/erlware_commons/src/ec_plists.erl"},
              {line,753}]}]}

=ERROR REPORT==== 7-Oct-2021::10:35:51.884461 ===
Error in process <0.10403.0> with exit value:
{{badmatch,{error,enametoolong}},
 [{rlx_prv_assembler,copy_dir,5,
                     [{file,"/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl"},
                      {line,327}]},
  {rlx_prv_assembler,'-copy_directory/5-lc$^0/1-0-',5,
                     [{file,"/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl"},
                      {line,309}]},
  {rlx_prv_assembler,copy_app_,5,
                     [{file,"/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl"},
                      {line,228}]},
  {lists,map,2,[{file,"lists.erl"},{line,1243}]},
  {ec_plists,'-local_runmany/3-fun-0-',3,
             [{file,"/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/_build/default/lib/erlware_commons/src/ec_plists.erl"},
              {line,753}]}]}

escript: exception exit: {{badmatch,{error,enametoolong}},
                 [{rlx_prv_assembler,copy_dir,5,
                      [{file,
                           "/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl"},
                       {line,327}]},
                  {rlx_prv_assembler,'-copy_directory/5-lc$^0/1-0-',5,
                      [{file,
                           "/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl"},
                       {line,309}]},
                  {rlx_prv_assembler,copy_app_,5,
                      [{file,
                           "/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl"},
                       {line,228}]},
                  {lists,map,2,[{file,"lists.erl"},{line,1243}]},
                  {ec_plists,'-local_runmany/3-fun-0-',3,
                      [{file,
                           "/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/_build/default/lib/erlware_commons/src/ec_plists.erl"},
                       {line,753}]}]}
  in function  ec_plists:handle_error/3 (/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/_build/default/lib/erlware_commons/src/ec_plists.erl, line 917)
  in call from ec_plists:local_runmany/3 (/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/_build/default/lib/erlware_commons/src/ec_plists.erl, line 764)
  in call from rlx_prv_assembler:copy_app_directories_to_output/3 (/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl, line 176)
  in call from rlx_prv_assembler:do/1 (/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/rlx_prv_assembler.erl, line 59)
  in call from relx:run_provider/2 (/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/relx.erl, line 308)
  in call from lists:foldl/3 (lists.erl, line 1267)
  in call from relx:run_providers_for_actions/2 (/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/relx.erl, line 291)
  in call from relx:main/2 (/usr/local/var/buildkite-agent/builds/macos-1/ninenines/relx/src/relx.erl, line 65)
make: *** [relx-rel] Error 127

[essen]

This suggests that you are running this in a deep sub directory, which results in file names too long for the operation Relx is trying to do. I'm not sure why going to 24 gives you problems when there were none before. Note that the tests pass just fine on 24 for me (tested just a few minutes ago).

Is this Windows by any chance?

[artman41]

I've updated my above comment with more info

@essen Linux, done a bit more digging and looks like relx is using an older version of git from what I can tell - any idea whether relx bundles git inside the release?

The only git I have installed supports the -C flag

$ yum list installed | grep git
crontabs.noarch                             1.11-6.20121102git.el7     @anaconda
hyperv-daemons.x86_64                       0-0.34.20180415git.el7     @anaconda
hyperv-daemons-license.noarch               0-0.34.20180415git.el7     @anaconda
hypervfcopyd.x86_64                         0-0.34.20180415git.el7     @anaconda
hypervkvpd.x86_64                           0-0.34.20180415git.el7     @anaconda
hypervvssd.x86_64                           0-0.34.20180415git.el7     @anaconda
ipxe-roms-qemu.noarch                       20180825-3.git133f4c.el7   @anaconda
libglvnd.i686                               1:1.0.1-0.8.git5baa1e5.el7 @base    
libglvnd.x86_64                             1:1.0.1-0.8.git5baa1e5.el7 @anaconda
libglvnd-egl.i686                           1:1.0.1-0.8.git5baa1e5.el7 @base    
libglvnd-egl.x86_64                         1:1.0.1-0.8.git5baa1e5.el7 @anaconda
libglvnd-gles.x86_64                        1:1.0.1-0.8.git5baa1e5.el7 @anaconda
libglvnd-glx.i686                           1:1.0.1-0.8.git5baa1e5.el7 @base    
libglvnd-glx.x86_64                         1:1.0.1-0.8.git5baa1e5.el7 @anaconda
linux-firmware.noarch                       20200421-80.git78c0348.el7_9
lldpad.x86_64                               1.0.1-5.git036e314.el7     @anaconda
lm_sensors.x86_64                           3.4.0-8.20160601gitf9185e5.el7
lm_sensors-libs.x86_64                      3.4.0-8.20160601gitf9185e5.el7
lockdev.x86_64                              1.0.4-0.13.20111007git.el7 @anaconda
net-tools.x86_64                            2.0-0.25.20131004git.el7   @anaconda
numad.x86_64                                0.5-18.20150602git.el7     @anaconda
polkit-gnome.x86_64                         0.106-0.1.20170423gita0763a2.el7
python-dns.noarch                           1.12.0-4.20150617git465785f.el7
python-pillow.x86_64                        2.0.0-21.gitd1c6db8.el7    @base    
rh-git227.x86_64                            3.6-1.el7                  @centos-sclo-rh
rh-git227-git.x86_64                        2.27.0-3.el7               @centos-sclo-rh
rh-git227-git-core.x86_64                   2.27.0-3.el7               @centos-sclo-rh
rh-git227-git-core-doc.noarch               2.27.0-3.el7               @centos-sclo-rh
rh-git227-perl-Git.noarch                   2.27.0-3.el7               @centos-sclo-rh
rh-git227-runtime.x86_64                    3.6-1.el7                  @centos-sclo-rh
squashfs-tools.x86_64                       4.3-0.21.gitaae0aff4.el7   @base    
taglib.x86_64                               1.8-8.20130218git.el7      @anaconda
xdg-utils.noarch                            1.1.0-0.17.20120809git.el7 @anaconda

[essen]

Sorry I had not seen the git info when I replied. I am not sure what calls an old git, but it's not bundled with erlang.mk or relx. The caller of the old git is most likely Erlang.mk. You can check the app file for folsom for example and confirm that the vsn line is the error that is printed.

Finding where this git program is and removing it might solve your issue. which git would likely tell you. If not, then perhaps it is only found in a path set by Make or similar.

[artman41]

@essen did some more digging and think I've found what the problem is - will raise a separate issue to refrain from muddying this one again :)

Worthwhile me deleting my comments on this issue do you think?

---

Looks like the same issue as #924 - an issue I raised back in feb 🤦
Apologies for wasting your time Essen, should've realised it was the same issue

[essen]

Let's keep all the context thanks!

[essen]

For the record this is done in the relx4 branch. Plans are to merge it for OTP-25. This currently requires a custom Relx branch until erlware/relx#913 gets merged or otherwise fixed, and that branch will be the one that Erlang.mk will use automatically unless specified otherwise.

Context is at #945 so I'll be closing this ticket. Thanks!