lomiri.geonames: init at 0.3.0 by OPNA2608 · Pull Request #241259 · NixOS/nixpkgs

OPNA2608 · 2023-07-03T10:52:12Z

Description of changes

Working towards #99090.

Geonames, a library for parsing and querying a local copy of the geonames.org database. Required by Lomiri & some of its system applications.

Cross-compilation is known-broken without some hostPlatform emulation: https://gitlab.com/ubports/development/core/geonames/-/issues/1. I've used stdenv.hostPlatform.emulator buildPackages to make at least aarch64-linux and armv7l-linux cross work. Upstream issue also mentions glibc -> musl cross for same buildPlatform being broken, but that fails much earlier in some dependencies.

Things done

wineee · 2023-07-06T02:01:26Z

pkgs/desktops/lomiri/development/geonames/default.nix

We have added glib to nativeBuildInputs， Setting CMAKE_PROGRAM_PATH here may no longer be necessary

Without this, it still picks up the one from buildInputs instead of nativeBuildInputs.

--argstr crossSystem riscv64-linux with this line removed:

[ 41%] Generating geonames-resources.c /nix/store/51sszqz1d9kpx480scb1vllc00kxlx79-bash-5.2-p15/bin/bash: line 1: /nix/store/76cd2vr6rsp3fzwlrpaq6n8phdklnvq0-glib-riscv64-unknown-linux-gnu-2.76.3-dev/bin/glib-compile-resources: cannot execute binary file: Exec format error make[2]: *** [src/CMakeFiles/geonames.dir/build.make:74: src/geonames-resources.c] Error 126

Test nix-build -A pkgsCross.riscv64.lomiri.geonames , It should be built successfully without CMAKE_PROGRAM_PATH

It does not, same error.

[ 41%] Generating geonames-resources.c /nix/store/51sszqz1d9kpx480scb1vllc00kxlx79-bash-5.2-p15/bin/bash: line 1: /nix/store/76cd2vr6rsp3fzwlrpaq6n8phdklnvq0-glib-riscv64-unknown-linux-gnu-2.76.3-dev/bin/glib-compile-resources: cannot execute binary file: Exec format error make[2]: *** [src/CMakeFiles/geonames.dir/build.make:74: src/geonames-resources.c] Error 126 make[1]: *** [CMakeFiles/Makefile2:199: src/CMakeFiles/geonames.dir/all] Error 2 make: *** [Makefile:136: all] Error 2 error: builder for '/nix/store/vccsfh93aaznxm0q30y4q6vjld7qa4nh-geonames-riscv64-unknown-linux-gnu-0.3.0.drv' failed with exit code 2;

[nix-shell:~/.cache/nixpkgs-review/pr-241259/nixpkgs]$ git diff pkgs/desktops/lomiri/development/geonames/default.nix ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ───────────────────────────────────────┐ 85: stdenv.mkDerivation (finalAttrs: { │ ───────────────────────────────────────┘ "-DWANT_DEMO=${lib.boolToString withExamples}" "-DWANT_TESTS=${lib.boolToString finalAttrs.doCheck}" # Keeps finding & using glib-compile-resources from buildInputs otherwise "-DCMAKE_PROGRAM_PATH=${lib.makeBinPath [ buildPackages.glib.dev ]}" #"-DCMAKE_PROGRAM_PATH=${lib.makeBinPath [ buildPackages.glib.dev ]}" ]; preInstall = lib.optionalString withDocumentation '' [nix-shell:~/.cache/nixpkgs-review/pr-241259/nixpkgs]$ nix-build -A lomiri.geonames --argstr crossSystem riscv64-linux /nix/store/3m45psnzk57q55zy8xgwd2q449a4jahq-geonames-riscv64-unknown-linux-gnu-0.3.0

Unable to reproduce build failure

Weird.

[ 25%] Built target generated-files [ 33%] Generating locales-gen [ 33%] Built target localesgen [ 41%] Generating geonames-resources.c /nix/store/51sszqz1d9kpx480scb1vllc00kxlx79-bash-5.2-p15/bin/bash: line 1: /nix/store/76cd2vr6rsp3fzwlrpaq6n8phdklnvq0-glib-riscv64-unknown-linux-gnu-2.76.3-dev/bin/glib-compile-resources: cannot execute binary file: Exec format error make[2]: *** [src/CMakeFiles/geonames.dir/build.make:74: src/geonames-resources.c] Error 126 make[1]: *** [CMakeFiles/Makefile2:199: src/CMakeFiles/geonames.dir/all] Error 2 make: *** [Makefile:136: all] Error 2 error: builder for '/nix/store/vccsfh93aaznxm0q30y4q6vjld7qa4nh-geonames-riscv64-unknown-linux-gnu-0.3.0.drv' failed with exit code 2; last 10 log lines: > /build/source/data/alternatenames/GS.txt > /build/source/data/alternatenames/GT.txt > [ 25%] Built target generated-files > [ 33%] Generating locales-gen > [ 33%] Built target localesgen > [ 41%] Generating geonames-resources.c > /nix/store/51sszqz1d9kpx480scb1vllc00kxlx79-bash-5.2-p15/bin/bash: line 1: /nix/store/76cd2vr6rsp3fzwlrpaq6n8phdklnvq0-glib-riscv64-unknown-linux-gnu-2.76.3-dev/bin/glib-compile-resources: cannot execute binary file: Exec format error > make[2]: *** [src/CMakeFiles/geonames.dir/build.make:74: src/geonames-resources.c] Error 126 > make[1]: *** [CMakeFiles/Makefile2:199: src/CMakeFiles/geonames.dir/all] Error 2 > make: *** [Makefile:136: all] Error 2 For full logs, run 'nix log /nix/store/vccsfh93aaznxm0q30y4q6vjld7qa4nh-geonames-riscv64-unknown-linux-gnu-0.3.0.drv'. [nix-shell:~/.cache/nixpkgs-review/pr-241259/nixpkgs]$ git diff diff --git a/pkgs/desktops/lomiri/development/geonames/default.nix b/pkgs/desktops/lomiri/development/geonames/default.nix index c4e327caf93..3532916a50c 100644 --- a/pkgs/desktops/lomiri/development/geonames/default.nix +++ b/pkgs/desktops/lomiri/development/geonames/default.nix @@ -86,7 +86,7 @@ stdenv.mkDerivation (finalAttrs: { "-DWANT_DEMO=${lib.boolToString withExamples}" "-DWANT_TESTS=${lib.boolToString finalAttrs.doCheck}" # Keeps finding & using glib-compile-resources from buildInputs otherwise - "-DCMAKE_PROGRAM_PATH=${lib.makeBinPath [ buildPackages.glib.dev ]}" + #"-DCMAKE_PROGRAM_PATH=${lib.makeBinPath [ buildPackages.glib.dev ]}" ]; preInstall = lib.optionalString withDocumentation ''

Do you maybe have riscv64-linux in your boot.binfmt.emulatedSystems so you can execute risc-v binaries on your system? I think that would cause it to "work".

Warning: the builder can execute all emulated systems within the same build, which introduces impurities in the case of cross compilation.

I have aarch64-linux in mine, and cross-compiling for that likely "works" without CMAKE_PROGRAM_PATH because of it.

yes, that makes sense

pkgs/desktops/lomiri/development/geonames/default.nix

nixos-discourse · 2023-07-11T09:37:01Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/2415

pkgs/desktops/lomiri/development/geonames/default.nix

SuperSandro2000 · 2023-07-11T21:31:00Z

pkgs/desktops/lomiri/development/geonames/default.nix

+    substituteInPlace doc/reference/CMakeLists.txt \
+      --replace "\''${CMAKE_INSTALL_DATADIR}/gtk-doc/html/\''${PROJECT_NAME}" "\''${CMAKE_INSTALL_DOCDIR}"
+    substituteInPlace demo/CMakeLists.txt \
+      --replace 'RUNTIME DESTINATION bin' 'RUNTIME DESTINATION ''${CMAKE_INSTALL_BINDIR}'
+  '' + lib.optionalString (stdenv.buildPlatform != stdenv.hostPlatform) ''
+    # Built for hostPlatform, executed during build
+    substituteInPlace src/CMakeLists.txt \
+      --replace 'COMMAND mkdb' 'COMMAND ${stdenv.hostPlatform.emulator buildPackages} mkdb'


Is this upstreamable?

I don't see how this could be upstreamed. Ideally upstream shouldn't rely on the user having an emulator to run the build-time executable - building for the native system (buildPlatform) in a cross situation with CMake is abit tricky but IMO the real solution here. This is more of a crutch to deal with CMake not having first-class support for something like that, and relies on us knowing the correct emulator command for the cross platform.

We could use CMAKE_CROSSCOMPILING_EMULATOR=${stdenv.hostPlatform.emulator buildPackages} instead of patching in our emulator call, I guess that would be slightly better, but still not a real fix.

Switched to setting CMAKE_CROSSCOMPILING_EMULATOR, confirmed that it works by cross-compiling for armv7l-linux.

Ideally upstream will address this with a proper fix Soon:tm:, either for or after the 22.04 release of ubtouch.

things like CMAKE_INSTALL_DOCDIR/CMAKE_INSTALL_BINDIR should be upstreamable

Ah right, that stuff is upstreamable: https://gitlab.com/ubports/development/core/geonames/-/merge_requests/3

…g in emulator call

wineee · 2023-07-15T15:46:59Z

Result of nixpkgs-review pr 241259 run on x86_64-linux 1

4 packages built:

lomiri.geonames
lomiri.geonames.bin
lomiri.geonames.dev
lomiri.geonames.doc

nixos-discourse · 2023-07-22T00:22:49Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/1027

OPNA2608 force-pushed the init/lomiri/geonames branch 2 times, most recently from f3630fa to f748b3c Compare July 5, 2023 12:26

OPNA2608 marked this pull request as ready for review July 5, 2023 12:27

ofborg bot added 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. labels Jul 5, 2023

OPNA2608 force-pushed the init/lomiri/geonames branch from f748b3c to 1cf2ad9 Compare July 5, 2023 21:52

wineee requested changes Jul 6, 2023

View reviewed changes

wineee requested changes Jul 11, 2023

View reviewed changes

pkgs/desktops/lomiri/development/geonames/default.nix Outdated Show resolved Hide resolved

lomiri.geonames: init at 0.3.0

983e3f7

OPNA2608 force-pushed the init/lomiri/geonames branch from 1cf2ad9 to 983e3f7 Compare July 11, 2023 09:02

wineee approved these changes Jul 11, 2023

View reviewed changes

SuperSandro2000 reviewed Jul 11, 2023

View reviewed changes

lomiri.geonames: Use CMAKE_CROSSCOMPILING_EMULATOR instead of patchin…

609c49f

…g in emulator call

OPNA2608 requested a review from SuperSandro2000 July 12, 2023 07:24

lomiri.geonames: Upstream better GNUInstallDirs variable usage

b2f4cdf

OPNA2608 requested a review from wineee July 14, 2023 16:07

wineee approved these changes Jul 15, 2023

View reviewed changes

OPNA2608 added 12.approvals: 1 This PR was reviewed and approved by one person. 8.has: package (new) This PR adds a new package and removed 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. labels Jul 15, 2023

SuperSandro2000 merged commit 61677c9 into NixOS:master Jul 22, 2023

OPNA2608 mentioned this pull request Oct 13, 2023

Tracking: Lomiri Desktop Environment #260859

Open

Uh oh!

Conversation

OPNA2608 commented Jul 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OPNA2608 Jul 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wineee Jul 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OPNA2608 Jul 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nixos-discourse commented Jul 11, 2023

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OPNA2608 Jul 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wineee commented Jul 15, 2023

Uh oh!

nixos-discourse commented Jul 22, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Comments

OPNA2608 commented Jul 3, 2023 •

edited

Loading

OPNA2608 Jul 6, 2023 •

edited

Loading

wineee Jul 11, 2023 •

edited

Loading

OPNA2608 Jul 11, 2023 •

edited

Loading

OPNA2608 Jul 12, 2023 •

edited

Loading