Skip to content

Draft: kaldi: fix build#410200

Closed
matthiasdotsh wants to merge 1 commit intoNixOS:masterfrom
matthiasdotsh:fix/kaldi
Closed

Draft: kaldi: fix build#410200
matthiasdotsh wants to merge 1 commit intoNixOS:masterfrom
matthiasdotsh:fix/kaldi

Conversation

@matthiasdotsh
Copy link
Contributor

Fixes #410123

Resolve build issues with Kaldi due to OpenFST dependency on GCC14

When compiling OpenFST (a Kaldi dependency) with GCC13, numerous warnings occur, such as:
warning: expected 'template' keyword before dependent template name.

However, with GCC14, these warnings escalate to errors, causing the build to fail. Consequently, the Kaldi package is no longer building with versions >25.05, as the standard environment (stdenv) now relies on GCC14.

Building with gcc13Stdenv however still fails with:

> ERROR: noBrokenSymlinks: found 40 dangling symlinks, 0 reflexive symlinks and 0 unreadable symlinks

Looks like this is related to #370750

Setting dontCheckForBrokenSymlinks = true; can prevent it from happening.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Nixpkgs 25.11 Release Notes (or backporting 24.11 and 25.05 Nixpkgs Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
  • NixOS 25.11 Release Notes (or backporting 24.11 and 25.05 NixOS Release notes)
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

Resolve build issues with Kaldi due to OpenFST dependency on GCC14

When compiling OpenFST (a Kaldi dependency) with GCC13,
numerous warnings occur, such as:
`warning: expected 'template' keyword before dependent template name`.

However, with GCC14, these warnings escalate to errors, causing the
build to fail. Consequently, the Kaldi package is no longer building
with versions >25.05, as the standard environment (stdenv) now relies on
GCC14.

Building with gcc13Stdenv however still fails with:
```
> ERROR: noBrokenSymlinks: found 40 dangling symlinks, 0 reflexive symlinks and 0 unreadable symlinks
```

Looks like this is related to NixOS#370750

Setting `dontCheckForBrokenSymlinks = true;` can prevent it from
happening.
@matthiasdotsh matthiasdotsh requested a review from Mic92 May 23, 2025 15:15
@github-actions github-actions bot added 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels May 23, 2025
@Aleksanaa
Copy link
Member

When compiling OpenFST (a Kaldi dependency) with GCC13, numerous warnings occur, such as: warning: expected 'template' keyword before dependent template name.

However, with GCC14, these warnings escalate to errors, causing the build to fail. Consequently, the Kaldi package is no longer building with versions >25.05, as the standard environment (stdenv) now relies on GCC14.

New compilers tend to build more optimized and more correct binary code, so it may not be appropriate to return to the old compiler because of stricter inspections. The best way here is to correct these errors, submit patch upstream, or extract patches from existing commits. A less good but working way is to surppress errors with Wno-error=.

Building with gcc13Stdenv however still fails with:

> ERROR: noBrokenSymlinks: found 40 dangling symlinks, 0 reflexive symlinks and 0 unreadable symlinks

Looks like this is related to #370750

The PR is adding the check, aiming to let you know there are broken symlinks. Instead of figuring out why there are dangling symlinks, you disabled the check. Why?

@eclairevoyant
Copy link
Contributor

eclairevoyant commented May 23, 2025

Please change the PR and commit titles to use lowercase kaldi, the CI is case-sensitive.

Instead of figuring out why there are dangling symlinks, you disabled the check.

Allegedly this is the correct action (if the cause is upstream's code). Though I've no idea the cause in this case.

@matthiasdotsh
Copy link
Contributor Author

The best way here is to correct these errors, submit patch upstream, or extract patches from existing commits.

I completely agree with you. Unfortunately, it seems to me that the version used by Kaldi, is no longer being properly maintained.

Perhaps we can patch this temporarily for the Nix package (though I don't fully understand how this works with the update scripts in the Nix code).

Basically, I feel that Kaldi is no longer being actively maintained; most people are now using the next-generation kaldi (Icefall, K2, Sherpa, and Lhotse) and Kaldi is just being kept alive on the side.

Looks like this is related to #370750

The PR is adding the check, aiming to let you know there are broken symlinks. Instead of figuring out why there are dangling symlinks, you disabled the check. Why?

I believe this is because we are only using files from the experiments folder (egs). It is common practice to copy files from this folder to your own experiment folder.

When using something like

   postInstall = ''
     mkdir -p $out/share/kaldi
-    cp -r ../egs $out/share/kaldi
+    cp -r --dereference ../egs $out/share/kaldi
   '';

There are no broken symlinks, but the folder size increases significantly.

# with --dereference
$ du -sh result/share/kaldi/egs
1.1G    result/share/kaldi/egs

# Using `dontCheckForBrokenSymlinks = true;` to ignore broken symlinks
$ du -sh result/share/kaldi/egs
49M     result/share/kaldi/egs

@matthiasdotsh matthiasdotsh changed the title Kaldi: fix build kaldi: fix build May 23, 2025
@matthiasdotsh matthiasdotsh changed the title kaldi: fix build Draft: kaldi: fix build May 26, 2025
@matthiasdotsh matthiasdotsh marked this pull request as draft May 26, 2025 14:34
@matthiasdotsh
Copy link
Contributor Author

I did some futher investigations and it looks like the root of the error is

/nix/store/xb8h6kfjbqclshgxq5ikh9maavxn3z1g-source/src/include/fst/fst.h:655:59: error: no match for 'operator=' (operand types are 'std::unique_ptr<fst::SymbolTable>' and 'fst::SymbolTable*')
  655 |     isymbols_ = impl.isymbols_ ? impl.isymbols_->Copy() : nullptr;
      |                                                           ^~~~~~~

With the following patch openfst builds fine when compiling in a dev shell:

$ cat pkgs/by-name/ka/kaldi/0001-Fix-build-for-gcc-14.patch
From 580bd3f0fea7ddc913329537070ab08fd3bf6033 Mon Sep 17 00:00:00 2001
From: matthiasdotsh <git@matthias.sh>
Date: Mon, 26 May 2025 11:22:01 +0200
Subject: [PATCH] Fix build for gcc >=14

---
 src/include/fst/fst.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/include/fst/fst.h b/src/include/fst/fst.h
index 20e6bb3..2cb1364 100644
--- a/src/include/fst/fst.h
+++ b/src/include/fst/fst.h
@@ -652,8 +652,8 @@ class FstImpl {
   FstImpl &operator=(const FstImpl<Arc> &impl) {
     properties_ = impl.properties_;
     type_ = impl.type_;
-    isymbols_ = impl.isymbols_ ? impl.isymbols_->Copy() : nullptr;
-    osymbols_ = impl.osymbols_ ? impl.osymbols_->Copy() : nullptr;
+    isymbols_ = impl.isymbols_ ? std::unique_ptr<fst::SymbolTable>(impl.isymbols_->Copy()) : nullptr;
+    osymbols_ = impl.osymbols_ ? std::unique_ptr<fst::SymbolTable>(impl.osymbols_->Copy()) : nullptr;
     return *this;
   }

--
2.49.0

Compile in a devshell

git clone https://github.com/kkm000/openfst
cd openfst
git checkout 338225416178ac36b8002d70387f5556e44c8d05
# Apply patch
nix-shell -p gcc automake115x  autoconf perl --pure
# gcc --version # gcc (GCC) 14.2.1 20250322
./configure
make

Can someone kindly give me a tip or help me with how I can add this to the Kaldi nix package?

@vcunat
Copy link
Member

vcunat commented May 31, 2025

Done one way in PR #412663
(not sure if there's some significantly better way)

@matthiasdotsh matthiasdotsh mentioned this pull request May 31, 2025
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build failure: kaldi

4 participants