fix(main/boost): prepend $TERMUX_PREFIX to all absolute /tmp paths

[robertkirkman]

• Fixes a problem explained here (that boost lacks needed /tmp path patches) addpkg(main/supercollider): 3.14.0 #25025

• Methodology to find paths to prepend:

  • Ignore all folders named doc
  • Ignore all folders named test
  • Find them using grep -rn $TERMUX_PKG_SRCDIR -e '"/tmp' -e 'data/local/tmp'
  • Ignore codepaths that don't compile for Android (#ifndef __ANDROID__)
  • Try to calculate the code behavior to make sure every patched instance is a true absolute path (and not a path subcomponent that could be appended to a longer path)

• Dependency of addpkg(main/supercollider): 3.14.0 #25826

• Enable on-device building because it does not modify any files owned by any packages that are not boost or one of its subpackages

[rene-descartes2021]

Dunno if this matters in os_file_functions.hpp their sanity check on buffer size is set to 5u: "/tmp" length plus null character. Dunno if you'd want to add the length of TERMUX_PREFIX to that. I noticed that but didn't fix it in my patch as it wasn't used by supercollider.

In my patch) I just replaced /tmp with a call to std::filesystem::temp_directory_path().string() which first tries env variables like TMPDIR one of which is set on Termux and works. If no such env var was set it'd default to /tmp, I didn't fix that, your fix to boost does probably.

Well that's all I have to say dunno if it matters.

[robertkirkman]

Dunno if this matters in os_file_functions.hpp their sanity check on buffer size is set to 5u: "/tmp" length plus null character. Dunno if you'd want to add the length of TERMUX_PREFIX to that. I noticed that but didn't fix it in my patch as it wasn't used by supercollider.

Wow thank you, you are right and I didn't notice that at all. Now I've changed it to ensure the size of the destination buffer is always correct, and otherwise return false from the function (to avoid buffer overflow).

[robertkirkman]

In my patch) I just replaced /tmp with a call to std::filesystem::temp_directory_path().string() which first tries env variables like TMPDIR one of which is set on Termux and works. If no such env var was set it'd default to /tmp

I'd like to make a note here that, technically, in Termux, we use and deploy upstream Android's official libc++_shared.so that Google provides with the official NDK, to provide the binary form of the C++ standard library and execute C++ code,

~ $ cat test.cpp
#include <iostream>
#include <filesystem>

int main() {
	std::cout << std::filesystem::temp_directory_path().string() << std::endl;
}
~ $ clang++ test.cpp -otest
~ $ strings $PREFIX/lib/libc++_shared.so | grep tmp
/data/local/tmp
~ $ ./test
/data/data/com.termux/files/usr/tmp
~ $ unset TMPDIR
~ $ ./test
/data/local/tmp
~ $ 

Since we don't actually patch or recompile this (at least currently), if TMPDIR is unset, the method std::filesystem::temp_directory_path() actually returns /data/local/tmp (not /tmp), and that is because /data/local/tmp is the TMPDIR for ADB sessions, which run as the ADB user, which has different permissions from the Termux app user.

I guess that if there is someone who wants to have the option of using Termux boost in the ADB shell as the ADB user, then I would want to call that function - however, there is one person I've seen using Termux packages as ADB user, and there are basically a lot of other problems with many packages as ADB user, since most Termux packages that patch /tmp do not fall back to /data/local/tmp, so they wouldn't work fully as the ADB user either.

Generally, I think it makes the most sense to assume that Termux packages will be installed in a writable folder $TERMUX_PREFIX, whether that is /data/data/com.termux/files/usr, /data/data/com.alternative.location.termux/files/usr, or something else that is in a regular app installation location under /data/data and uses the subdir files/usr, because unfortunately trying to accommodate more edge cases beyond that can result in various problems with other packages. For one recent example:

• [Bug]: clang package hardcoded dir #25787

[robertkirkman]

Ok, assuming that this PR fixes the problem sufficiently for your PR supercollider to have -DSYSTEM_BOOST=ON instead of -DSYSTEM_BOOST=OFF, I will merge this 24 hours from now and then you can rebase your PR on master after that in order to import this to your PR.

[robertkirkman]

There exists some sort of plugin to SuperCollider which requires the same version of boost to be used. I'm not really sure if this is a reason I haven't really looked into it, but I think it's the reason they bundle it in the first place.

In my opinion, just from what I would guess, probably the plugin will work if it's recompiled targeting the Termux versions of everything,

and if it wasn't recompiled, it would probably only work on GNU/Linux, Windows etc. if it was only tested on official builds of SuperCollider that already exist.

but if you realize that ends up being a problem, please let me know anything else you find out about it.

[rene-descartes2021]

Ok, assuming that this PR fixes the problem sufficiently for your PR supercollider to have -DSYSTEM_BOOST=ON instead of -DSYSTEM_BOOST=OFF, I will merge this 24 hours from now and then you can rebase your PR on master after that in order to import this to your PR.

Tried to build this PR but it wouldn't let me:
ERROR: Package 'boost' is not safe for on-device builds.

Oh I'm sure it'll work, simple change which already worked with the patch I did. This PR is more robust on top of that.

There exists some sort of plugin to SuperCollider which requires the same version of boost to be used. I'm not really sure if this is a reason I haven't really looked into it, but I think it's the reason they bundle it in the first place.

On the bundled boost problem a SuperCollider dev read my concern there and said it wasn't a problem and not to worry about it, using Termux's boost will be fine. My paraphrase of his words.

Thank you for the fix!

[robertkirkman]

@rene-descartes2021

ERROR: Package 'boost' is not safe for on-device builds.

There have been a series of people who made decisions about which packages are safe to build on-device before me, but unfortunately, my personal methodology for determining whether a package is safe to build on-device or not does not line up with the methodology that others used for many packages that have existed for a long time.

The result has been:

• Unfortunately, sometimes I have accidentally corrupted my Termux installation (under the criteria I use to determine the presence of data corruption) by attempting to build packages that I would have personally marked as unsafe for on-device build, but which others did not feel it was necessary to at the time when they wrote the build.sh files, and in those cases, I have made PRs to either mark them as unsafe for on-device build, or clean them up to make them newly safe for on-device builds, so that doesn't happen to me again.
• And unfortunately, vice versa has also happened - I have found some of the preexisting checks that mark other packages as unsafe for on-device building, to have used criteria for marking them so that don't meet the threshold at which I personally would calculate that building the package on-device would measurably corrupt the Termux installation, which is inconvenient because it can prevent building packages on-device that, in my opinion, should be considered safe to build on-device.

Since the exact methodology others use has never been clearly documented beyond "it is unsafe to build on-device if it removes files in $TERMUX_PREFIX" (which I believe is a flawed methodology if not amended by additional conditions, and could explain why I believe that if asked), I am going to use this situation as an opportunity to apply my own methodology to the on-device build safety status of boost, re-evaluate its status, apply the new status based on my evaluation, and document the whole process,

so that hopefully if others read this and either agree, or disagree, or have additional criteria to add, we can have a conversation about what the correct methodology for determining whether to mark a package as unsafe for on-device building should be.

Below is the full process I have used to re-evaluate the on-device build safety status of boost using my own preferred methodology.

pkg upgrade
pkg install apt-file
apt-file update
# data point 1: gather proof through apt-file search that all files removed
# during build.sh are ONLY provided by the package being built
# or one of its subpackages.
apt-file search 'lib/libboost' | grep -v glibc
apt-file search 'include/boost' | grep -v glibc
pkg install boost-headers boost-static boost # install package and all subpackages
pkg install python # install all build dependencies of the package
git clone https://github.com/termux/termux-packages.git
cd termux-packages
# temporarily exclude folders named '.git' from the build-package.sh system
git apply -v << 'EOF'
--- a/scripts/build/termux_step_copy_into_massagedir.sh
+++ b/scripts/build/termux_step_copy_into_massagedir.sh
@@ -2,6 +2,6 @@ termux_step_copy_into_massagedir() {
 	local DEST="$TERMUX_PKG_MASSAGEDIR/$TERMUX_PREFIX_CLASSICAL"
 	mkdir -p "$DEST"
 	# Copy files changed during the build into massagedir in order to massage them
-	tar -C "$TERMUX_PREFIX_CLASSICAL" -N "$TERMUX_BUILD_TS_FILE" --exclude='tmp' -cf - . | \
+	tar -C "$TERMUX_PREFIX_CLASSICAL" -N "$TERMUX_BUILD_TS_FILE" --exclude='tmp' --exclude='.git' -cf - . | \
 		tar -C "$DEST" -xf -
 }
EOF
# remove the on-device build safety condition to perform the test
git apply -v << 'EOF'
--- a/packages/boost/build.sh
+++ b/packages/boost/build.sh
@@ -16,14 +16,6 @@ TERMUX_PKG_BREAKS="libboost-python (<= 1.65.1-2), boost-dev"
 TERMUX_PKG_REPLACES="libboost-python (<= 1.65.1-2), boost-dev"
 TERMUX_PKG_BUILD_IN_SRC=true
 
-termux_step_pre_configure() {
-	# Certain packages are not safe to build on device because their
-	# build.sh script deletes specific files in $TERMUX_PREFIX.
-	if $TERMUX_ON_DEVICE_BUILD; then
-		termux_error_exit "Package '$TERMUX_PKG_NAME' is not safe for on-device builds."
-	fi
-}
-
 termux_step_make_install() {
 	CXXFLAGS+=" -std=c++14"
 
EOF
scripts/setup-termux.sh
cd $PREFIX
git init
git add .
git commit -m "tmp"
cd -
./build-package.sh -I -f boost
cd $PREFIX
# data point 2: gather real-world proof through manipulation of the capabilities
# of the `git` command that actually building the package on-device does NOT
# modify any files that are not provided by the package being built or one of
# its subpackages.
git status
# clean up
rm -rf $PREFIX/.git

Annotated Results:

.../files/usr $ git status
Refresh index: 100% (280649/280649), done.
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   include/boost/interprocess/detail/os_file_functions.hpp # boost-headers
	modified:   include/boost/interprocess/detail/shared_dir_helpers.hpp # boost-headers
	modified:   lib/libboost_atomic.a # boost-static
	modified:   lib/libboost_atomic.so # boost
	modified:   lib/libboost_charconv.a # boost-static
	modified:   lib/libboost_charconv.so # boost
	modified:   lib/libboost_chrono.a # boost-static
	modified:   lib/libboost_chrono.so # boost
	modified:   lib/libboost_container.a # boost-static
	modified:   lib/libboost_container.so # boost
	modified:   lib/libboost_context.a # boost-static
	modified:   lib/libboost_context.so # boost
	modified:   lib/libboost_contract.a # boost-static
	modified:   lib/libboost_contract.so # boost
	modified:   lib/libboost_coroutine.a # boost-static
	modified:   lib/libboost_coroutine.so # boost
	modified:   lib/libboost_date_time.a # boost-static
	modified:   lib/libboost_date_time.so # boost
	modified:   lib/libboost_exception.a # boost-static
	modified:   lib/libboost_fiber.a # boost-static
	modified:   lib/libboost_fiber.so # boost
	modified:   lib/libboost_filesystem.a # boost-static
	modified:   lib/libboost_filesystem.so # boost
	modified:   lib/libboost_graph.a # boost-static
	modified:   lib/libboost_graph.so # boost
	modified:   lib/libboost_iostreams.a # boost-static
	modified:   lib/libboost_iostreams.so # boost
	modified:   lib/libboost_json.a # boost-static
	modified:   lib/libboost_json.so # boost
	modified:   lib/libboost_locale.a # boost-static
	modified:   lib/libboost_locale.so # boost
	modified:   lib/libboost_log.a # boost-static
	modified:   lib/libboost_log.so # boost
	modified:   lib/libboost_log_setup.a # boost-static
	modified:   lib/libboost_log_setup.so # boost
	modified:   lib/libboost_math_c99.a # boost-static
	modified:   lib/libboost_math_c99.so # boost
	modified:   lib/libboost_math_c99f.a # boost-static
	modified:   lib/libboost_math_c99f.so # boost
	modified:   lib/libboost_math_c99l.a # boost-static
	modified:   lib/libboost_math_c99l.so # boost
	modified:   lib/libboost_math_tr1.a # boost-static
	modified:   lib/libboost_math_tr1.so # boost
	modified:   lib/libboost_math_tr1f.a # boost-static
	modified:   lib/libboost_math_tr1f.so # boost
	modified:   lib/libboost_math_tr1l.a # boost-static
	modified:   lib/libboost_math_tr1l.so # boost
	modified:   lib/libboost_nowide.a # boost-static
	modified:   lib/libboost_nowide.so # boost
	modified:   lib/libboost_prg_exec_monitor.a # boost-static
	modified:   lib/libboost_prg_exec_monitor.so # boost
	modified:   lib/libboost_process.a # boost-static
	modified:   lib/libboost_process.so # boost
	modified:   lib/libboost_program_options.a # boost-static
	modified:   lib/libboost_program_options.so # boost
	modified:   lib/libboost_python312.a # boost-static
	modified:   lib/libboost_python312.so # boost
	modified:   lib/libboost_random.a # boost-static
	modified:   lib/libboost_random.so # boost
	modified:   lib/libboost_regex.a # boost-static
	modified:   lib/libboost_regex.so # boost
	modified:   lib/libboost_serialization.a # boost-static
	modified:   lib/libboost_serialization.so # boost
	modified:   lib/libboost_system.a # boost-static
	modified:   lib/libboost_system.so # boost
	modified:   lib/libboost_test_exec_monitor.a # boost-static
	modified:   lib/libboost_thread.a # boost-static
	modified:   lib/libboost_thread.so # boost
	modified:   lib/libboost_timer.a # boost-static
	modified:   lib/libboost_timer.so # boost
	modified:   lib/libboost_type_erasure.a # boost-static
	modified:   lib/libboost_type_erasure.so # boost
	modified:   lib/libboost_unit_test_framework.a # boost-static
	modified:   lib/libboost_unit_test_framework.so
	modified:   lib/libboost_url.a # boost-static
	modified:   lib/libboost_url.so # boost
	modified:   lib/libboost_wave.a # boost-static
	modified:   lib/libboost_wave.so # boost
	modified:   lib/libboost_wserialization.a # boost-static
	modified:   lib/libboost_wserialization.so # boost
	modified:   lib/python3.12/__pycache__/platform.cpython-312.pyc # will be handled by PR #23652

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	lib/cmake/boost_numpy-1.87.0/libboost_numpy-variant-shared-py3.12.cmake # prefix pollution (safe) (ask me if you need an explanation what this means)
	lib/cmake/boost_numpy-1.87.0/libboost_numpy-variant-static-py3.12.cmake # prefix pollution (safe) (ask me if you need an explanation what this means)
	lib/libboost_numpy312.a # prefix pollution (safe) (ask me if you need an explanation what this means)
	lib/libboost_numpy312.so # prefix pollution (safe) (ask me if you need an explanation what this means)
	tmp/build-package-call-building-packages-list-2025-08-27-17.41.16.284 # tmp files are temporary
	tmp/build-package-call-built-packages-list-2025-08-27-17.41.16.180 # tmp files are temporary

no changes added to commit (use "git add" and/or "git commit -a")

Based on my evaluation shown above, I have determined that according to my personal methodology, boost should be considered safe for on-device building, so I have added an additional commit into this PR that enables on-device building of boost, so you should now be able to test what you were trying to by pulling the changes!

(you don't need to run any of the extra commands I sent in the above explanation ^^^ those are just my recorded proof of how people checking my work can calculate that boost is safe for on-device builds the same way I have. You should be able to just try what you were already trying again, with the newer version of this PR)

[rene-descartes2021]

Awesome! I'll be away from internet till Saturday so don't wait on me.

Using git to track things like that is clever. I did notice that building on device appeared to not use package management. I wanted to point out the checkinstall utility if you hadn't heard of it. It was removed from Debian repos for being unmaintained but there were patches I found somewhere, mailing list or forum (dunno), and was able to keep using it in future Debian releases.
Was an awesome utility to make a .deb from make install of whatnot. Well gotta go thanks again.

[robertkirkman]

I did notice that building on device appeared to not use package management.

checkinstall

It's a bit complicated to explain, but yes, you get the idea of how Termux packages work!

some details from my perspective:

• Termux is currently the most popular existing distro that still uses a timestamp-file-based packaging engine instead of a fakeroot-based packaging engine. I did some research a while ago and as far as I know, there are no others that still do.
  • This means that Termux should be polished to serve as a good example of how that kind of distro should be implemented, and given the best-possible implementations of the necessary ugly workarounds and kludges that such a distro needs to remain viable in the present day, both so that it can serve as a good example of what someone else who needs a distro like that for personal reasons should do to implement one, and also so that it can serve as an example of what not to do for people who are trying to learn the reasons why all other major distros use fakeroot instead.
  • In PRs like chore: optimize termux_step_get_dependencies #24567 (done) and tree-wide: port debpython to termux  #23652 (not done yet), I hope to gradually improve Termux's timestamp-file-based packaging engine to serve the purposes described above
  • checkinstall would be Debian's old, abandoned implementation of a timestamp-file-based packaging engine. Like I mentioned, some other distros have also had one at some time or another, but every single one has eventually been replaced or abandoned except Termux's.
    • This means that there is already code in Termux that currently implements pretty much the same thing, but in a maintained way that is already integrated with and adapted to the rest of Termux's code
  • Also, actually, it is not only on-device building that uses the timestamp-file-based packaging engine. cross-compiling using the Ubuntu termux-package-builder container also does use the same exact code to monitor the build and calculate packaging metadata. The major difference is that typically, the ubuntu package builder does not need to run the packages most of the time, and instead it only needs to build the packages, so it's possible to use workarounds for various inconvenient problems during cross-compilation of some packages, that involve deleting files, which wouldn't be safe to delete in a real Termux installation.

emacs

Unfortunately, the reason why emacs is considered unsafe for on-device builds is pretty much valid and not easily avoidable. It's because the build.sh of emacs in Termux needs to use the command rm -Rf "$TERMUX_PREFIX/share/info", which does unfortunately delete a lot of files that are owned by other packages, if they're installed.

termux-packages/packages/emacs/build.sh

Line 103 in a5b6f7c

rm -Rf "$TERMUX_PREFIX/share/info"

In order to make emacs safe for on-device builds in Termux, it would be necessary to work on that first and figure out if there's a way to build emacs for Termux that doesn't require removing that entire folder during the build.