Skip to content

ditaa: convert to pname + version; cleanup#10

Closed
SuperSandro2000 wants to merge 1 commit intomasterfrom
SuperSandro2000-patch-5
Closed

ditaa: convert to pname + version; cleanup#10
SuperSandro2000 wants to merge 1 commit intomasterfrom
SuperSandro2000-patch-5

Conversation

@SuperSandro2000
Copy link
Owner

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • 21.11 Release Notes (or backporting 21.05 Relase notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

SuperSandro2000 pushed a commit that referenced this pull request Apr 25, 2022
The following error occurs when using `imagemagickBig`:

    $ ./result/bin/identify sample.jp2
    [1]    699089 IOT instruction (core dumped)  ./result/bin/identify sample.jp2

When looking at the call-trace it seems as if certain symbols, e.g.
`opj_malloc` are mixed up:

    #8  0x00007f78c79ad2f5 in MagickSignalHandler.cold () from /nix/store/bqy80qiw6czqh7vsmmmivwdswp9zzjgl-imagemagick-7.1.0-29/lib/libMagickCore-7.Q16HDRI.so.10
    #9  <signal handler called>
    #10 0x00007f78c5a6095f in opj_malloc () from /nix/store/wg6ly83k1k1fjiygiv1jr7li3p6dwsvq-ghostscript-with-X-9.55.0/lib/libgs.so.9
    #11 0x00007f78c5a60981 in opj_calloc () from /nix/store/wg6ly83k1k1fjiygiv1jr7li3p6dwsvq-ghostscript-with-X-9.55.0/lib/libgs.so.9
    #12 0x00007f78c4f48e24 in opj_create_decompress () from /nix/store/qwalb0kjz1p9c4j48qkk6ql47ds2lnhh-openjpeg-2.4.0/lib/libopenjp2.so.7

The `opj_create_decompress()` is called from the `openjpeg`-integration
of `imagemagick` and thus shouldn't affect `ghostscript` at all.
However, `ghostscript` (`libgs.so` to be precise) also exposes e.g.
`opj_malloc`:

    $ objdump -t /nix/store/wg6ly83k1k1fjiygiv1jr7li3p6dwsvq-ghostscript-with-X-9.55.0/lib/libgs.so.9.55|grep opj_malloc
    0000000000205940 g     F .text	000000000000002b              opj_malloc

Because of that, two incompatible symbols are used in the same process
and thus the `identify`-call breaks because the wrong one is used. To
work around that I decided to use the system-wide openjpeg instead.
I'm not sure why `libgs.so` wants to expose these symbols anyways, but
with that workaround the problem is solved.

Even though it's mentioned that ghostscript's openjpeg is heavily
patched, I think that this is somewhat outdated or at least irrelevant
considering that both ArchLinux[1] and Fedora[2] use the system-wide
`openjpeg` instead.

[1] https://github.com/archlinux/svntogit-packages/blob/bafcb5473b59d5386dd110d1cb249372dce9ea6c/trunk/PKGBUILD#L50
[2] https://src.fedoraproject.org/rpms/ghostscript/blob/e4eec13ab6ace2bad64b740d352964bbf61d1aa7/f/ghostscript.spec#_245
wegank pushed a commit that referenced this pull request Aug 5, 2023
Pull in _FORTIFY_SOURCE=3 stack smashing fix. Without the change on
current `master` `rtorrent` crashes at start as:

*** buffer overflow detected ***: terminated
                                                                                        __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
44      pthread_kill.c: No such file or directory.
(gdb) bt
    #0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
    #1  0x00007ffff7880af3 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
    #2  0x00007ffff7831c86 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
    #3  0x00007ffff781b8ba in __GI_abort () at abort.c:79
    #4  0x00007ffff781c5f5 in __libc_message (fmt=fmt@entry=0x7ffff7992540 "*** %s ***: terminated\n") at ../sysdeps/posix/libc_fatal.c:150
    #5  0x00007ffff7910679 in __GI___fortify_fail (msg=msg@entry=0x7ffff79924e6 "buffer overflow detected") at fortify_fail.c:24
    #6  0x00007ffff790eea4 in __GI___chk_fail () at chk_fail.c:28
    #7  0x00007ffff790ea85 in ___snprintf_chk (s=<optimized out>, maxlen=<optimized out>, flag=<optimized out>, slen=<optimized out>, format=<optimized out>) at snprintf_chk.c:29
    #8  0x0000000000472acf in utils::Lockfile::try_lock() ()
    #9  0x000000000044b524 in core::DownloadStore::enable(bool) ()
    #10 0x00000000004b1f7b in Control::initialize() ()
    #11 0x000000000043000b in main ()
SuperSandro2000 pushed a commit that referenced this pull request Feb 29, 2024
Without the change `unnethack` startup crashes as:

    (gdb) bt
    #0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
    #1  0x00007f734250c0e3 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
    #2  0x00007f73424bce06 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
    #3  0x00007f73424a58f5 in __GI_abort () at abort.c:79
    #4  0x00007f73424a67a1 in __libc_message (fmt=fmt@entry=0x7f734261e2f8 "*** %s ***: terminated\n") at ../sysdeps/posix/libc_fatal.c:150
    #5  0x00007f734259b1d9 in __GI___fortify_fail (msg=msg@entry=0x7f734261e2df "buffer overflow detected") at fortify_fail.c:24
    #6  0x00007f734259ab94 in __GI___chk_fail () at chk_fail.c:28
    #7  0x00000000005b2ac5 in strcpy (__src=0x7ffe68838b00 "Shall I pick a character's race, role, gender and alignment for you? [YNTQ] (y)",
        __dest=0x7ffe68838990 "\001") at /nix/store/B0S2LKF593R3585038WS4JD3LYLF2WDX-glibc-2.38-44-dev/include/bits/string_fortified.h:79
    #8  curses_break_str (str=str@entry=0x7ffe68838b00 "Shall I pick a character's race, role, gender and alignment for you? [YNTQ] (y)", width=width@entry=163,
        line_num=line_num@entry=1) at ../win/curses/cursmisc.c:275
    #9  0x00000000005b3f51 in curses_character_input_dialog (prompt=prompt@entry=0x7ffe68838cf0 "Shall I pick a character's race, role, gender and alignment for you?",
        choices=choices@entry=0x7ffe68838d70 "YNTQ", def=def@entry=121) at ../win/curses/cursdial.c:211
    #10 0x00000000005b9ca0 in curses_choose_character () at ../win/curses/cursinit.c:556
    #11 0x0000000000404eb1 in main (argc=<optimized out>, argv=<optimized out>) at ./../sys/unix/unixmain.c:309

which corresponds to `gcc` warning:

    ../win/curses/cursmisc.c: In function 'curses_break_str':
    ../win/curses/cursmisc.c:275:5: warning: '__builtin___strcpy_chk' writing one too many bytes into a region of a size that depends on 'strlen' [-Wstringop-overflow=]
      275 |     strcpy(substr, str);
          |     ^

I did not find a single small upstream change that fixes it. Let's
disable `fortify3` until next release.

Closes: NixOS#292113
SuperSandro2000 pushed a commit that referenced this pull request Jul 30, 2024
This adds some extremely helpful and popular encoders in by default:
* openjpeg
* celt
* libwebp
* libaom

On the `master` branch, closure size for ffmpeg-headless went up 18.5 MiB.
```
$ nix store diff-closures nixpkgs#ffmpeg-headless^bin .#ffmpeg-headless^bin
celt: ∅ → 0.11.3, +168.4 KiB
ffmpeg-headless: +70.0 KiB
giflib: ∅ → 5.2.2, +398.7 KiB
lcms2: ∅ → 2.16, +466.2 KiB
lerc: ∅ → 4.0.0, +840.2 KiB
libaom: ∅ → 3.9.0, +8047.8 KiB
libdeflate: ∅ → 1.20, +427.0 KiB
libtiff: ∅ → 4.6.0, +655.9 KiB
libvmaf: ∅ → 3.0.0, +2665.0 KiB
libwebp: ∅ → 1.4.0, +2559.7 KiB
openjpeg: ∅ → 2.5.2, +1525.1 KiB
zstd: ∅ → 1.5.6, +1158.0 KiB

$ nvd diff $(nix build nixpkgs#ffmpeg-headless^bin --print-out-paths --no-link) $(nix build .#ffmpeg-headless^bin --print-out-paths --no-link)
<<< /nix/store/4n60lnj3zkjpasd4c56bzhpx2m8lc1sx-ffmpeg-headless-6.1.1-bin
>>> /nix/store/884f487w5hac6rs94jq6hq5zqkxdv666-ffmpeg-headless-6.1.1-bin
Added packages:
[A.]  #1  celt        0.11.3
[A.]  #2  giflib      5.2.2
[A.]  #3  lcms2       2.16
[A.]  #4  lerc        4.0.0
[A.]  #5  libaom      3.9.0
[A.]  #6  libdeflate  1.20
[A.]  #7  libtiff     4.6.0
[A.]  #8  libvmaf     3.0.0
[A.]  #9  libwebp     1.4.0 x2
[A.]  #10  openjpeg    2.5.2
[A.]  #11  zstd        1.5.6
Closure size: 66 -> 78 (15 paths added, 3 paths removed, delta +12, disk usage +18.5MiB).
```
SuperSandro2000 pushed a commit that referenced this pull request Apr 3, 2025
fluent-bit 3.2.7, 3.2.8 and 3.2.9 are segfaulting when
used in combination with the systemd input. Lets
revert to 3.2.6 for now.

Upstream bug: fluent/fluent-bit#10139

Note that fluent-bit-3.2.7 fixes two high CVEs which we are now
reintroducing. However they are only exploitable if you are
using the OpenTelemetry input or the Prometheus Remote Write input.

OpenTelemetry input: [CVE-2024-50609](https://nvd.nist.gov/vuln/detail/CVE-2024-50609)
Prometheus Remote Write input: [CVE-2024-50608](https://nvd.nist.gov/vuln/detail/CVE-2024-50608)

The problem is as follows:

3.2.7 started vendoring a copy of `libzstd` in tree and statically
linking against it. Also, the fluent-bit binary exports the symbols
of static libraries it links against.

This is a problem because `libzstd` gets `dlopen()`ed by `libsystemd`
when enumerating the journal (as journal logs are zstd compressed). and `libzstd` in Nixpkgs is built
with `-DZSTD_LEGACY_SUPPORT=0` which causes `struct ZSTD_DCtx` to be 16
bytes smaller than without this flag https://github.com/facebook/zstd/blob/dev/lib/decompress/zstd_decompress_internal.h#L183-L187

`libsystemd` calls [`sym_ZSTD_createDCtx()`](https://github.com/systemd/systemd/blob/1e79a2923364b65fc9f347884dd5b9b2087f6e32/src/basic/compress.c#L480)
which calls the function pointer returned by `dlsym()` which is calling into
the `libzstd` that comes with `nixpkgs` and thus allocates a struct that is 16 bytes smaller.

Later then `sym_ZSTD_freeDCtx()` is called. However because fluent-bit
has `zstd` in its global symbol table, any functions that `sym_ZSTD_freeDCtx()`
calls will be calls to the functions in the vendored fluent-bit version of the library
which expects the larger struct. This then causes enough heap corruption to cause
a segfault.

E.g. the subsequent calls to `ZSTD_clearDict(dctx)` and `ZSTD_customFree(dctx->inBuff)`
in https://github.com/facebook/zstd/blob/dev/lib/decompress/zstd_decompress.c#L324
will be working on a struct that is 16 bytes smaller than the one that was allocated
by `libsystemd` and will cause a segfault at some point and thus are probably modifying
pieces of memory that they shouldn't

	(gdb) bt
	#0  0x00007f10e7e9916c in __pthread_kill_implementation () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6
	#1  0x00007f10e7e40e86 in raise () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6
	#2  0x00007f10e7e2893a in abort () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6
	#3  0x000000000046a938 in flb_signal_handler ()
	#4  <signal handler called>
	#5  0x00007f10e7ea42b7 in unlink_chunk.isra () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6
	#6  0x00007f10e7ea45cd in _int_free_create_chunk () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6
	#7  0x00007f10e7ea5a1c in _int_free_merge_chunk () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6
	#8  0x00007f10e7ea5dc9 in _int_free () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6
	#9  0x00007f10e7ea8613 in free () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6
	#10 0x00007f10e80ad3b5 in ZSTD_freeDCtx () from /nix/store/wy0slah6yvchgra8nhp6vgrqa6ay72cq-zstd-1.5.6/lib/libzstd.so.1
	#11 0x00007f10e8c90f6b in decompress_blob_zstd () from /nix/store/b2cfj7yk3wfg1jdwjzim7306hvsc5gnl-systemd-257.3/lib/libsystemd.so.0
	#12 0x00007f10e8bf0efe in journal_file_data_payload () from /nix/store/b2cfj7yk3wfg1jdwjzim7306hvsc5gnl-systemd-257.3/lib/libsystemd.so.0
	#13 0x00007f10e8c00f74 in sd_journal_enumerate_data () from /nix/store/b2cfj7yk3wfg1jdwjzim7306hvsc5gnl-systemd-257.3/lib/libsystemd.so.0
	#14 0x00000000004eae2f in in_systemd_collect ()
	#15 0x00000000004eb5a0 in in_systemd_collect_archive ()
	#16 0x000000000047aa18 in flb_input_collector_fd ()
	#17 0x0000000000495223 in flb_engine_start ()
	#18 0x000000000046f304 in flb_lib_worker ()
	#19 0x00007f10e7e972e3 in start_thread () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6
	#20 0x00007f10e7f1b2fc in __clone3 () from /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6

Reverts 7310ab3
Reverts 4fbc6cf
SuperSandro2000 pushed a commit that referenced this pull request Aug 12, 2025
Change the message in the wait screen from "This may take up to 20
seconds" to "This may take up to two minutes". That's what Mobian needs,
and with "up to" in the sentence, it's not wrong, even if the actual
time is much shorter.

Creating a config option was considered (#10), but would only add
technical debt since the wait screen is already a hack (as described in
the big comment in runPartitionJobThenLeave()).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant