Heap-buffer-overflow in scale.c:214

[chameleon10712]

Description

Heap-buffer-overflow in scale.c:214 scale_without_resampling() (SEGV)

Case 1

Normal build

$ /home/oceane/libsixel_norm/libsixel/build/bin/img2sixel -r nearest  -h 3 ./poc_min
Segmentation fault

with ASan

$  /home/oceane/libsixel_asan/build_asan/bin/img2sixel -r nearest  -h 3 ./poc_min
AddressSanitizer:DEADLYSIGNAL
=================================================================
==2489639==ERROR: AddressSanitizer: SEGV on unknown address 0x7fc5ba7376cb (pc 0x7fc5bebe78da bp 0x000000000000 sp 0x7ffe4e0c2430 T0)
==2489639==The signal is caused by a READ memory access.
    #0 0x7fc5bebe78d9 in scale_without_resampling /home/oceane/libsixel_asan/src/scale.c:214
    #1 0x7fc5bebe78d9 in sixel_helper_scale_image /home/oceane/libsixel_asan/src/scale.c:348
    #2 0x7fc5bebe1db9 in sixel_frame_resize /home/oceane/libsixel_asan/src/frame.c:570
    #3 0x7fc5bec9e18d in sixel_encoder_do_resize /home/oceane/libsixel_asan/src/encoder.c:641
    #4 0x7fc5bec9fd94 in sixel_encoder_encode_frame /home/oceane/libsixel_asan/src/encoder.c:968
    #5 0x7fc5bec86f6a in load_with_builtin /home/oceane/libsixel_asan/src/loader.c:963
    #6 0x7fc5bec90e67 in sixel_helper_load_image_file /home/oceane/libsixel_asan/src/loader.c:1418
    #7 0x7fc5becadde1 in sixel_encoder_encode /home/oceane/libsixel_asan/src/encoder.c:1743
    #8 0x558352bf5dcb in main /home/oceane/libsixel_asan/converters/img2sixel.c:457
    #9 0x7fc5be795082 in __libc_start_main ../csu/libc-start.c:308
    #10 0x558352bf67dd in _start (/home/oceane/libsixel_asan/build_asan/bin/img2sixel+0x67dd)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/oceane/libsixel_asan/src/scale.c:214 in scale_without_resampling
==2489639==ABORTING

Case 2

Normal build

$ /home/oceane/libsixel_norm/libsixel/build/bin/img2sixel -w 40% -h 300% -r nearest poc_min2
Segmentation fault

with ASan

$ /home/oceane/libsixel_asan/build_asan/bin/img2sixel -w 40% -h 300% -r nearest poc_min2
=================================================================
==2012689==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f0b6a674610 at pc 0x7f0b702d6e05 bp 0x7ffdf2689970 sp 0x7ffdf2689960
READ of size 1 at 0x7f0b6a674610 thread T0
    #0 0x7f0b702d6e04 in scale_without_resampling /home/oceane/libsixel_asan/src/scale.c:214
    #1 0x7f0b702d6e04 in sixel_helper_scale_image /home/oceane/libsixel_asan/src/scale.c:348
    #2 0x7f0b702c5db9 in sixel_frame_resize /home/oceane/libsixel_asan/src/frame.c:570
    #3 0x7f0b7038218d in sixel_encoder_do_resize /home/oceane/libsixel_asan/src/encoder.c:641
    #4 0x7f0b70383d94 in sixel_encoder_encode_frame /home/oceane/libsixel_asan/src/encoder.c:968
    #5 0x7f0b7036af6a in load_with_builtin /home/oceane/libsixel_asan/src/loader.c:963
    #6 0x7f0b70374e67 in sixel_helper_load_image_file /home/oceane/libsixel_asan/src/loader.c:1418
    #7 0x7f0b70391de1 in sixel_encoder_encode /home/oceane/libsixel_asan/src/encoder.c:1743
    #8 0x55df72adadcb in main /home/oceane/libsixel_asan/converters/img2sixel.c:457
    #9 0x7f0b6fe79082 in __libc_start_main ../csu/libc-start.c:308
    #10 0x55df72adb7dd in _start (/home/oceane/libsixel_asan/build_asan/bin/img2sixel+0x67dd)

0x7f0b6a674610 is located 0 bytes to the right of 10952208-byte region [0x7f0b69c02800,0x7f0b6a674610)
allocated by thread T0 here:
    #0 0x7f0b70505808 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0x7f0b702c5bdc in sixel_frame_resize /home/oceane/libsixel_asan/src/frame.c:562

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/oceane/libsixel_asan/src/scale.c:214 in scale_without_resampling
Shadow bytes around the buggy address:
  0x0fe1ed4c6870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe1ed4c6880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe1ed4c6890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe1ed4c68a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0fe1ed4c68b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0fe1ed4c68c0: 00 00[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0fe1ed4c68d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0fe1ed4c68e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0fe1ed4c68f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0fe1ed4c6900: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0fe1ed4c6910: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==2012689==ABORTING

pocs.zip

Environment

git commit 6a5be8b
Ubuntu 20.04.6 LTS
13th Gen Intel(R) Core(TM) i9-13900

[t-bltg]

See #154.

[j4james]

This bug is caused by an integer overflow when calculating the scaled coordinates. The values go negative and it ends up trying to access an out-of-bounds index. It should be fixed by a simple cast like this:

diff --git a/src/scale.c b/src/scale.c
index 4767f80..35d87e0 100644
--- a/src/scale.c
+++ b/src/scale.c
@@ -204,8 +204,8 @@ scale_without_resampling(
 
     for (h = 0; h < dsth; h++) {
         for (w = 0; w < dstw; w++) {
-            x = w * srcw / dstw;
-            y = h * srch / dsth;
+            x = (long)w * srcw / dstw;
+            y = (long)h * srch / dsth;
             for (i = 0; i < depth; i++) {
                 pos = (y * srcw + x) * depth + i;
                 dst[(h * dstw + w) * depth + i] = src[pos];

[Kreijstal]

So many lonely patches, surely a PR would be better?

[j4james]

A PR where? It's pointless creating a PR here, since it won't be merged, and the patches are relative to the libsixel/libsixel fork anyway. And I can't create a PR in libsixel/libsixel because that repo is now archived. I'm also not willing to start filing PRs on random forks that may or may not be maintained. But if someone does decide they want to take on the job of maintaining libsixel on an ongoing basis, they're welcome to apply these patches to their fork. If that's too much effort for them, they're likely not serious about being a maintainer anyway.

[j4james]

FTR, this is a duplicate of libsixel#71.