Baike.dev
Log in
Back to tool/Back to issues
#4931·libvips

Remaining UBSan reproducers for int format images

Author: kleisaukeCreated Mar 5, 2026Updated Aug 25, 2026
Labelsbug

This issue tracks the remaining UBSan reproducers for images tagged as VIPS_FORMAT_INT / VIPS_FORMAT_UINT, similar to the one fixed in PR #4922 (and related).

Reproducer - conva
bash
$ printf 'P5\n2 2\n65536\n' > max_int_2x2.ppm
$ printf '\x7F\xFF\xFF\xFF%.0s' {1..4} >> max_int_2x2.ppm
$ vips cast max_int_2x2.ppm max_int_2x2.v int
$ vips conva max_int_2x2.v x.v max_int_2x2.v
../libvips/convolution/conva.c:959:5: runtime error: signed integer overflow: -3 - 2147483647 cannot be represented in type 'int'
    #0 0x7ff5fa54d082 in vips_conva_hgenerate /home/kleisauke/libvips/build/../libvips/convolution/conva.c:959:5
    #1 0x7ff5fa967fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #2 0x7ff5fa949ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #3 0x7ff5fa96779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #4 0x7ff5fa55b3e0 in vips_conva_vgenerate /home/kleisauke/libvips/build/../libvips/convolution/conva.c:1143:6
    #5 0x7ff5fa967fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #6 0x7ff5fa949ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #7 0x7ff5fa96779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #8 0x7ff5fa8b89e7 in vips_image_write_gen /home/kleisauke/libvips/build/../libvips/iofuncs/image.c:2600:6
    #9 0x7ff5fa967fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #10 0x7ff5fa949ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #11 0x7ff5fa96779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #12 0x7ff5fa380767 in vips_copy_gen /home/kleisauke/libvips/build/../libvips/conversion/copy.c:140:6
    #13 0x7ff5fa967fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #14 0x7ff5fa949ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #15 0x7ff5fa96779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #16 0x7ff5fa8b89e7 in vips_image_write_gen /home/kleisauke/libvips/build/../libvips/iofuncs/image.c:2600:6
    #17 0x7ff5fa967fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #18 0x7ff5fa96afcb in vips_region_prepare_to_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1737:6
    #19 0x7ff5fa969c1c in vips_region_prepare_to /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1854:7
    #20 0x7ff5fa902eca in wbuffer_work_fn /home/kleisauke/libvips/build/../libvips/iofuncs/sinkdisc.c:438:11
    #21 0x7ff5fa82f5ec in vips_worker_work_unit /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:368:6
    #22 0x7ff5fa82ddf0 in vips_thread_main_loop /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:393:3
    #23 0x7ff5fa828c2e in vips_threadset_work /home/kleisauke/libvips/build/../libvips/iofuncs/threadset.c:187:3
    #24 0x7ff5fa823ba4 in vips_thread_run /home/kleisauke/libvips/build/../libvips/iofuncs/thread.c:108:11
    #25 0x7ff5f991f741  (/lib64/libglib-2.0.so.0+0x75741) (BuildId: 2932f63ee7c53ae67d9b0b3ff877ece14c13edd5)
    #26 0x0000004a380a in asan_thread_start(void*) asan_interceptors.cpp.o
    #27 0x7ff5f9469463 in start_thread (/lib64/libc.so.6+0x72463) (BuildId: ff0267465bc3d76e21003b3bc5598fd5ee63e261)
    #28 0x7ff5f94ec5eb in __GI___clone3 (/lib64/libc.so.6+0xf55eb) (BuildId: ff0267465bc3d76e21003b3bc5598fd5ee63e261)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../libvips/convolution/conva.c:959:5 
Aborted                    vips conva max_int_2x2.v x.v max_int_2x2.v
Reproducer - convasep
bash
$ printf 'P5\n1 2\n65536\n' > max_int_1x2.ppm
$ printf '\x7F\xFF\xFF\xFF%.0s' {1..2} >> max_int_1x2.ppm
$ vips cast max_int_1x2.ppm max_int_1x2.v int
$ vips convasep max_int_1x2.v x.v max_int_1x2.v
../libvips/convolution/convasep.c:612:4: runtime error: signed integer overflow: 2147483647 + 2147483647 cannot be represented in type 'int'
    #0 0x7f8b6d22fe35 in vips_convasep_generate_horizontal /home/kleisauke/libvips/build/../libvips/convolution/convasep.c:612:4
    #1 0x7f8b6d567fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #2 0x7f8b6d549ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #3 0x7f8b6d56779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #4 0x7f8b6d23bbb1 in vips_convasep_generate_vertical /home/kleisauke/libvips/build/../libvips/convolution/convasep.c:745:6
    #5 0x7f8b6d567fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #6 0x7f8b6d549ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #7 0x7f8b6d56779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #8 0x7f8b6d4b89e7 in vips_image_write_gen /home/kleisauke/libvips/build/../libvips/iofuncs/image.c:2600:6
    #9 0x7f8b6d567fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #10 0x7f8b6d549ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #11 0x7f8b6d56779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #12 0x7f8b6cf80767 in vips_copy_gen /home/kleisauke/libvips/build/../libvips/conversion/copy.c:140:6
    #13 0x7f8b6d567fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #14 0x7f8b6d549ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #15 0x7f8b6d56779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #16 0x7f8b6d4b89e7 in vips_image_write_gen /home/kleisauke/libvips/build/../libvips/iofuncs/image.c:2600:6
    #17 0x7f8b6d567fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #18 0x7f8b6d56afcb in vips_region_prepare_to_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1737:6
    #19 0x7f8b6d569c1c in vips_region_prepare_to /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1854:7
    #20 0x7f8b6d502eca in wbuffer_work_fn /home/kleisauke/libvips/build/../libvips/iofuncs/sinkdisc.c:438:11
    #21 0x7f8b6d42f5ec in vips_worker_work_unit /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:368:6
    #22 0x7f8b6d42ddf0 in vips_thread_main_loop /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:393:3
    #23 0x7f8b6d428c2e in vips_threadset_work /home/kleisauke/libvips/build/../libvips/iofuncs/threadset.c:187:3
    #24 0x7f8b6d423ba4 in vips_thread_run /home/kleisauke/libvips/build/../libvips/iofuncs/thread.c:108:11
    #25 0x7f8b6c51f741  (/lib64/libglib-2.0.so.0+0x75741) (BuildId: 2932f63ee7c53ae67d9b0b3ff877ece14c13edd5)
    #26 0x0000004a380a in asan_thread_start(void*) asan_interceptors.cpp.o
    #27 0x7f8b6c07b463 in start_thread (/lib64/libc.so.6+0x72463) (BuildId: ff0267465bc3d76e21003b3bc5598fd5ee63e261)
    #28 0x7f8b6c0fe5eb in __GI___clone3 (/lib64/libc.so.6+0xf55eb) (BuildId: ff0267465bc3d76e21003b3bc5598fd5ee63e261)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../libvips/convolution/convasep.c:612:4 
Aborted                    vips convasep max_int_1x2.v x.v max_int_1x2.v
Reproducer - convi

Edit: Fixed via commit 94c4e15baebe479de42eebeb229772db22e87584.

bash
$ printf 'P5\n1 2\n65536\n' > max_uint_1x2.ppm
$ printf '\xFF\xFF\xFF\xFF%.0s' {1..2} >> max_uint_1x2.ppm
$ vips convi max_uint_1x2.ppm x.v max_uint_1x2.ppm
../libvips/convolution/convi.c:825:4: runtime error: signed integer overflow: -9223372034707292160 + -9223372034707292160 cannot be represented in type 'int64_t' (aka 'long')
    #0 0x7fe1e3657825 in vips_convi_gen /home/kleisauke/libvips/build/../libvips/convolution/convi.c:825:4
    #1 0x7fe1e39f477f in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1747:6
    #2 0x7fe1e39d00a3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #3 0x7fe1e39f3f41 in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1815:7
    #4 0x7fe1e33fd727 in vips_copy_gen /home/kleisauke/libvips/build/../libvips/conversion/copy.c:140:6
    #5 0x7fe1e39f477f in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1747:6
    #6 0x7fe1e39d00a3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #7 0x7fe1e39f3f41 in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1815:7
    #8 0x7fe1e393db27 in vips_image_write_gen /home/kleisauke/libvips/build/../libvips/iofuncs/image.c:2647:6
    #9 0x7fe1e39f477f in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1747:6
    #10 0x7fe1e39f776b in vips_region_prepare_to_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1870:6
    #11 0x7fe1e39f63b8 in vips_region_prepare_to /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1987:7
    #12 0x7fe1e3988d9a in wbuffer_work_fn /home/kleisauke/libvips/build/../libvips/iofuncs/sinkdisc.c:438:11
    #13 0x7fe1e38b468e in vips_worker_work_unit /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:362:6
    #14 0x7fe1e38b31bf in vips_thread_main_loop /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:388:12
    #15 0x7fe1e38ae48e in vips_threadset_work /home/kleisauke/libvips/build/../libvips/iofuncs/threadset.c:187:3
    #16 0x7fe1e38a93e4 in vips_thread_run /home/kleisauke/libvips/build/../libvips/iofuncs/thread.c:112:11
    #17 0x7fe1e291bc51  (/lib64/libglib-2.0.so.0+0x7dc51) (BuildId: a55397c41cae330fee603dd8a00cf8cd2ac399ca)
    #18 0x0000004a435a in asan_thread_start(void*) asan_interceptors.cpp.o
    #19 0x7fe1e24d2c18 in start_thread (/lib64/libc.so.6+0x72c18) (BuildId: 5bd941be836f538fe5e10eff508f7f5dd94905a6)
    #20 0x7fe1e25565cb in __GI___clone3 (/lib64/libc.so.6+0xf65cb) (BuildId: 5bd941be836f538fe5e10eff508f7f5dd94905a6)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../libvips/convolution/convi.c:825:4 
Aborted                    vips convi max_uint_1x2.ppm x.v max_uint_1x2.ppm
Reproducer - merge
bash
$ printf 'P5\n2 2\n65536\n' > max_int_2x2.ppm
$ printf '\x7F\xFF\xFF\xFF%.0s' {1..4} >> max_int_2x2.ppm
$ vips cast max_int_2x2.ppm max_int_2x2.v int
$ vips merge max_int_2x2.v max_int_2x2.v x.v horizontal 0 0
../libvips/mosaicing/lrmerge.c:645:4: runtime error: signed integer overflow: 4096 * 2147483647 cannot be represented in type 'int'
    #0 0x7fe9deda997b in lr_blend /home/kleisauke/libvips/build/../libvips/mosaicing/lrmerge.c:645:4
    #1 0x7fe9ded7c75f in vips__merge_gen /home/kleisauke/libvips/build/../libvips/mosaicing/lrmerge.c:1035:8
    #2 0x7fe9deb67fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #3 0x7fe9deb49ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #4 0x7fe9deb6779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #5 0x7fe9de580767 in vips_copy_gen /home/kleisauke/libvips/build/../libvips/conversion/copy.c:140:6
    #6 0x7fe9deb67fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #7 0x7fe9deb49ea3 in vips_region_fill /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:867:7
    #8 0x7fe9deb6779c in vips_region_prepare /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1682:7
    #9 0x7fe9deab89e7 in vips_image_write_gen /home/kleisauke/libvips/build/../libvips/iofuncs/image.c:2600:6
    #10 0x7fe9deb67fe2 in vips_region_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1614:6
    #11 0x7fe9deb6afcb in vips_region_prepare_to_generate /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1737:6
    #12 0x7fe9deb69c1c in vips_region_prepare_to /home/kleisauke/libvips/build/../libvips/iofuncs/region.c:1854:7
    #13 0x7fe9deb02eca in wbuffer_work_fn /home/kleisauke/libvips/build/../libvips/iofuncs/sinkdisc.c:438:11
    #14 0x7fe9dea2f5ec in vips_worker_work_unit /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:368:6
    #15 0x7fe9dea2ddf0 in vips_thread_main_loop /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:393:3
    #16 0x7fe9dea28c2e in vips_threadset_work /home/kleisauke/libvips/build/../libvips/iofuncs/threadset.c:187:3
    #17 0x7fe9dea23ba4 in vips_thread_run /home/kleisauke/libvips/build/../libvips/iofuncs/thread.c:108:11
    #18 0x7fe9ddb1f741  (/lib64/libglib-2.0.so.0+0x75741) (BuildId: 2932f63ee7c53ae67d9b0b3ff877ece14c13edd5)
    #19 0x0000004a380a in asan_thread_start(void*) asan_interceptors.cpp.o
    #20 0x7fe9dd706463 in start_thread (/lib64/libc.so.6+0x72463) (BuildId: ff0267465bc3d76e21003b3bc5598fd5ee63e261)
    #21 0x7fe9dd7895eb in __GI___clone3 (/lib64/libc.so.6+0xf55eb) (BuildId: ff0267465bc3d76e21003b3bc5598fd5ee63e261)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../libvips/mosaicing/lrmerge.c:645:4 
Aborted                    vips merge max_int_2x2.v max_int_2x2.v x.v horizontal 0 0
Reproducer - project
bash
$ printf 'P5\n1 2\n65536\n' > max_int_1x2.ppm
$ printf '\x7F\xFF\xFF\xFF%.0s' {1..2} >> max_int_1x2.ppm
$ vips cast max_int_1x2.ppm max_int_1x2.v int
$ vips project max_int_1x2.v sum1.v sum2.v
../libvips/arithmetic/project.c:238:3: runtime error: signed integer overflow: 2147483647 + 2147483647 cannot be represented in type 'int'
    #0 0x7fb6ffd27d9c in vips_project_scan /home/kleisauke/libvips/build/../libvips/arithmetic/project.c:238:3
    #1 0x7fb6ffda5d4d in vips_statistic_scan /home/kleisauke/libvips/build/../libvips/arithmetic/statistic.c:86:7
    #2 0x7fb7006f7400 in sink_work /home/kleisauke/libvips/build/../libvips/iofuncs/sink.c:417:12
    #3 0x7fb70062f5ec in vips_worker_work_unit /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:368:6
    #4 0x7fb70062ddf0 in vips_thread_main_loop /home/kleisauke/libvips/build/../libvips/iofuncs/threadpool.c:393:3
    #5 0x7fb700628c2e in vips_threadset_work /home/kleisauke/libvips/build/../libvips/iofuncs/threadset.c:187:3
    #6 0x7fb700623ba4 in vips_thread_run /home/kleisauke/libvips/build/../libvips/iofuncs/thread.c:108:11
    #7 0x7fb6ff71f741  (/lib64/libglib-2.0.so.0+0x75741) (BuildId: 2932f63ee7c53ae67d9b0b3ff877ece14c13edd5)
    #8 0x0000004a380a in asan_thread_start(void*) asan_interceptors.cpp.o
    #9 0x7fb6ff2da463 in start_thread (/lib64/libc.so.6+0x72463) (BuildId: ff0267465bc3d76e21003b3bc5598fd5ee63e261)
    #10 0x7fb6ff35d5eb in __GI___clone3 (/lib64/libc.so.6+0xf55eb) (BuildId: ff0267465bc3d76e21003b3bc5598fd5ee63e261)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../libvips/arithmetic/project.c:238:3 
Aborted                    vips project max_int_1x2.v sum1.v sum2.v

Usually, the fix is to use a 64-bit intermediate in these paths, but for these operations this is somewhat non-trivial.

Source: libvips/libvips

View original on GitHubView discussion on GitHub