Project

General

Profile

Bug #10883

dt_develop_blend_process sigsegv

Added by Paul Herbert about 4 years ago. Updated over 3 years ago.

Status:
Fixed: To be cherry-picked
Priority:
Low
Category:
Masks
Target version:
Start date:
01/18/2016
Due date:
% Done:

90%

Estimated time:
Affected Version:
2.0.0
System:
other GNU/Linux
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Hi,

I am running Darktable 2.0.0 on Kubuntu Linux in a VirtualBox guest environment on a Windows 10 64-bit host. The Linux guest has 8192MB base memory (RAM) allocated to it.

I opened a Konsole session in Kubuntu and entered the command to generate a cache of thumbnail images using -

darktable-generate-cache

All seemed OK initially but the process seemed to stall after just over 33% images had been generated. The last few lines of output on the console were -

image 4803/14385 (33.39%)
image 4804/14385 (33.40%)
image 4805/14385 (33.40%)
image 4806/14385 (33.41%)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f559447ce9c in __libc_waitpid (pid=pid@entry=2741, stat_loc=stat_loc@entry=0x0, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:31
31 ../sysdeps/unix/sysv/linux/waitpid.c: No such file or directory.
Traceback (most recent call last):
File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in <module>
from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'
backtrace written to /tmp/darktable_bt_L60IBY.txt
Segmentation fault (core dumped)
paul@paul-VB-DT-64:~$

I've attached backtrace file darktable_bt_L60IBY.txt

Do you have any ideas on what the problem is and how I can resolve it?

No other user applications were in use when I ran the darktable-generate-cache command.

Many thanks, Paul

darktable_bt_L60IBY.txt (27.2 KB) darktable_bt_L60IBY.txt Paul Herbert, 01/18/2016 04:19 PM
IMG_0398.CR2.xmp (7.91 KB) IMG_0398.CR2.xmp Paul Herbert, 01/21/2016 04:02 PM
IMG_0398_01.CR2.xmp (11.6 KB) IMG_0398_01.CR2.xmp Paul Herbert, 01/21/2016 04:02 PM
IMG_0398_02.CR2.xmp (10.4 KB) IMG_0398_02.CR2.xmp Paul Herbert, 01/21/2016 04:02 PM
IMG_0398.CR2 (21.9 MB) IMG_0398.CR2 Paul Herbert, 01/21/2016 04:07 PM

History

#1 Updated by Roman Lebedev about 4 years ago

  • Subject changed from darktable-generate-cache stops before completion to dt_develop_blend_process sigsegv

Seems like an blending issue.

#2 Updated by Ulrich Pegelow about 4 years ago

Really strange. In that code line we try to get the blend mask_mode from the blending parameter set:

const unsigned int mask_mode = d->mask_mode;

Before we checked that d is not NULL. Now, in your case 'd' is set to 0x3f800000 which certainly is no valid pointer. In fact that bit code is the float representation of 1.0f. No idea how this value can get there ...

#3 Updated by Ulrich Pegelow about 4 years ago

Question: does this crash happen reliably? According to the backtrace darktable fails in the thumbnail generation step with an image that has the ID 11980. If you can isolate the corresponding input image and attach it here together with the XMP we might be able to dig further into this.

#4 Updated by Paul Herbert about 4 years ago

Here is the original RAW file and the three .xmp files associated with it. Hope this helps.

#5 Updated by Ulrich Pegelow about 4 years ago

  • % Done changed from 0 to 10
  • Status changed from New to Confirmed

I kinda confirm that there is something wrong with the history stack of version 2 of your image (the third xmp file). In that file it's namely the settings of the spot removal module that causes problems.

No crash here on my side. However, I get clear indications of heap corruption with various malfunctionings of darktable as soon as I open that image version in the darkroom mode and as long as I let the spot removal settings in that xmp take action.

#6 Updated by Roman Lebedev about 4 years ago

  • Category changed from General to Masks

Happens with both IMG_0398_01.CR2.xmp and IMG_0398_02.CR2.xmp

=================================================================
==8238==ERROR: AddressSanitizer: heap-use-after-free on address 0x60b0006de880 at pc 0x7f21669e79a9 bp 0x7f2149f3d180 sp 0x7f2149f3d178
WRITE of size 4 at 0x60b0006de880 thread T11
    #0 0x7f21669e79a8 in dt_path_get_mask /home/lebedevri/darktable/src/develop/masks/path.c:2139
    #1 0x7f2166a336d3 in dt_masks_get_mask /home/lebedevri/darktable/src/develop/masks/masks.c:467
    #2 0x7f2136a7ed19 in process /home/lebedevri/darktable/src/iop/spots.c:480
    #3 0x7f216695f70f in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:1681
    #4 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #5 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #6 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #7 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #8 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #9 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #10 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #11 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #12 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #13 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #14 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #15 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #16 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #17 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #18 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #19 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #20 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #21 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #22 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #23 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #24 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #25 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #26 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #27 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #28 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #29 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #30 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #31 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #32 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #33 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #34 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #35 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #36 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #37 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #38 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #39 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #40 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #41 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #42 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #43 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #44 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #45 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #46 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #47 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #48 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #49 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #50 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #51 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #52 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #53 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #54 0x7f2166958a28 in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:737
    #55 0x7f216695a04c in dt_dev_pixelpipe_process_rec /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:869
    #56 0x7f2166968066 in dt_dev_pixelpipe_process_rec_and_backcopy /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:2298
    #57 0x7f2166968a55 in dt_dev_pixelpipe_process /home/lebedevri/darktable/src/develop/pixelpipe_hb.c:2386
    #58 0x7f2166913f3a in dt_dev_process_preview_job /home/lebedevri/darktable/src/develop/develop.c:255
    #59 0x7f216690c1d7 in dt_dev_process_preview_job_run /home/lebedevri/darktable/src/control/jobs/develop_jobs.c:25
    #60 0x7f21668fce37 in dt_control_run_job_res /home/lebedevri/darktable/src/control/jobs.c:204
    #61 0x7f21668fe370 in dt_control_work_res /home/lebedevri/darktable/src/control/jobs.c:486
    #62 0x7f21628c6283 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7283)
    #63 0x7f215ec6774c in clone (/lib/x86_64-linux-gnu/libc.so.6+0xe874c)

0x60b0006de880 is located 80 bytes inside of 112-byte region [0x60b0006de830,0x60b0006de8a0)
freed by thread T0 here:
    #0 0x7f2166e931ba in realloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x941ba)
    #1 0x7f2164145b5c in g_slice_alloc (/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x66b5c)

previously allocated by thread T0 here:
    #0 0x7f2166e931ba in realloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x941ba)
    #1 0x7f2164145b5c in g_slice_alloc (/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x66b5c)

Thread T11 created by T0 here:
    #0 0x7f2166e34ef4 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x35ef4)
    #1 0x7f21668feb13 in dt_control_jobs_init /home/lebedevri/darktable/src/control/jobs.c:567
    #2 0x7f21668f1857 in dt_control_init /home/lebedevri/darktable/src/control/control.c:119
    #3 0x7f2166808479 in dt_init /home/lebedevri/darktable/src/common/darktable.c:857
    #4 0x400cec in main /home/lebedevri/darktable/src/main.c:24
    #5 0x7f215eb9f86f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2086f)

SUMMARY: AddressSanitizer: heap-use-after-free /home/lebedevri/darktable/src/develop/masks/path.c:2139 dt_path_get_mask
Shadow bytes around the buggy address:
  0x0c16800d3cc0: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c16800d3cd0: fd fa fa fa fa fa fa fa fa fa fd fd fd fd fd fd
  0x0c16800d3ce0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
  0x0c16800d3cf0: fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa
  0x0c16800d3d00: fa fa fa fa fa fa fd fd fd fd fd fd fd fd fd fd
=>0x0c16800d3d10:[fd]fd fd fd fa fa fa fa fa fa fa fa fd fd fd fd
  0x0c16800d3d20: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa
  0x0c16800d3d30: fa fa fd fd fd fd fd fd fd fd fd fd fd fd fd fa
  0x0c16800d3d40: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c16800d3d50: fd fd fd fd fd fa fa fa fa fa fa fa fa fa fd fd
  0x0c16800d3d60: fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==8238==ABORTING

#7 Updated by Ulrich Pegelow about 4 years ago

  • Assignee set to Aldric Renaudin

@Aldric: could you please have a look. Seems like dt_path_get_mask() writes way out of the allocated region of its buffer. See heap analysis above.

#8 Updated by Ulrich Pegelow about 4 years ago

  • % Done changed from 10 to 50
  • Status changed from Confirmed to In Progress

I digged a bit into the topic. The reason seems to be obvious. There is a loop in path.c:dt_path_get_mask() that renders the path shape by going over all points of the path outline twice. One needs to take into account here that the first few entrys in points[] are reserved for the path nodes and must not be processed. That works perfect for the first run, and even for the second run those entries are skipped. However, the calculation of the length of points[] gets wrong then and we read bad data beyond the end of the array. The fix should be obvious.

Also as we write into a buffer with dimensions that are not known a priori we should make sure to use a size_t index into that buffer. Else we may face problems in cases where the dimensions of the buffer get really big.

@Aldric: if you want I can take care and fix the issue.

#9 Updated by Aldric Renaudin about 4 years ago

@Ulrich you beat me on that topic... I was currently looking at the code...
...and found the same pb : The loop bound should be reduced by the amount of reserved space.

As you have other change to make, I think it's better that you take care of this.

Thanks

#10 Updated by Ulrich Pegelow about 4 years ago

  • % Done changed from 50 to 90
  • Status changed from In Progress to Fixed: To be cherry-picked

Fixed in master.

Commit b4da143fab95ca435ab43dc1fe8b91590f20d189 should be cherry-picked into the 2.0.x branch.

#11 Updated by Roman Lebedev over 3 years ago

  • Target version set to 2.2.0

Also available in: Atom PDF

Go to top