Project

General

Profile

Bug #11386

SIGSEGV when deleting duplicated Equalizer module

Added by Christian Stussak over 2 years ago. Updated over 2 years ago.

Status:
Fixed
Priority:
High
Assignee:
-
Category:
Darkroom
Target version:
Start date:
12/25/2016
Due date:
% Done:

100%

Affected Version:
git master branch
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Steps to reproduce: Open darktable, go to darkroom mode, duplicate the equalizer module (doesn't even need to be enabled), delete the new instance

This results in

Magick: abort due to signal 11 (SIGSEGV) "Segmentation Fault"...
Abort trap: 6

I tried with a couple of other modules (3-4), but so far darktable only crashed when I used the equalizer module. It also crashes each time I try it, not just occasionally.

darktable.log Magnifier - darktable -d all >darktable.log 2>&1 (428 KB) Christian Stussak, 12/25/2016 01:59 PM

darktable-formula.rb Magnifier (1.55 KB) Roman Lebedev, 12/25/2016 10:01 PM

darktable.rb Magnifier - Homebrew formula for darktable 2.2.0 + debug symbols (2.11 KB) Christian Stussak, 12/25/2016 10:50 PM

gdb.txt Magnifier (63 KB) Christian Stussak, 12/25/2016 10:50 PM

darktable.rb Magnifier (2.57 KB) Christian Stussak, 12/30/2016 12:59 AM

Associated revisions

Revision 1e1f60ef
Added by Roman Lebedev over 2 years ago

Imageop: properly delete iop widget. Fixes #11386

We need to explicitly remove the widet to cause
cascade removal, that will disconnect all the signals.

Revision f85d97a7
Added by Roman Lebedev over 2 years ago

Imageop: properly delete iop widget. Fixes #11386

We need to explicitly remove the widet to cause
cascade removal, that will disconnect all the signals.

(cherry picked from commit 1e1f60ef8e9103c05a68f72e71a71842dafd501c)

History

#1 Updated by Roman Lebedev over 2 years ago

  • Status changed from New to Incomplete
  • % Done changed from 0 to 20

-d all output is useless. why does everyone attach it?
Is there a backtrace? please do attach the backtrace.

#2 Updated by Christian Stussak over 2 years ago

I attached the -d all log because it was the only thing I could do quickly. I am not very familiar with gdb nor is it easy to get this working on macOS X.

TLDR;

I tried to get a backtrace, but failed, because darktable (pid 1356) itself exited normally:

Magick: abort due to signal 11 (SIGSEGV) "Segmentation Fault"...
[Inferior 1 (process 1356) exited normally]
(gdb)
The program is not being run.
(gdb) bt
No stack.

Long version

I am on macOS 10.11.6. I installed gdb via homebrew. The procedure to get gdb working is not straight forward. The binary needs to be code-signed by a system trusted certificate. Finally, https://opensource.apple.com/source/lldb/lldb-69/docs/code-signing.txt worked (it is for lldb, but works the same for gdb). Now I got gdb working.

But

gdb darktable

does not work because darktable is not in the path when using the GitHub-provided .dmg package. I tried
gdb /Applications/darktable.app/Contents/MacOS/darktable

but it doesn't work either because this file is a shell script.

The next thing I tried was to attach gdb to the running process via

gdb -p `cat ~/.config/darktable/library.db.lock`

gdb seems to attach properly and stops (pauses) the process and I did

(gdb) continue

Now, in darktable, I go to the darkroom, duplicate and delete the equalizer module and it crashes.

But now,

(gdb) bt
No stack.

The only additional info gdb provides when darktable-bin (pid 1356) exits is
[Inferior 1 (process 1356) exited normally]

Also, the darktable macOS package provided on github doesn't seem to contain debug symbols. Probably, I would need to rebuild darktable myself. Something, I tried to avoid so far. Maybe you can integrate it into your build process and provide two packages for download, one with and one without debug symbols. This would make it much easier for regular users to supply backtraces (for this particular bug, it probably wouldn't change anything).

#3 Updated by Roman Lebedev over 2 years ago

Christian Stussak wrote:

Also, the darktable macOS package provided on github doesn't seem to contain debug symbols. Probably, I would need to rebuild darktable myself. Something, I tried to avoid so far. Maybe you can integrate it into your build process and provide two packages for download, one with and one without debug symbols. This would make it much easier for regular users to supply backtraces (for this particular bug, it probably wouldn't change anything).

Why is there so much issues with these dt dmg's i wonder? Packaging issue?
I wonder if we should get dt packaged in homebrew as formula...

#4 Updated by Christian Stussak over 2 years ago

I am right now investigating to build darktable myself via homebrew using the MacPorts build instructions (https://github.com/darktable-org/darktable/blob/master/packaging/macosx/BUILD.txt). I am already using homebrew a lot, but it doesn't play well with MacPorts installed at the same time ...

Providing a darktable-maintained homebrew formula would certainly make it easier for many people to provide better bug reports.

Nevertheless, I don't think that a homebrew build will help with the current issue. To my untrained eye it looks like it is related to GraphicsMagick somehow (due to the Magick SIGSEGV error message). Is GraphicsMagick called as a child process? Otherwise, I can't imagine how darktable exits normally ...

#5 Updated by Roman Lebedev over 2 years ago

Christian Stussak wrote:

I am right now investigating to build darktable myself via homebrew using the MacPorts build instructions (https://github.com/darktable-org/darktable/blob/master/packaging/macosx/BUILD.txt). I am already using homebrew a lot, but it doesn't play well with MacPorts installed at the same time ...

Or maybe this is just an usual issue due to weird library mix.

I have attached the formula. Basically, the only thing needed for homebrew to contain it is tests.
Not my authorship, https://github.com/ilovezfs

#6 Updated by Christian Stussak over 2 years ago

Thanks! The homebrew formula made it a lot easier to build darktable on macOS! I had to modify it a little to use darktable 2.2.0 (not the rc) and also keep the debug info (see attachment).

And finally, I am proudly able to present a backtrace! ;-) See attached gdb.txt.

It boils down to

...

Thread 1 received signal SIGSEGV, Segmentation fault.
0x0000000112ccc63a in area_leave_notify () from /usr/local/lib/darktable/plugins/libatrous.so

...

Thread 1 (Thread 0x1103 of process 79254):
#0  0x0000000112ccc63a in area_leave_notify () from /usr/local/lib/darktable/plugins/libatrous.so
No symbol table info available.
#1  0x00000001004f4fe5 in _gtk_marshal_BOOLEAN__BOXEDv () from /usr/local/opt/gtk+3/lib/libgtk-3.0.dylib
No symbol table info available.
#2  0x0000000100d54c41 in _g_closure_invoke_va () from /usr/local/opt/glib/lib/libgobject-2.0.0.dylib
No symbol table info available.
#3  0x0000000100d689e1 in g_signal_emit_valist () from /usr/local/opt/glib/lib/libgobject-2.0.0.dylib
No symbol table info available.
#4  0x0000000100d69362 in g_signal_emit () from /usr/local/opt/glib/lib/libgobject-2.0.0.dylib
No symbol table info available.
#5  0x0000000100624ca0 in gtk_widget_event_internal () from /usr/local/opt/gtk+3/lib/libgtk-3.0.dylib
No symbol table info available.
#6  0x00000001004f2f76 in gtk_main_do_event () from /usr/local/opt/gtk+3/lib/libgtk-3.0.dylib
No symbol table info available.
#7  0x00000001009e6a15 in _gdk_event_emit () from /usr/local/opt/gtk+3/lib/libgdk-3.0.dylib
No symbol table info available.
#8  0x0000000100a0b8d8 in gdk_event_dispatch () from /usr/local/opt/gtk+3/lib/libgdk-3.0.dylib
No symbol table info available.
#9  0x00000001002ad516 in g_main_context_dispatch () from /usr/local/opt/glib/lib/libglib-2.0.0.dylib
No symbol table info available.
#10 0x00000001002ad805 in g_main_context_iterate () from /usr/local/opt/glib/lib/libglib-2.0.0.dylib
No symbol table info available.
#11 0x00000001002ada5b in g_main_loop_run () from /usr/local/opt/glib/lib/libglib-2.0.0.dylib
No symbol table info available.
#12 0x00000001004f2a97 in gtk_main () from /usr/local/opt/gtk+3/lib/libgtk-3.0.dylib
No symbol table info available.
#13 0x00000001000fa2cb in dt_gui_gtk_run () from /usr/local/Cellar/darktable/2.2.0/lib/darktable/libdarktable.dylib
No symbol table info available.
#14 0x000000010000cf65 in main ()
No symbol table info available.
...

#7 Updated by Roman Lebedev over 2 years ago

Christian Stussak wrote:

Thanks! The homebrew formula made it a lot easier to build darktable on macOS! I had to modify it a little to use darktable 2.2.0 (not the rc) and also keep the debug info (see attachment).

And finally, I am proudly able to present a backtrace! ;-) See attached gdb.txt.

Nice!

It boils down to
[...]

Aha, can you specify CFLAGS/CXXFLAGS?
Could you please try compiling with -fsanitize=address (do not know which compiler supports it, the clang-3.9 from formula should)

#8 Updated by Christian Stussak over 2 years ago

With -fsanitize=address, the build fails at

cd /tmp/darktable-20161225-83000-w3f6a1/darktable-2.2.0/build/src &&
/usr/local/Homebrew/Library/Homebrew/shims/super/llvm_clang
-DGDK_DISABLE_DEPRECATED -DGDK_VERSION_MIN_REQUIRED=GDK_VERSION_3_14
-DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_MIN_REQUIRED
-DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_40 -DGTK_DISABLE_DEPRECATED
-DGTK_DISABLE_SINGLE_INCLUDES -DHAVE_CONFIG_H -DHAVE_GPHOTO2
-DHAVE_GPHOTO_25_OR_NEWER -DHAVE_GRAPHICSMAGICK -DHAVE_HTTP_SERVER
-DHAVE_KWALLET -DHAVE_LENSFUN -DHAVE_LIBSECRET -DHAVE_MAP -DHAVE_OPENCL
-DHAVE_OPENEXR -DHAVE_OPENJPEG -DHAVE_OSMGPSMAP_110_OR_NEWER -DHAVE_PRINT
-DMAC_INTEGRATION -DOS_OBJECT_USE_OBJC=0 -DUSE_LUA -D_RELEASE
-D_XOPEN_SOURCE=700 -D__GDK_KEYSYMS_COMPAT_H__ -Dlib_darktable_EXPORTS
-I/tmp/darktable-20161225-83000-w3f6a1/darktable-2.2.0/src
-I/tmp/darktable-20161225-83000-w3f6a1/darktable-2.2.0/src/external -isystem
/usr/local/include/glib-2.0 -isystem /usr/local/include/../lib/glib-2.0/include
-isystem /usr/local/Cellar/pcre/8.39/include -isystem
/usr/local/Cellar/glib/2.50.2/include/gio-unix-2.0 -isystem
/usr/local/Cellar/glib/2.50.2/include/glib-2.0 -isystem
/usr/local/Cellar/glib/2.50.2/lib/glib-2.0/include -isystem
/usr/local/opt/gettext/include -isystem
/usr/local/Cellar/libpng/1.6.26/include/libpng16 -isystem
/usr/local/opt/freetype/include/freetype2 -isystem
/usr/local/Cellar/fontconfig/2.12.1_2/include -isystem
/usr/local/Cellar/pixman/0.34.0/include/pixman-1 -isystem
/usr/local/Cellar/cairo/1.14.8/include/cairo -isystem
/usr/local/Cellar/gdk-pixbuf/2.36.2/include/gdk-pixbuf-2.0 -isystem
/usr/local/Cellar/atk/2.22.0/include/atk-1.0 -isystem
/usr/local/Cellar/libepoxy/1.3.1/include -isystem
/usr/local/Cellar/harfbuzz/1.3.4/include/harfbuzz -isystem
/usr/local/Cellar/pango/1.40.3/include/pango-1.0 -isystem
/usr/local/Cellar/gtk+3/3.22.5/include/gtk-3.0 -isystem /usr/include/libxml2
-isystem /usr/local/Cellar/libsoup/2.56.0/include/libsoup-2.4 -isystem
/usr/local/include -isystem /usr/local/include/OpenEXR -isystem
/usr/local/Cellar/lensfun/0.3.2_1/include/lensfun -isystem
/usr/local/include/pango-1.0 -isystem /usr/local/include/librsvg-2.0 -isystem
/usr/local/opt/sqlite/include -isystem /usr/local/include/json-glib-1.0 -isystem
/usr/local/Cellar/openjpeg/2.1.2/include/openjpeg-2.1 -isystem
/usr/local/Cellar/libsecret/0.18.5/include/libsecret-1 -isystem
/usr/local/include/gtkmacintegration -isystem /usr/local/include/GraphicsMagick
-I/tmp/darktable-20161225-83000-w3f6a1/darktable-2.2.0/build/src -isystem
/usr/local/opt/lua/include
-I/tmp/darktable-20161225-83000-w3f6a1/darktable-2.2.0/src/external/LuaAutoC
-isystem /usr/local/Cellar/pugixml/1.8.1/include/pugixml-1.8 -isystem
/usr/local/Cellar/osm-gps-map/1.1.0/include/osmgpsmap-1.0  -fsanitize=address
-D_DARWIN_C_SOURCE -Wall -fno-strict-aliasing -Wformat -Wformat-security
-Wshadow -Wtype-limits -Wvla -Wold-style-declaration -Wno-error=varargs
-Wno-error=address-of-packed-member -Wframe-larger-than=32768
-Wstack-usage=32768 -Wlarger-than=524288 -std=c99 -fopenmp=libomp -pthread
-mtune=generic -msse2 -g -O2 -g -DNDEBUG -O2 -isysroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk
-mmacosx-version-min=10.11 -fPIC   -o
CMakeFiles/lib_darktable.dir/common/cache.c.o   -c
/tmp/darktable-20161225-83000-w3f6a1/darktable-2.2.0/src/common/cache.c

with
/tmp/darktable-20161225-83000-w3f6a1/darktable-2.2.0/src/common/cache.c:372:18: error: no member named '__data' in 'struct _opaque_pthread_rwlock_t'
  if(entry->lock.__data.__nr_readers <= 1)
     ~~~~~~~~~~~ ^
1 error generated.

This doesn't surprise me. The line containing the error is part of this HIGHLY unportable function ;-) :

void dt_cache_release_with_caller(dt_cache_t *cache, dt_cache_entry_t *entry, const char *file, int line)
{
#if((__has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)) && 1)
  // yes, this is *HIGHLY* unportable and is accessing implementation details.
#ifdef _DEBUG
  if(entry->lock.lock.__data.__nr_readers <= 1)
#else
  if(entry->lock.__data.__nr_readers <= 1)
#endif
  {
    // only if there are no other reades we may poison.
    assert(entry->data_size);
    ASAN_POISON_MEMORY_REGION(entry->data, entry->data_size);
  }
#endif

  dt_pthread_rwlock_unlock(&entry->lock);
}

I patched the file such that the function only does

dt_pthread_rwlock_unlock(&entry->lock);

I don't know if my patch has any bad consequences. Someone with more darktable knowledge should fix that and make it more portable ... (open new ticket?)

Anyway. Now, I can compile with -fsanitize=address and get this when I delete the duplicated module:

=================================================================
==35173==ERROR: AddressSanitizer: heap-use-after-free on address 0x6190000f2e88 at pc 0x0001156740fc bp 0x7fff5d222400 sp 0x7fff5d2223f8
READ of size 8 at 0x6190000f2e88 thread T0
    #0 0x1156740fb in area_leave_notify (libatrous.so+0x1d0fb)
    #1 0x10326dfe4 in _gtk_marshal_BOOLEAN__BOXEDv (libgtk-3.0.dylib+0x186fe4)
    #2 0x103ae8c40 in _g_closure_invoke_va (libgobject-2.0.0.dylib+0x7c40)
    #3 0x103afc9e0 in g_signal_emit_valist (libgobject-2.0.0.dylib+0x1b9e0)
    #4 0x103afd361 in g_signal_emit (libgobject-2.0.0.dylib+0x1c361)
    #5 0x10339dc9f in gtk_widget_event_internal (libgtk-3.0.dylib+0x2b6c9f)
    #6 0x10326bf75 in gtk_main_do_event (libgtk-3.0.dylib+0x184f75)
    #7 0x103762a14 in _gdk_event_emit (libgdk-3.0.dylib+0x11a14)
    #8 0x1037878d7 in gdk_event_dispatch (libgdk-3.0.dylib+0x368d7)
    #9 0x103026515 in g_main_context_dispatch (libglib-2.0.0.dylib+0x2d515)
    #10 0x103026804 in g_main_context_iterate (libglib-2.0.0.dylib+0x2d804)
    #11 0x103026a5a in g_main_loop_run (libglib-2.0.0.dylib+0x2da5a)
    #12 0x10326ba96 in gtk_main (libgtk-3.0.dylib+0x184a96)
    #13 0x102c4c31b in dt_gui_gtk_run (libdarktable.dylib+0x25831b)
    #14 0x1029eaeca in main (darktable+0x10000deca)
    #15 0x7fff9730a5ac in start (libdyld.dylib+0x35ac)

0x6190000f2e88 is located 264 bytes inside of 952-byte region [0x6190000f2d80,0x6190000f3138)
freed by thread T0 here:
    #0 0x104795329 in wrap_free (libclang_rt.asan_osx_dynamic.dylib+0x4f329)
    #1 0x102b35fb2 in dt_iop_gui_delete_callback (libdarktable.dylib+0x141fb2)
    #2 0x103ae8a0c in g_closure_invoke (libgobject-2.0.0.dylib+0x7a0c)
    #3 0x103afbf69 in signal_emit_unlocked_R (libgobject-2.0.0.dylib+0x1af69)
    #4 0x103afccaf in g_signal_emit_valist (libgobject-2.0.0.dylib+0x1bcaf)
    #5 0x103afd361 in g_signal_emit (libgobject-2.0.0.dylib+0x1c361)
    #6 0x10339e41d in gtk_widget_activate (libgtk-3.0.dylib+0x2b741d)
    #7 0x103288566 in gtk_menu_shell_activate_item (libgtk-3.0.dylib+0x1a1566)
    #8 0x1032896bb in gtk_menu_shell_button_release (libgtk-3.0.dylib+0x1a26bb)
    #9 0x10326dfe4 in _gtk_marshal_BOOLEAN__BOXEDv (libgtk-3.0.dylib+0x186fe4)
    #10 0x103ae8c40 in _g_closure_invoke_va (libgobject-2.0.0.dylib+0x7c40)
    #11 0x103afc9e0 in g_signal_emit_valist (libgobject-2.0.0.dylib+0x1b9e0)
    #12 0x103afd361 in g_signal_emit (libgobject-2.0.0.dylib+0x1c361)
    #13 0x10339dc9f in gtk_widget_event_internal (libgtk-3.0.dylib+0x2b6c9f)
    #14 0x10326cbc3 in propagate_event (libgtk-3.0.dylib+0x185bc3)
    #15 0x10326c1bc in gtk_main_do_event (libgtk-3.0.dylib+0x1851bc)
    #16 0x103762a14 in _gdk_event_emit (libgdk-3.0.dylib+0x11a14)
    #17 0x1037878d7 in gdk_event_dispatch (libgdk-3.0.dylib+0x368d7)
    #18 0x103026515 in g_main_context_dispatch (libglib-2.0.0.dylib+0x2d515)
    #19 0x103026804 in g_main_context_iterate (libglib-2.0.0.dylib+0x2d804)
    #20 0x103026a5a in g_main_loop_run (libglib-2.0.0.dylib+0x2da5a)
    #21 0x10326ba96 in gtk_main (libgtk-3.0.dylib+0x184a96)
    #22 0x102c4c31b in dt_gui_gtk_run (libdarktable.dylib+0x25831b)
    #23 0x1029eaeca in main (darktable+0x10000deca)
    #24 0x7fff9730a5ac in start (libdyld.dylib+0x35ac)

previously allocated by thread T0 here:
    #0 0x1047956c3 in wrap_calloc (libclang_rt.asan_osx_dynamic.dylib+0x4f6c3)
    #1 0x102b286a0 in dt_dev_module_duplicate (libdarktable.dylib+0x1346a0)
    #2 0x102b3632d in dt_iop_gui_duplicate (libdarktable.dylib+0x14232d)
    #3 0x103ae8a0c in g_closure_invoke (libgobject-2.0.0.dylib+0x7a0c)
    #4 0x103afbf69 in signal_emit_unlocked_R (libgobject-2.0.0.dylib+0x1af69)
    #5 0x103afccaf in g_signal_emit_valist (libgobject-2.0.0.dylib+0x1bcaf)
    #6 0x103afd361 in g_signal_emit (libgobject-2.0.0.dylib+0x1c361)
    #7 0x10339e41d in gtk_widget_activate (libgtk-3.0.dylib+0x2b741d)
    #8 0x103288566 in gtk_menu_shell_activate_item (libgtk-3.0.dylib+0x1a1566)
    #9 0x1032896bb in gtk_menu_shell_button_release (libgtk-3.0.dylib+0x1a26bb)
    #10 0x10326dfe4 in _gtk_marshal_BOOLEAN__BOXEDv (libgtk-3.0.dylib+0x186fe4)
    #11 0x103ae8c40 in _g_closure_invoke_va (libgobject-2.0.0.dylib+0x7c40)
    #12 0x103afc9e0 in g_signal_emit_valist (libgobject-2.0.0.dylib+0x1b9e0)
    #13 0x103afd361 in g_signal_emit (libgobject-2.0.0.dylib+0x1c361)
    #14 0x10339dc9f in gtk_widget_event_internal (libgtk-3.0.dylib+0x2b6c9f)
    #15 0x10326cbc3 in propagate_event (libgtk-3.0.dylib+0x185bc3)
    #16 0x10326c1bc in gtk_main_do_event (libgtk-3.0.dylib+0x1851bc)
    #17 0x103762a14 in _gdk_event_emit (libgdk-3.0.dylib+0x11a14)
    #18 0x1037878d7 in gdk_event_dispatch (libgdk-3.0.dylib+0x368d7)
    #19 0x103026515 in g_main_context_dispatch (libglib-2.0.0.dylib+0x2d515)
    #20 0x103026804 in g_main_context_iterate (libglib-2.0.0.dylib+0x2d804)
    #21 0x103026a5a in g_main_loop_run (libglib-2.0.0.dylib+0x2da5a)
    #22 0x10326ba96 in gtk_main (libgtk-3.0.dylib+0x184a96)
    #23 0x102c4c31b in dt_gui_gtk_run (libdarktable.dylib+0x25831b)
    #24 0x1029eaeca in main (darktable+0x10000deca)
    #25 0x7fff9730a5ac in start (libdyld.dylib+0x35ac)

SUMMARY: AddressSanitizer: heap-use-after-free (libatrous.so+0x1d0fb) in area_leave_notify
Shadow bytes around the buggy address:
  0x1c320001e580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c320001e590: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c320001e5a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c320001e5b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c320001e5c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x1c320001e5d0: fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c320001e5e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c320001e5f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c320001e600: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==35164==ABORTING

[1]+  Abort trap: 6           darktable

#9 Updated by Roman Lebedev over 2 years ago

Christian Stussak wrote:

With -fsanitize=address, the build fails at
[...]
with
[...]

This doesn't surprise me. The line containing the error is part of this HIGHLY unportable function ;-) :
[...]

Yeah, that is my comment :)

I patched the file such that the function only does
[...]
I don't know if my patch has any bad consequences. Someone with more darktable knowledge should fix that and make it more portable ... (open new ticket?)

No, it should be fine.
The idea is that in
#if((__has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)) && 1), the last && 1 should be changed to && 0.
It kind-of makes sense, but maybe it should be done automatically, or maybe only for osx.

Anyway. Now, I can compile with -fsanitize=address and get this when I delete the duplicated module:
[...]

Hmm, i read that as if gtk delivers some notification after the gui for module has been freed.
Right now this looks like gtk issue i'm afraid...

#10 Updated by Roman Lebedev over 2 years ago

Christian Stussak wrote:

Anyway. Now, I can compile with -fsanitize=address and get this when I delete the duplicated module:
[...]

Oh, while there, can you somehow make it show the line numbers and not binary offsets?
https://tsdgeos.blogspot.ru/2014/03/asan-and-gcc-how-to-get-line-numbers-in.html
http://clang.llvm.org/docs/AddressSanitizer.html#symbolizing-the-reports

#11 Updated by Christian Stussak over 2 years ago

I can't get it show the line numbers easily. Homebrew is building in a sandbox and so far I was not able to access the line numbers stored in the .o files. Also, if I get it working, it will only show the line numbers for the darktable sources, not for GTK et al.

I will try again tomorrow. I have other duties today.

#12 Updated by Christian Stussak over 2 years ago

It took me several hours over the past days to get the address sanitizer to show the line numbers. The setup with homebrew and macOS makes it quite complicated to provide the necessary information to the address sanitizer. Especially, it is not easy to figure out what's wrong if no line numbers are shown. I will clean up my homebrew formula and upload it later. But now, the ASAN output WITH line numbers for the darktable 2.2.0 source:

=================================================================
==73310==ERROR: AddressSanitizer: heap-use-after-free on address 0x6190016a5e88 at pc 0x000113232450 bp 0x7fff5f552360 sp 0x7fff5f552358
READ of size 8 at 0x6190016a5e88 thread T0
    #0 0x11323244f in area_leave_notify /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/iop/atrous.c:1159:67
    #1 0x100ff5fe4 in _gtk_marshal_BOOLEAN__BOXEDv (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x186fe4)
    #2 0x101879c40 in _g_closure_invoke_va (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x7c40)
    #3 0x10188d9e0 in g_signal_emit_valist (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1b9e0)
    #4 0x10188e361 in g_signal_emit (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1c361)
    #5 0x101125c9f in gtk_widget_event_internal (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x2b6c9f)
    #6 0x100ff3f75 in gtk_main_do_event (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x184f75)
    #7 0x1014eda14 in _gdk_event_emit (/usr/local/opt/gtk+3/lib/libgdk-3.0.dylib+0x11a14)
    #8 0x1015128d7 in gdk_event_dispatch (/usr/local/opt/gtk+3/lib/libgdk-3.0.dylib+0x368d7)
    #9 0x100dad515 in g_main_context_dispatch (/usr/local/opt/glib/lib/libglib-2.0.0.dylib+0x2d515)
    #10 0x100dad804 in g_main_context_iterate (/usr/local/opt/glib/lib/libglib-2.0.0.dylib+0x2d804)
    #11 0x100dada5a in g_main_loop_run (/usr/local/opt/glib/lib/libglib-2.0.0.dylib+0x2da5a)
    #12 0x100ff3a96 in gtk_main (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x184a96)
    #13 0x10091f274 in dt_gui_gtk_run /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/gui/gtk.c:981:3
    #14 0x1006baeca in main /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/main.c:25:3
    #15 0x7fff9730a5ac in start (/usr/lib/system/libdyld.dylib+0x35ac)

0x6190016a5e88 is located 264 bytes inside of 952-byte region [0x6190016a5d80,0x6190016a6138)
freed by thread T0 here:
    #0 0x102512329 in wrap_free (/usr/local/opt/llvm/lib/clang/3.9.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x4f329)
    #1 0x100804c23 in dt_iop_gui_delete_callback /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/develop/imageop.c:565:3
    #2 0x101879a0c in g_closure_invoke (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x7a0c)
    #3 0x10188cf69 in signal_emit_unlocked_R (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1af69)
    #4 0x10188dcaf in g_signal_emit_valist (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1bcaf)
    #5 0x10188e361 in g_signal_emit (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1c361)
    #6 0x10112641d in gtk_widget_activate (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x2b741d)
    #7 0x101010566 in gtk_menu_shell_activate_item (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x1a1566)
    #8 0x1010116bb in gtk_menu_shell_button_release (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x1a26bb)
    #9 0x100ff5fe4 in _gtk_marshal_BOOLEAN__BOXEDv (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x186fe4)
    #10 0x101879c40 in _g_closure_invoke_va (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x7c40)
    #11 0x10188d9e0 in g_signal_emit_valist (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1b9e0)
    #12 0x10188e361 in g_signal_emit (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1c361)
    #13 0x101125c9f in gtk_widget_event_internal (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x2b6c9f)
    #14 0x100ff4bc3 in propagate_event (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x185bc3)
    #15 0x100ff41bc in gtk_main_do_event (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x1851bc)
    #16 0x1014eda14 in _gdk_event_emit (/usr/local/opt/gtk+3/lib/libgdk-3.0.dylib+0x11a14)
    #17 0x1015128d7 in gdk_event_dispatch (/usr/local/opt/gtk+3/lib/libgdk-3.0.dylib+0x368d7)
    #18 0x100dad515 in g_main_context_dispatch (/usr/local/opt/glib/lib/libglib-2.0.0.dylib+0x2d515)
    #19 0x100dad804 in g_main_context_iterate (/usr/local/opt/glib/lib/libglib-2.0.0.dylib+0x2d804)
    #20 0x100dada5a in g_main_loop_run (/usr/local/opt/glib/lib/libglib-2.0.0.dylib+0x2da5a)
    #21 0x100ff3a96 in gtk_main (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x184a96)
    #22 0x10091f274 in dt_gui_gtk_run /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/gui/gtk.c:981:3
    #23 0x1006baeca in main /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/main.c:25:3
    #24 0x7fff9730a5ac in start (/usr/lib/system/libdyld.dylib+0x35ac)

previously allocated by thread T0 here:
    #0 0x1025126c3 in wrap_calloc (/usr/local/opt/llvm/lib/clang/3.9.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x4f6c3)
    #1 0x1007f7271 in dt_dev_module_duplicate /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/develop/develop.c:1503:48
    #2 0x100804f9e in dt_iop_gui_duplicate /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/develop/imageop.c:746:29
    #3 0x101879a0c in g_closure_invoke (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x7a0c)
    #4 0x10188cf69 in signal_emit_unlocked_R (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1af69)
    #5 0x10188dcaf in g_signal_emit_valist (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1bcaf)
    #6 0x10188e361 in g_signal_emit (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1c361)
    #7 0x10112641d in gtk_widget_activate (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x2b741d)
    #8 0x101010566 in gtk_menu_shell_activate_item (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x1a1566)
    #9 0x1010116bb in gtk_menu_shell_button_release (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x1a26bb)
    #10 0x100ff5fe4 in _gtk_marshal_BOOLEAN__BOXEDv (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x186fe4)
    #11 0x101879c40 in _g_closure_invoke_va (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x7c40)
    #12 0x10188d9e0 in g_signal_emit_valist (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1b9e0)
    #13 0x10188e361 in g_signal_emit (/usr/local/opt/glib/lib/libgobject-2.0.0.dylib+0x1c361)
    #14 0x101125c9f in gtk_widget_event_internal (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x2b6c9f)
    #15 0x100ff4bc3 in propagate_event (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x185bc3)
    #16 0x100ff41bc in gtk_main_do_event (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x1851bc)
    #17 0x1014eda14 in _gdk_event_emit (/usr/local/opt/gtk+3/lib/libgdk-3.0.dylib+0x11a14)
    #18 0x1015128d7 in gdk_event_dispatch (/usr/local/opt/gtk+3/lib/libgdk-3.0.dylib+0x368d7)
    #19 0x100dad515 in g_main_context_dispatch (/usr/local/opt/glib/lib/libglib-2.0.0.dylib+0x2d515)
    #20 0x100dad804 in g_main_context_iterate (/usr/local/opt/glib/lib/libglib-2.0.0.dylib+0x2d804)
    #21 0x100dada5a in g_main_loop_run (/usr/local/opt/glib/lib/libglib-2.0.0.dylib+0x2da5a)
    #22 0x100ff3a96 in gtk_main (/usr/local/opt/gtk+3/lib/libgtk-3.0.dylib+0x184a96)
    #23 0x10091f274 in dt_gui_gtk_run /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/gui/gtk.c:981:3
    #24 0x1006baeca in main /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/main.c:25:3
    #25 0x7fff9730a5ac in start (/usr/lib/system/libdyld.dylib+0x35ac)

SUMMARY: AddressSanitizer: heap-use-after-free /tmp/darktable-20161229-59442-1vbxvm7/darktable-2.2.0/src/iop/atrous.c:1159:67 in area_leave_notify
Shadow bytes around the buggy address:
  0x1c32002d4b80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c32002d4b90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c32002d4ba0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c32002d4bb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c32002d4bc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x1c32002d4bd0: fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c32002d4be0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c32002d4bf0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c32002d4c00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c32002d4c10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c32002d4c20: fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==73310==ABORTING
Abort trap: 6

#13 Updated by Christian Stussak over 2 years ago

I added the new homebrew formula. It adds the necessary

-gdwarf-2 -fno-omit-frame-pointer -fsanitize=address

compiler flags and then uses dsymutil to extract the DWARF debug information out of the object files for each of darktables executables and shared libraries into a separate DSYM folder located next to the respective executable or library (/usr/local/Cellar/darktable/...). This is necessary since homebrew deletes all the object files after building and installing such that things like line numbers would be lost.

The formula isn't really perfect (keeps debug info in /usr/local/Cellar/darktable ...), but was good enough for me to get the line numbers working with ASAN. Maybe this is helpful for someone else ...

#14 Updated by Roman Lebedev over 2 years ago

  • Priority changed from Low to High
  • % Done changed from 20 to 10
  • Affected Version changed from 2.2.0 to git master branch
  • System changed from Mac OS X to all
  • Status changed from Incomplete to Confirmed

Great, exactly what i thought :(
The gtk delivers signals after the module is deleted by us/user.

My first guess have been g_signal_handlers_disconnect_by_data(), but it requires to be called on the each widget in iop-to-be-deleted.

#15 Updated by Roman Lebedev over 2 years ago

Awesome report!

Please, try current git master / 2.2.x branch, and try to reproduce again.

#16 Updated by Roman Lebedev over 2 years ago

  • Status changed from Confirmed to Fixed
  • % Done changed from 10 to 100

#17 Updated by Roman Lebedev over 2 years ago

  • Target version set to 2.4.0

#18 Updated by Christian Stussak over 2 years ago

I tested latest 2.2.x (fa933c9f4885119a54def3b929b670f5e95fd9d2) and the bug seems gone. Awesome!

Thank you very much!

Also available in: Atom PDF