Project

General

Profile

Bug #12149

Darkrtable segfaults when recreating many thumbs.

Added by Stefan Klinger 5 months ago. Updated 4 months ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
-
Target version:
-
Start date:
04/22/2018
Due date:
% Done:

0%

Affected Version:
git master branch
System:
other GNU/Linux
bitness:
64-bit
hardware architecture:
amd64/x86

Description

See

https://www.mail-archive.com/darktable-user@lists.darktable.org/msg05366.html

for discussion so far. I'll summarize in the following:

Darktable segfaults shortly after startup.

This occurred for the first time when I tried to apply a history stack
to a couple of images (about 15). Darktable crashed during this
operation. Then it alwys sefaulted on startup.

A couple of times, just before crashing, it said something similar to
“not enough memory for thumbnail export”, but that message disappeared
in the crash, and now it is not shown any more. I've got about 8GB
RAM, and the amount of available RAM seems not to be an issue (never
allocated).

$ darktable --version
this is darktable 2.5.0+319~gace20731f
copyright (c) 2009-2018 johannes hanika
compile options:
bit depth is 64 bit
normal build
SSE2 optimized codepath enabled
OpenMP support enabled
OpenCL support enabled
Lua support enabled, API version 5.0.0
Colord support disabled
gPhoto2 support enabled
GraphicsMagick support enabled
OpenEXR support enabled

I never see all memory being allocated (conky, top, time). Using
time(1), I see

$ time -f 'usr=%Us ker=%Ss rss=%MkB' darktable
free(): invalid pointer
Command terminated by signal 6
usr=3.61s ker=4.81s rss=940436kB

Error messages vary:

$ darktable
Segmentation fault (core dumped)
$ darktable
libgomp: Thread creation failed: Resource temporarily unavailable
darktable: hb-object-private.hh:154: Type* hb_object_reference(Type*) [with Type = hb_unicode_funcs_t]: Assertion `hb_object_is_valid (obj)' failed.
Aborted (core dumped)

and others.

A first backtrace creatied with gdb is in file `gdb.txt`

johannes hanika (2018-Apr-21, excerpt):

did you at some point adjust the number of worker threads by any
chance?

Not that I'd be aware of. I have

$ grep work ~/.config/darktable/darktablerc 
worker_threads=8

Anyplace else?

there is a surprising amount of different stuff going on that
all allocates memory.. 8GB isn't all that much in the end.

Well I never see darktable using more than 2GB, so there's like 6GB
left unused.

among others, the nvidia opencl backend tries to malloc something, too. so i'd

  • look through settings for super large number of worker threads somewhere
  • try with `darktable --disable-opencl` to see whether the driver
    needs all the memory

Also segfaults. This is in file `gdb-disable-opencl.txt`.

$ gdb --args darktable --disable-opencl
(gdb) set logging on
(gdb) run
(gdb) thread apply all backtrace

File `nvidia-smi.txt` is the output from Nvidia's monitoring tool
`nvidia-smi dmon`, started before, and terminated after a crash. To
me it does not look like running out of memory.

Now I'm convinced that it has to do with thumbnail creation: I can
easily trigger the crash with

$ rm -rf ~/.cache/darktable

and then scrolling up throught the photo collection. Now darktable
always crashes immediately when I start it. I have attached a ggb
backtrace `gdb-after-clear-cache.txt` of this.

Michael Below (2018-Apr-22, excerpt):

This may be a general load issue on your system. Maybe check the
temperature sensors and run a RAM test.

Generally my system is very stable, even under heavy load. I do not
see crashes other than the one related to darktable, and that one is
limited in its effect to darktabel.

E.g., compiling darkrtable may rise temperature to 49℃, without any
issues in stability. On the other hand, when darkrtable segfaults as
described, temperature stays well below 36℃.

Also, when the problem occurs, the system is not actually under heavy
load! There is no slow down in interaction (e.g., sticky mouse
pointer) nor other signs of overload as I sometimes see on my old
netbook when firefox opens too many tabs...

As expected, memtest86+ says my RAM is fine.

After recreating the cache with `darktable-generate-cache`, darktable
starts fine.

I have also observed this with the following version:

$ darktable --version
this is darktable 2.5.0+329~g81ef6a2c4
copyright (c) 2009-2018 johannes hanika
compile options:
bit depth is 64 bit
normal build
SSE2 optimized codepath enabled
OpenMP support enabled
OpenCL support enabled
Lua support enabled, API version 5.0.0
Colord support disabled
gPhoto2 support enabled
GraphicsMagick support enabled
OpenEXR support enabled

gdb.txt Magnifier - gdb backtrace (78.8 KB) Stefan Klinger, 04/22/2018 06:29 PM

gdb-disable-opencl.txt Magnifier - gdb backtrace without OpenCL (133 KB) Stefan Klinger, 04/22/2018 06:31 PM

nvidia-smi.txt Magnifier - nvidia's monitoring of graphics card (2.85 KB) Stefan Klinger, 04/22/2018 06:33 PM

gdb-after-clear-cache.txt Magnifier - gdb backtrace after clearing cache (114 KB) Stefan Klinger, 04/22/2018 06:36 PM

History

#1 Updated by Stefan Klinger 5 months ago

I've upgraded darktable to version 2.5.0+333~g1f26b4f44 this morning.
Now the problem is gone.

#2 Updated by Tobias Ellinghaus 4 months ago

All of those crashes were inside malloc. So maybe the memory consumption wasn't too high yet, but it tried to do something stupid. It might help to lower the number of worker threads so there are less thumbnails processed in parallel, leading to lower memory consumption.

Also available in: Atom PDF