Project

General

Profile

Bug #12393

Segmentation fault when host_memory_limit is too high

Added by Ioannis Galanommatis 3 months ago. Updated 2 months ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
General
Target version:
-
Start date:
11/06/2018
Due date:
% Done:

0%

Affected Version:
2.4.4
System:
other GNU/Linux
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Hello,
There is an issue that it appears when host_memory_limit is set too high. DT may initially report memory allocation failure while displaying thumbnails and will continue working normally. However clicking at a picture (.dng) may crash with SEGV. I also got some ABORTs but could not understand when exactly.

A case of SEGV:

Thread 8 "worker 3" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffd5ffb700 (LWP 23038)]
0x00007fff8c30a1c8 in demosaic_ppg (thrs=0, filters=<optimized out>, roi_in=0x7fffd5fd8be0,
roi_out=0x7fffd5fd8c00, in=0x7ffdf064e010, out=0x0)
at /usr/src/debug/media-gfx/darktable-2.4.4/darktable-2.4.4/src/iop/demosaic.c:2431
2431 out[4 * ((size_t)j * roi_out->width + i) + c]

and backtrace is:
#0 0x00007fff8c30a1c8 in demosaic_ppg (thrs=0, filters=<optimized out>, roi_in=0x7fffd5fd8be0,
roi_out=0x7fffd5fd8c00, in=0x7ffdf064e010, out=0x0)
at /usr/src/debug/media-gfx/darktable-2.4.4/darktable-2.4.4/src/iop/demosaic.c:2431
#1 process (self=<optimized out>, piece=<optimized out>, i=<optimized out>, o=0x7ffddcb44010,
roi_in=<optimized out>, roi_out=0x7fffd5fd9300)
at /usr/src/debug/media-gfx/darktable-2.4.4/darktable-2.4.4/src/iop/demosaic.c:2848
#2 0x00007ffff7a1f005 in dt_dev_pixelpipe_process_rec (pipe=pipe@entry=0x7fffd5fe9540,
dev=dev@entry=0x7fffd5fe9bd0, output=output@entry=0x7fffd5fdb908,
cl_mem_output=cl_mem_output@entry=0x7fffd5fdb910, out_format=out_format@entry=0x7fffd5fdb918,
roi_out=roi_out@entry=0x7fffd5fd9300, modules=0x555557bc5200, pos=8, pieces=<optimized out>,
pieces=<optimized out>)
at /usr/src/debug/media-gfx/darktable-2.4.4/darktable-2.4.4/src/develop/pixelpipe_hb.c:1543
#3 0x00007ffff7a1e8eb in dt_dev_pixelpipe_process_rec (pipe=pipe@entry=0x7fffd5fe9540,
dev=dev@entry=0x7fffd5fe9bd0, output=output@entry=0x7fffd5fdb908,
cl_mem_output=cl_mem_output@entry=0x7fffd5fdb910, out_format=out_format@entry=0x7fffd5fdb918,
roi_out=roi_out@entry=0x7fffd5fd9740, modules=0x7fffc0041c40, pos=9, pieces=<optimized out>,
pieces=<optimized out>)
at /usr/src/debug/media-gfx/darktable-2.4.4/darktable-2.4.4/src/develop/pixelpipe_hb.c:599

We can see out being zero, and a step back at caller, we can see at demosaic:2792 that the tmp pointer, passed to out, was allocated but not checked against null.

For the case of ABORT a gdb output is attached. I really don't know why pthread_mutex_lock would raise an abort.

(distro: Gentoo, kernel 4.19, gcc 8.2.0)

darktable.txt Magnifier - crash with ABORT (8.23 KB) Ioannis Galanommatis, 11/06/2018 12:40 AM

History

#1 Updated by Aurélien PIERRE 3 months ago

Do you want dt to work even if you set it wrong ? You said yourself the memory was too large. Decrease it then.

#2 Updated by Ioannis Galanommatis 2 months ago

Aurélien PIERRE wrote:

Do you want dt to work even if you set it wrong ? You said yourself the memory was too large. Decrease it then.

It depends on how we define "to work". I do not expect dt to perform an action that requires more memory than the system has, but it must not terminate uncontrollably with a segmentation fault.

As I mentioned, if the allocation error happens during thumbnail generation, it is handled gracefully, notifying me to reduce the number of threads etc. But when it happens during demosaicing, it just crashes. I believe every malloc should be checked against returning a NULL, giving the program a chance to recover without losing unsaved work...

#3 Updated by Aurélien PIERRE 2 months ago

It seems to me that's it's super hard to check for the memory use at a global level, provided that every modules does its own internal memcopies silently. The most you can do is triggering a warning if the user has done some obvious stupidities.

#4 Updated by Ioannis Galanommatis 2 months ago

Aurélien PIERRE wrote:

It seems to me that's it's super hard to check for the memory use at a global level, provided that every modules does its own internal memcopies silently. The most you can do is triggering a warning if the user has done some obvious stupidities.

The thing is, a value that usually works good may break under load. But generally, limiting the cache at a percentage of the physical memory would prevent a crash in case of wrong entry, downgrading memory or swapping pcs.

But anyway, for the specific case, I'll try to understand how DT usually handles memory allocation errors (I have never read its sources) and I will propose a patch.

Thanks.

Also available in: Atom PDF