Project

General

Profile

Bug #8570

OOM to allocate for large image causes SegFault.

Added by dakiri - over 8 years ago. Updated almost 8 years ago.

Status:
Closed: invalid
Priority:
High
Assignee:
-
Category:
General
Start date:
Due date:
% Done:

0%

Estimated time:
Affected Version:
0.9.3
System:
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Error observed in darktable version 0.9.2-1 of Debian 'unstable/sid'.

To encounter this error, load a large image in darktable.
I do it from the command line, so:

darktable [[LargeImage]].tiff

You can limit memory available to the process using:
ulimit -Sv

.
In this case, I load a high resolution TIFF scan which has 8 bit color depth for each of 3 color channels.
darktable will emit some errors like:
[image_alloc] malloc of 6000 x 8000 x 16 for image Scan-111031-0001.tif mip 6 failed!
[tiff_open] could not alloc full buffer for image [[LargeImage]].tiff'
[image_alloc] malloc of 600 x 800 x 16 for image Scan-111031-0001.tif mip 5 failed!
[tiff_open_preview] warning: config other than contig found, trying anyways
Segmentation fault

If darktable is not limited by ulimit and there is much swap space available, darktable may effectively halt the system by using much swap.

What unit is the x * y * 16 ? If it is bits, is that for 1 color channel or for all 3? If it is allocating 16 bits for each channel where the image only has 8 bits, that may be more memory allocated than necessary.

Some suggestions if you wish to load such images, rather than fail:
If clearly more memory is being allocated than necessary, allocate less. Other programs are able to use
If there is insufficient memory, perhaps it would work to allocate a file and mmap it. This may create as much disk access as swap, though.

libvips is licensed LGPL?2 and indexes files it loads so it does not need to load the whole thing into RAM simultaneously. Maybe using it would be an option. It works better with little RAM than any other free software.

The backtrace follows:

(gdb) bt
#0  dt_iop_clip_and_zoom._omp_fn.2 () at /usr/lib/gcc/i486-linux-gnu/4.6.1/include/xmmintrin.h:170
#8271  0xb6ce72a4 in dt_iop_clip_and_zoom (out=0x942c7040, in=0x0, roi_out=0xbfffb9ac, roi_in=0xbfffb998, out_stride=544, in_stride=4422)
    at /build/buildd-darktable_0.9.2-1-i386-DdNqdC/darktable-0.9.2/src/develop/imageop.c:1034
#8272  0xb6cb69b1 in dt_image_raw_to_preview (img=0xb1246008, raw=0x0)
    at /build/buildd-darktable_0.9.2-1-i386-DdNqdC/darktable-0.9.2/src/common/image.c:402
#8273  0xb6cb7448 in dt_image_load (img=0xb1246008, mip=DT_IMAGE_FULL)
    at /build/buildd-darktable_0.9.2-1-i386-DdNqdC/darktable-0.9.2/src/common/image.c:963
#8274  0xb6cb77ed in dt_image_get_blocking (img=0xb1246008, mip_in=DT_IMAGE_FULL, mode=114 'r')
    at /build/buildd-darktable_0.9.2-1-i386-DdNqdC/darktable-0.9.2/src/common/image.c:1378
#8275  0xb6ca0234 in dt_init (argc=2, argv=0xbffff214, init_gui=1)
    at /build/buildd-darktable_0.9.2-1-i386-DdNqdC/darktable-0.9.2/src/common/darktable.c:542
#8276  0x08048893 in main (argc=2, argv=0xbffff214) at /build/buildd-darktable_0.9.2-1-i386-DdNqdC/darktable-0.9.2/src/main.c:24

History

#1 Updated by dakiri - over 8 years ago

As for my absolute statement that vips was the best with respect to RAM usage, I may have to recind that statement.
There is a Java library I have not tried and there may be some geographic programs using GeoTIFF which handle RAM well.

#2 Updated by Simon Spannagel almost 8 years ago

Hi,

could you check with version 1.0 again? There is a complete rewrite of the cache and memory usage in that version so this might be fixed - not sure though...

#3 Updated by Simon Spannagel over 7 years ago

  • Target version changed from 1.0.3 to Candidate for next patch release

#4 Updated by Simon Spannagel over 7 years ago

  • Affected Version set to 0.9.3
  • Status changed from New to Closed: invalid

No answer. Given the complete cache rewrite for darktable 1.0 I tag this "invalid".
Best would be to open a new ticket for any memory/cache problems with latest stable version.

Also available in: Atom PDF

Go to top