Project

General

Profile

Bug #11044

darktable consumes too much memory exporting >150 megapixel images

Added by Joxean Koret about 3 years ago. Updated about 3 years ago.

Status:
Incomplete
Priority:
Low
Assignee:
-
Category:
General
Target version:
-
Start date:
06/02/2016
Due date:
% Done:

20%

Estimated time:
Affected Version:
2.0.4
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Hi,

I'm trying to export some enhancements I have made to various somewhat big panoramas (>150 megapixels) and, with some pictures, it is simple impossible.
I have tried with both DarkTable 2.0.0 (build by myself) in Ubuntu Linux x64 and with DarkTable 2.0.4 in MacOSX El Capitan. In Linux, it eats almost all available memory until I kill it (because the amount of memory is increasing continuously). In MacOSX, selecting just 1 such picture and trying to export it results in no result for >8 hours.

If you need any such image, please tell, and I will upload to somewhere (they are way-way bigger than 39,1 MB, naturally).

Thanks & regards,
Joxean Koret

History

#1 Updated by Roman Lebedev about 3 years ago

  • % Done changed from 0 to 20
  • Status changed from New to Incomplete

Pretty sure that is not a bug.

But, how much memory does your machine have?
What is the value of "host memory limit (in MB) for tiling" ?
I would highly recommend to close all other apps when exporting such image, and set that ^ pref to slightly less than the total system memory

Also, have you read https://www.darktable.org/usermanual/ch10.html.php#darktable_and_memory ?

That being said, with 32Gb system memory i did successfully export 350+ MPix images successfully.

Also, be sure to read whole thread: http://thread.gmane.org/gmane.comp.graphics.darktable.user/8763

#2 Updated by Roman Lebedev about 3 years ago

  • Subject changed from DarkTable consumes too much memory exporting >150 megapixel images to darktable consumes too much memory exporting >150 megapixel images

Hm, DarkTable? What is that?

#3 Updated by Joxean Koret about 3 years ago

Hi,

But, how much memory does your machine have?

One of them 32GB, the other just 8GB.

What is the value of "host memory limit (in MB) for tiling" ?

The default value, I guess: 1500. I haven't touched it.

Also, have you read https://www.darktable.org/usermanual/ch10.html.php#darktable_and_memory ?

Aaah, no. I didn't. It explains things. Nevertheless, I think there is a memory leak somewhere. See below.

That being said, with 32Gb system memory i did successfully export 350+ MPix images successfully.

Finally, I have been able to export them by doing the following:

1) Open DT, select just 1 pic and export it.
2) Close DT.
3) Goto #1.

If I select various images, it crashes (OOM) or simply eats too much memory forcing the box to use swap and slowing down everything a lot. If I try to select and export, without closing DT, each image, there is a point where it eats all the memory it can. So, my guess, is that there is some memory leak somewhere.

I will debug it with Valgrind to check where is the leak.

Also, be sure to read whole thread: http://thread.gmane.org/gmane.comp.graphics.darktable.user/8763

Interesting thread that explains many things. Thanks!

#4 Updated by Tobias Ellinghaus about 3 years ago

What modules are you using on the leaking (?) images? Maybe try exporting with no modules active at all, just to help pinpoint if the leak is in the core.

Also available in: Atom PDF

Go to top