Lighttable can't show all the thumbnails selected
Here's what I'm seeing:
- I select 54 images
- I use PgUp/PgDown to navigate in the lighttable
- As I'm doing this images that previously have been shown disappear and new ones appear
- At the preview level I'm using (5), 25 images fit on the screen and yet it's never possible to see all of them at once
- darktable is happily using 100% of 1 CPU continuously to do this
Here are some relevant details:
- I'm running darktable git master on an Ubuntu 12.04 32bit running inside Virtualbox on a Lenovo 220t running Windows 7. The VM has 1GB of memory and 2 CPUs.
- I've enabled half-size raws for previews
- I have 200MB for mipmap cache, 500MB for tiling and 8MB for single buffer in tiling
- I set max widthXheight of the drawing area to 1920x1200 to be able to use a large external monitor
Here's what I think is happening:
- 200MB isn't enough to hold all the images in cache
- Images are constantly being evicted from memory and new ones calculated and put back in
- When an image is evicted it's removed from the lighttable view
- Since the cache can't even hold 25 images darktable is continuously cycling through the 25 images evicting one and calculating the next
#1 Updated by Ulrich Pegelow over 5 years ago
I'm running darktable git master on an Ubuntu 12.04 32bit running inside Virtualbox on a Lenovo 220t running Windows 7. The VM has 1GB of memory and 2 CPUs.
You likely know this: [[http://www.darktable.org/usermanual/ch08.html.php#darktable_and_memory]]
Your setup is extremely tight for darktable and what you see is probably the maximum you can get (I am surprised that you even get darktable to generate thumbs with this low memory).
Know, fixing that issue would more or less require a complete rewrite of darktable's pixelpipe with the focus of low mem/address space systems in mind. As world moves to 64-bit that would not be a reasonable step IMHO.
What you could do: try to give your VM at least 4GB of memory.
#2 Updated by Pedro Côrte-Real over 5 years ago
I'm planning on getting a machine with more memory but darktable works fine if slow in this setup. To work around this bug I lowered the maximum image size to 800x500 and it displays the thumbnail fine now.
I don't think this problem can ever be solved with more memory. What I reproduced here with 25 images is what this user reports on a full collection:
By making the thumbnail cache memory only you can never have thumbnails generated for the full collection so you're constantly throwing away work you've already done. My suggestion here would be to make the cache write through in one of two ways:
1) Stay with the single large cache file but mmap it in, so it becomes the kernel's job to page it in and out of memory. This also avoids the writing/reading of the full file on startup. This is probably the most similar to the current setup. The problem it may have is that if you corrupt it you lose the whole file
2) Store individual mipmaps, one file per image in the cache dir. You can still mmap these in and let the kernel deal with paging and if any of them are corrupt you just lose one file.
#3 Updated by Pedro Côrte-Real over 5 years ago
Sent a message to darktable-devel describing in more detail a cache setup that would fix this issue and others:
#4 Updated by Pedro Côrte-Real over 5 years ago
The bug here seems to be that lighttable is removing and recalculating the already displayed thumbnail just because it was evicted from cache, so it ends up in an infinite loop if the cache is not large enough to hold all the images in view. This is an edge case from having a very small cache and very large thumbnail size. The slow cache behavior for large collections is actually a separate issue and I'll send in a new bug report for that.
#5 Updated by Pedro Côrte-Real over 5 years ago
Sent in a pull request that partially fixes this:
By setting the max thumbnail size (DT_MIPMAP_3) to never be above 640x480 many more thumbnails will now fit in cache even when the max size is at 1920x1200. Darkroom mode and lighttable Z mode still let you view a full size image though.
This is only a partial fix as the underlying bug is that lighttable will go into an infinite loop if the cache can't fit all the visible images at the current thumbnail size. This doesn't fix that or guarantee that it never happens but makes it less likely by making sure more thumbnails fit in cache.
#6 Updated by Pedro Côrte-Real over 4 years ago
- Affected Version changed from git development version to git stable branch
- % Done changed from 0 to 100
- Target version set to Candidate for next major release
- Status changed from New to Fixed
This is now fixed in current git master with the new multi-level cache. Will be in a stable version when we release 2.0.