Project

General

Profile

Bug #9443

hovering the mouse over the light table seeks the disk

Added by Antoine Beaupré about 6 years ago. Updated over 5 years ago.

Status:
Triaged
Priority:
Low
Assignee:
-
Category:
Lighttable
Target version:
Start date:
05/26/2013
Due date:
% Done:

20%

Affected Version:
1.2
System:
other GNU/Linux
bitness:
64-bit
hardware architecture:
amd64/x86

Description

My photo collection is not (yet?) on a SSD drive, so I actually here when the disk is seeked to.

When I mouse over pictures, darktable quickly seeks over the disk. This is a minor annoyance, but I don't understand why this is necessary. After strace'ing the process, it seems those are the reads:

[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16
[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16
[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16
[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16
[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16
[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16
[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16
[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16
[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16
[pid  2960] read(3, "\0\2%@\0\0\0\217\0\0\0\0\0\0\0\0", 16) = 16

Those can happen many times a second if the zoom level shows a lot of thumbnails, and can therefore be a significant performance hit.

File descriptor 3 is the library, of course:

COMMAND    PID    USER   FD   TYPE             DEVICE SIZE/OFF     NODE NAME
darktable 2960 anarcat    3u   REG              254,4  4685824  2933403 /home/anarcat/.config/darktable/library.db

This is on Debian (wheezy), which is not in the System menu, unfortunately :).

History

#1 Updated by Antoine Beaupré about 6 years ago

i suspect this is to update the "image information" pane, so I guess this would be a reasonable cost if the pane was open.

even then, i am not sure we should show that information on mouse over but rather on selection - but I won't get in that bikeshed. :)

#2 Updated by Antoine Beaupré about 6 years ago

I tried something suggested on IRC, adding the following indexes:

CREATE INDEX tag_images_id ON tagged_images(tagid); CREATE INDEX tag_id ON tags(id);

.. doesn't help.

What I don't understand is why darktable would need to seek the disk at all, considering the DB is about 5MB... I am not all that familiar with sqlite (I work more with database servers like MySQL), but it seems to me this DB should snugly fit in the 8GB of ram I have available. ;)

#3 Updated by Antoine Beaupré about 6 years ago

running darktable with -d sql shows a lot of SQL requests.

hilighting a different image does those requests:

[sql] prepare "SELECT DISTINCT T.id, T.name FROM tagged_images JOIN tags T on T.id = tagged_images.tagid WHERE tagged_images.imgid = 8332" 
[sql] prepare "select folder from film_rolls where id = ?1" 
[sql] prepare "select folder || '/' || filename from images, film_rolls where images.film_id = film_rolls.id and images.id = ?1" 
[sql] prepare "select value from meta_data where id = ?1 and key = ?2 order by value" 
[sql] prepare "select value from meta_data where id = ?1 and key = ?2 order by value" 
[sql] prepare "select value from meta_data where id = ?1 and key = ?2 order by value" 

but, simply moving the mouse a single pixel does this request:

[sql] prepare "select count(id) from images where   (flags & 256) != 256 and ((film_id in (select id from film_rolls where folder like '%'))) order by datetime_taken desc limit ?1, ?2" 

That query is therefore done hundreds of times simply when the mouse is dragged across the light table.

A serious audit of SQL queries would probably help a lot in improving performance of DT. Simply starting the software generates around 1800 requests, for example...

#4 Updated by Antoine Beaupré about 6 years ago

chatting with jcsogo on irc, we have identified two distinct possible problems here:

  1. SQL queries are hitting the disk, whereas they should actually hit the memory using some SQL cache, since the DB is small enough to be fully loaded in memory
  2. SQL queries shouldn't be issued for data structures that are cached in memory by DT

The first issue may be a configuration problem with sqlite. We have found a few resources to read up on the web regarding that:

Other interesting sqlite optimisation ideas:

The second issue may not be a problem after all as the queries do not concern the data structure (dt_image_t) we were concerned about.

#5 Updated by Simon Spannagel over 5 years ago

  • Status changed from New to Triaged
  • % Done changed from 0 to 20
  • bitness set to 64-bit

#6 Updated by Tobias Ellinghaus over 5 years ago

The count updating should be gone by now, and I am not sure how much we can do about sqlite deciding to look at the disk all the time.

Also available in: Atom PDF