darktable-cli confused if image is a symlink
When processing a picture that is a symbolic link,
darktable-cli at some point tries to open the name of the symlink destination, from the directory of the symlink source. As a result, file is not found, image is not processed.
Interactive darktable is not affected when opening a directory.
How to reproduce¶
Prepare a simple layout¶
Here is the test layout:
lrwxrwxrwx 1 37 janv. 17 20:26 symlinksourcedir/symlinksourcename.jpg -> ../symlinkdestdir/symlinkdestname.jpg -rw-rw-r-- 1 1459972 janv. 17 20:26 symlinkdestdir/symlinkdestname.jpg
Test layout can be reproduced by copy-pasting this:
cd /tmp ; rm -rf symlinksourcedir symlinkdestdir mkdir -p symlinksourcedir symlinkdestdir cp someimage.jpg symlinkdestdir/symlinkdestname.jpg ln -s ../symlinkdestdir/symlinkdestname.jpg symlinksourcedir/symlinksourcename.jpg ls -gG symlinksourcedir/* ; ls -gG symlinkdestdir/*
darktable-cli symlinksourcedir/symlinksourcename.jpg out.jpg
Expected vs observed¶
[export_job] exported to `out.jpg'
allocation failed??? [imageio_storage_disk] could not export to file: `out.jpg'!
- Interactive darktable is not affected when opening a directory.
- darktable-cli is observed to be affected from darktable 2.4.0, and on current master branch.
Attached trace file¶
Trace obtained with:
strace -fF -s 1024 -o dt-cli.strace.txt darktable-cli symlinksourcedir/symlinksourcename.jpg out.jpg
Indeed darktable-cli mixes source and destination: no symlinksourcedir/symlinkdestname.jpg exists.
grep source.*dest.*jpg dt-cli.strace.txt 7623 readlink("/tmp/symlinksourcedir/symlinksourcename.jpg", "../symlinkdestdir/symlinkdestname.jpg", 4095) = 37 7623 stat("symlinksourcedir/symlinkdestname.jpg", 0x7ffefd61de70) = -1 ENOENT (No such file or directory) 7623 open("symlinksourcedir/symlinkdestname.jpg", O_RDONLY) = -1 ENOENT (No such file or directory)
Interactive darktable is not affected:
[export_job] exported to `/tmp/symlinksourcedir/darktable_exported/symlinksourcename.jpg
#1 Updated by Stéphane Gourichon about 1 year ago
Regression commit identified¶
This bug is a regression because is did not happen on older version, and was observed on 2.4.0 on.
Tracing in code it looks like the bug was introduced in this commit:
commit 0da09d66610f9b7bd07783469d188b9beb323750 Author: Tobias Ellinghaus <firstname.lastname@example.org> Date: 2017-08-28 17:09:30 +0200 Windows: Try to normalize filenames during import On Windows there are a myriad different ways to refer to the same file thanks to the file systems not being case sensitive, different path separators ('/' vs. '\'), different naming schemes and probably more. We try to make filenames unique before importing to be able to figure out if we know an image already. There are cases where this doesn't work at the moment and we refuse to load those files for now. Feedback is more than welcome!
This commit introduces
normalized_filename and uses it a number of ways.
Alas, the following chunk causes downstream code to concatenate non-normalized path with the normalized name set in imgfname.
@@ -777,7 +786,7 @@ static uint32_t dt_image_import_internal(const int32_t film_id, const char *file uint32_t id = 0; // select from images; if found => return gchar *imgfname; - imgfname = g_path_get_basename((const gchar *)filename); + imgfname = g_path_get_basename(normalized_filename); sqlite3_stmt *stmt; DT_DEBUG_SQLITE3_PREPARE_V2(dt_database_get(darktable.db), "SELECT id FROM main.images WHERE film_id = ?1 AND filename = ?2", -1, &stmt, NULL);
This is one change that causes "symlinksourcedir/symlinkdestname.jpg" in repro case.
One might argue that the real trouble is when
dt_util_normalize_path which calls
g_realpath, that function calls
realpath which resolves symbolic links and thus replaces, in our example,
This commit seems to be motivated by Windows, which isn't concerned with symlinks, so it does not make much sense to fiddle with symlinks here.
Another point of view is that logic is broken anyway because it pastes a directory name with a mismatched file name, why not storing the whole name then?
Choosing one fix or the other will not affect this exact case but will affect other situations, so best to choose wisely. See you on IRC?