Project

General

Profile

Bug #11944

darktable-cli confused if image is a symlink

Added by Stéphane Gourichon about 1 month ago. Updated about 1 month ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
General
Target version:
-
Start date:
01/17/2018
Due date:
% Done:

0%

Affected Version:
git master branch
System:
Ubuntu
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Summary

When processing a picture that is a symbolic link, darktable-cli at some point tries to open the name of the symlink destination, from the directory of the symlink source. As a result, file is not found, image is not processed.

Interactive darktable is not affected when opening a directory.

How to reproduce

Prepare a simple layout

Here is the test layout:

lrwxrwxrwx 1 37 janv. 17 20:26 symlinksourcedir/symlinksourcename.jpg -> ../symlinkdestdir/symlinkdestname.jpg
-rw-rw-r-- 1 1459972 janv. 17 20:26 symlinkdestdir/symlinkdestname.jpg

Test layout can be reproduced by copy-pasting this:

cd /tmp ; rm -rf symlinksourcedir symlinkdestdir
mkdir -p symlinksourcedir symlinkdestdir
cp someimage.jpg symlinkdestdir/symlinkdestname.jpg
ln -s ../symlinkdestdir/symlinkdestname.jpg symlinksourcedir/symlinksourcename.jpg
ls -gG symlinksourcedir/* ; ls -gG symlinkdestdir/*

Run darkable-cli

darktable-cli symlinksourcedir/symlinksourcename.jpg out.jpg

Expected vs observed

Expected result

[export_job] exported to `out.jpg'

Observed result

allocation failed???
[imageio_storage_disk] could not export to file: `out.jpg'!

Reproducible

  • Always
  • Interactive darktable is not affected when opening a directory.
  • darktable-cli is observed to be affected from darktable 2.4.0, and on current master branch.

Attached trace file

Trace obtained with:

strace -fF -s 1024 -o dt-cli.strace.txt darktable-cli symlinksourcedir/symlinksourcename.jpg out.jpg

Indeed darktable-cli mixes source and destination: no symlinksourcedir/symlinkdestname.jpg exists.

grep source.*dest.*jpg dt-cli.strace.txt

7623  readlink("/tmp/symlinksourcedir/symlinksourcename.jpg", "../symlinkdestdir/symlinkdestname.jpg", 4095) = 37
7623  stat("symlinksourcedir/symlinkdestname.jpg", 0x7ffefd61de70) = -1 ENOENT (No such file or directory)
7623  open("symlinksourcedir/symlinkdestname.jpg", O_RDONLY) = -1 ENOENT (No such file or directory)

Interactive darktable is not affected:

[export_job] exported to `/tmp/symlinksourcedir/darktable_exported/symlinksourcename.jpg

dt-cli.strace.txt Magnifier (956 KB) Stéphane Gourichon, 01/17/2018 09:16 PM

History

#1 Updated by Stéphane Gourichon about 1 month ago

Regression commit identified

This bug is a regression because is did not happen on older version, and was observed on 2.4.0 on.

Tracing in code it looks like the bug was introduced in this commit:

commit 0da09d66610f9b7bd07783469d188b9beb323750
Author: Tobias Ellinghaus <me@houz.org>
Date:   2017-08-28 17:09:30 +0200

    Windows: Try to normalize filenames during import

    On Windows there are a myriad different ways to refer to the same file
    thanks to the file systems not being case sensitive, different path
    separators ('/' vs. '\'), different naming schemes and probably more. We
    try to make filenames unique before importing to be able to figure out
    if we know an image already. There are cases where this doesn't work at
    the moment and we refuse to load those files for now.
    Feedback is more than welcome!

Discussion

This commit introduces normalized_filename and uses it a number of ways.

Alas, the following chunk causes downstream code to concatenate non-normalized path with the normalized name set in imgfname.

@@ -777,7 +786,7 @@ static uint32_t dt_image_import_internal(const int32_t film_id, const char *file
   uint32_t id = 0;
   // select from images; if found => return
   gchar *imgfname;
-  imgfname = g_path_get_basename((const gchar *)filename);
+  imgfname = g_path_get_basename(normalized_filename);
   sqlite3_stmt *stmt;
   DT_DEBUG_SQLITE3_PREPARE_V2(dt_database_get(darktable.db),
                               "SELECT id FROM main.images WHERE film_id = ?1 AND filename = ?2", -1, &stmt, NULL);

This is one change that causes "symlinksourcedir/symlinkdestname.jpg" in repro case.

Possible fixes.

One might argue that the real trouble is when dt_image_import_internal calls dt_util_normalize_path which calls g_realpath, that function calls realpath which resolves symbolic links and thus replaces, in our example, symlinksourcename into symlinkdestname.

This commit seems to be motivated by Windows, which isn't concerned with symlinks, so it does not make much sense to fiddle with symlinks here.

Another point of view is that logic is broken anyway because it pastes a directory name with a mismatched file name, why not storing the whole name then?

Choosing one fix or the other will not affect this exact case but will affect other situations, so best to choose wisely. See you on IRC?

Also available in: Atom PDF