Project

General

Profile

Bug #10614

Duplicate 'ghost' images left when moving folders

Added by Robin Rainton about 4 years ago. Updated over 3 years ago.

Status:
Incomplete
Priority:
Low
Assignee:
-
Category:
Lighttable
Target version:
-
Start date:
08/24/2015
Due date:
% Done:

20%

Estimated time:
Affected Version:
1.6.7
System:
Fedora/RHEL
bitness:
64-bit
hardware architecture:
amd64/x86

Description

In the past (read: with this same release) I have been able to arrange images within Darktable's lighttable section without problem.

Sometimes I would like to move some images from one folder (physically on the filesystem) to another (new) folder. What I do is highlight the images in lighttable, hit 'move' over on the right, navigate to the destination parent, create a new folder and voila! The images are no longer in the original location and appear (all automatically collected) in the new folder.

The last couple of times I have tried to do this though something has gone wrong. Darktable does successfully move the physical files on the disk and it does show this new folder automatically collected. However, it leaves what would appear to be 'ghost' duplicates in the source.

To be clear: the files are physically moved (they are NOT copied and there is only one physical version of them) but Darktable shows them in two locations in lighttable. It shows them still in the source as well as in the new destination using the 'folders' view.

When one tries to remove these 'ghosts' by selecting them in the original location and hitting 'remove' the destination images disappear too. Trying to re-import just the destination folder causes them to appear in the source too.

Basically - I now have one physical copy of these images yet they show twice in lighttable.

I tried removing the entire directory tree and re-importing but again, these images were duplicated.

Screenshot from 2015-08-24 20-37-30.png (456 KB) Screenshot from 2015-08-24 20-37-30.png Correct 'Flydalsjuvet' folder Robin Rainton, 08/24/2015 09:40 PM
Screenshot from 2015-08-24 20-37-46.png (695 KB) Screenshot from 2015-08-24 20-37-46.png Incorrect 'generic' folder with local image Robin Rainton, 08/24/2015 09:40 PM
Screenshot from 2015-08-24 20-37-48.png (696 KB) Screenshot from 2015-08-24 20-37-48.png Incorrect 'generic' folder with 'ghost' image Robin Rainton, 08/24/2015 09:42 PM

History

#1 Updated by Robin Rainton about 4 years ago

I attach three screenshots to illustrate the point:

1. Selection of the 'Flydalsjuvet' folder, correctly showing the images that are in that folder (image 0510 highlighted). Note that on the left the filmroll also shows them as in this folder.

2. 'generic' folder (it's just the date) showing image number 0509 which is local to the folder. Ie. this is in the correct place. However, see how the images that are in the 'Flydalsjuvet' folder are also shown here.

3. 'generic' folder (it's just the date) showing image number 0510 which is not even in this folder. I'm calling it a 'ghost'. As has been shown by first screenshot it's in the 'Flydalsjuvet' folder. Note that on the left the filmroll shows the image is in the 'Flydalsjuvet' roll and even shows it in that physical location on the disk. So why is it still shown in this folder?

#2 Updated by Robin Rainton about 4 years ago

Robin Rainton wrote:

I tried removing the entire directory tree and re-importing but again, these images were duplicated.

I also tried removing the library.db file and re-importing. This didn't help. It also removed all my presets (which I found a little annoying so raised #10615).

#3 Updated by Robin Rainton over 3 years ago

Robin Rainton wrote:

Sometimes I would like to move some images from one folder (physically on the filesystem) to another (new) folder.

I should add that I have given up trying to use Darktable for moving files and now use the O/S explorer. However, this problem occurs when doing that too.

Ie:

- Import a folder with lots of images.
- Decide there are too many in this one folder and that you will split some out.
- In the file explorer select a few images and move them to a new folder.
- Import the new folder in Darktable.
- Within Darktable 'folder' view images appear in both the new folder, and are shown in the old folder still. Despite the fact they are not physically in that folder.
- Removing/re-importing the old folder doesn't make any difference.

#4 Updated by Pascal Obry over 3 years ago

Moving via the OS explorer is indeed a recipe for a good mess :)

So better to try the new version darktable 2.0. For the record I'm moving files as you do often but never had this issue!

#5 Updated by Robin Rainton over 3 years ago

Pascal Obry wrote:

Moving via the OS explorer is indeed a recipe for a good mess :)

Well, yes, but... Darktable team is very adamant that they do not want to implement a file manager within Darktable so what choice is left?

#6 Updated by Pascal Obry over 3 years ago

Learn to use darktable which can move and delete pictures. What else? Sometime it is important to change habits to fit the software. And note that this is NOT darktable only, I come from Lightroom where the same could be said (at least up to version 3.x). Frankly what darktable propose is just fine to me, but maybe that's my workflow... I really don't care about the filename, the important feature is tag on the picture. I could live with picture being named 1.nef, 2.nef, 3.nef... I'm not expecting any meaning from filename.

#7 Updated by Robin Rainton over 3 years ago

Pascal Obry wrote:

Learn to use darktable which can move and delete pictures.

I don't understand your point.

I have accepted one must move pictures outside of Darktable, but... I have this problem when the same physical image appears in two folders within Darktable after a move when on the filesystem it is only in one of these folders.

#8 Updated by Roman Lebedev over 3 years ago

  • % Done changed from 0 to 20
  • Status changed from New to Incomplete

Robin Rainton wrote:

Pascal Obry wrote:

Learn to use darktable which can move and delete pictures.

I don't understand your point.

I have accepted one must move pictures outside of Darktable, but... I have this problem when the same physical image appears in two folders within Darktable after a move when on the filesystem it is only in one of these folders.

So, you have moved a file outside of darktable, either using mv, or some file manager like dolphin/etc, NOT using darktable "move" function, correct?

How do you suppose darktable should find out that you did that?

Either do not move files behind darktable's back, or use darktable "move" function, or move images BEFORE importing them into darktable, or use https://github.com/darktable-org/darktable/blob/master/tools/purge_non_existing_images.sh

#9 Updated by Robin Rainton over 3 years ago

Roman Lebedev wrote:

So, you have moved a file outside of darktable, either using mv, or some file manager like dolphin/etc, NOT using darktable "move" function, correct?

How do you suppose darktable should find out that you did that?

By removing and re-importing the folder.

Only when I do this for both source and destination folder Darktable thinks the images exist in both places for some reason, not just in the destination which would be the correct place.

#10 Updated by Roman Lebedev over 3 years ago

Robin Rainton wrote:

Roman Lebedev wrote:

So, you have moved a file outside of darktable, either using mv, or some file manager like dolphin/etc, NOT using darktable "move" function, correct?

How do you suppose darktable should find out that you did that?

By removing and re-importing the folder.

Are you positively sure that you did actually remove all the images from that folder from darktable library BEFORE re-importing?

Only when I do this for both source and destination folder Darktable thinks the images exist in both places for some reason, not just in the destination which would be the correct place.

Are you sure you have actually moved the files, not copied them? Does ls show them in old place, or in both places?

#11 Updated by Robin Rainton over 3 years ago

Roman Lebedev wrote:

Are you sure you have actually moved the files, not copied them?

Yes.

Does ls show them in old place, or in both places?

No.

#12 Updated by Roman Lebedev over 3 years ago

Robin Rainton wrote:

Roman Lebedev wrote:

Are you sure you have actually moved the files, not copied them?

Yes.

Does ls show them in old place, or in both places?

No.

Ok.

One more time,
Are you absolutely positively sure that you did actually remove all the images from that folder from darktable library BEFORE re-importing?
Please double check.

#13 Updated by Robin Rainton over 3 years ago

Roman Lebedev wrote:

One more time,
Are you absolutely positively sure that you did actually remove all the images from that folder from darktable library BEFORE re-importing?
Please double check.

Yes, I am absolutely sure I moved the files and did not copy them. Believe me, this is very frustrating to say the least, and especially so when your actions are questioned. Do you think it's fun to report a bug in as much detail as I have done here?

All that said - I have just run another test with completely new images in the /tmp folder of my machine and after removing the source and re-importing it there are no 'ghosts'.

I have absolutely guaranteed seeing this problem again since reporting it and absolutely guaranteed I moved, not copied the images.

My conclusion is that this problem is intermittent at best.

I'm sorry if you don't like reports of problems that are intermittent. I am a developer myself. I know these kinds of problems are annoying and very hard to track down. All I can do is report what I see in the hope it will help someone else.

#14 Updated by Roman Lebedev over 3 years ago

Robin Rainton wrote:

Roman Lebedev wrote:

One more time,
Are you absolutely positively sure that you did actually remove all the images from that folder from darktable library BEFORE re-importing?
Please double check.

Yes, I am absolutely sure I moved the files and did not copy them.

But that was not the question...

Believe me, this is very frustrating to say the least, and especially so when your actions are questioned. Do you think it's fun to report a bug in as much detail as I have done here?

All that said - I have just run another test with completely new images in the /tmp folder of my machine and after removing the source and re-importing it there are no 'ghosts'.

I have absolutely guaranteed seeing this problem again since reporting it and absolutely guaranteed I moved, not copied the images.

My conclusion is that this problem is intermittent at best.

I'm sorry if you don't like reports of problems that are intermittent. I am a developer myself. I know these kinds of problems are annoying and very hard to track down. All I can do is report what I see in the hope it will help someone else.

#15 Updated by Pascal Obry over 3 years ago

Robin, don't take this so badly. We are also developers :) And also users, I'm always using the move from darktable and never seen this behavior since years! So either I'm very lucky or there is something specific to your setting that we need to understand for fixing this issue if any.

See Roman question which you did not clearly answered.

Also, not sure now if you have tested dt 2.0? Do you reproduce with this latest version?

#16 Updated by Robin Rainton over 3 years ago

Pascal Obry wrote:

See Roman question which you did not clearly answered.

I answered it as clearly as I could. Which was basically... it seems intermittent and I don't see the cause as yet.

Also, not sure now if you have tested dt 2.0? Do you reproduce with this latest version?

I'm on FC22 and the 2.0 package doesn't seem to be available yet.

As for moving within DT, this is pretty hideous. I found threads here where people complain about that and where the DT team reply that they have better things to do than implement a file manager and recommend that an external file manager be used. So I did that. And found this bug.

#17 Updated by Roman Lebedev over 3 years ago

Robin Rainton wrote:

Pascal Obry wrote:

See Roman question which you did not clearly answered.

I answered it as clearly as I could. Which was basically... it seems intermittent and I don't see the cause as yet.

I have asked 3 (three) questions in http://redmine.darktable.org/issues/10614#note-10
You have answered to only last 2 (two) of them.

Also, not sure now if you have tested dt 2.0? Do you reproduce with this latest version?

I'm on FC22 and the 2.0 package doesn't seem to be available yet.

As for moving within DT, this is pretty hideous. I found threads here where people complain about that and where the DT team reply that they have better things to do than implement a file manager and recommend that an external file manager be used. So I did that. And found this bug.

Also available in: Atom PDF

Go to top