Project

General

Profile

Bug #9578

darktable does neither ignore nor hide hidden folders

Added by Chaos 99 about 6 years ago. Updated about 6 years ago.

Status:
Incomplete
Priority:
Low
Assignee:
-
Category:
General
Target version:
-
Start date:
09/08/2013
Due date:
% Done:

20%

Estimated time:
Affected Version:
git development version
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Being written for Linux & OSX, ignoring or hiding hidden folders (dot-folders, folderst starting with a dot) during import and
when viewing collections by folder should be standard.

Is this purposefully ignored or a bug?

I'm using nfs-mounted folders. Does this play a role in this?

_2e_IMGP9624.PEF (4 KB) _2e_IMGP9624.PEF Chaos 99, 09/15/2013 01:00 PM

Associated revisions

Revision a1fe583a (diff)
Added by Tobias Ellinghaus about 6 years ago

Don't try to import empty file

See bug #9578.

Revision 26276765 (diff)
Added by Tobias Ellinghaus about 6 years ago

Bug #9578: Don't import dot folders

Dot files are still imported though ...

Revision 48d74985 (diff)
Added by Tobias Ellinghaus about 6 years ago

Fix #9578: This time also for dot files

Revision ecb8bbdf (diff)
Added by Tobias Ellinghaus about 6 years ago

Don't try to import empty file

See bug #9578.
(cherry picked from commit a1fe583a1f43473b07befb7c56fd38b92df6b34e)

Revision e1e81d1d (diff)
Added by Tobias Ellinghaus about 6 years ago

Don't try to import empty file

See bug #9578.
(cherry picked from commit a1fe583a1f43473b07befb7c56fd38b92df6b34e)

Revision a9179447 (diff)
Added by Tobias Ellinghaus about 6 years ago

Bug #9578: Don't import dot folders

Dot files are still imported though ...
(cherry picked from commit 26276765d30664b4872df7a6d4c7efd97eb0db3e)

Revision 2073d8ba (diff)
Added by Tobias Ellinghaus about 6 years ago

Fix #9578: This time also for dot files
(cherry picked from commit 48d74985fa2e2954ea640012701d37799106e406)

History

#1 Updated by Tobias Ellinghaus about 6 years ago

  • % Done changed from 0 to 20
  • Status changed from New to Incomplete

I am not sure what you mean. Don't you want to see them in the import dialog? Or do you actually import them but don't want to see the filmrolls on the lighttable afterwards?

#2 Updated by Chaos 99 about 6 years ago

I want darkroom to behave like any other linux tool: ignore them.

Don't show them in any dialog (except explicitly told so).
Don't import anything from them (except explicitly ordered to do so).

If that's not possible and you somehow have to import them (to have consistent model of the underlying file system for instance),
then don't show them in the folder/filmroll view.

My use case: importing a large folder structure with various sub-folders with the 'import recursively' switch set.
Gives me 400k+ images extra from assorted hidden thumbnail folders set by other programs.

Also doesn't help that you can't mark more than one folder to remove from the DB.
Also doesn't help that darkroom does no sanity check if it really is an image it is importing. You can skip 0 byte files ...

#3 Updated by Chaos 99 about 6 years ago

Same goes for hidden files.
Don't try to import/display a .DSC083627.JPG. It's probably not a real image file, but a shadow copy or backup.

#4 Updated by Tobias Ellinghaus about 6 years ago

  • System changed from other GNU/Linux to all
  • Affected Version changed from 1.2.2 to git development version
  • Status changed from Incomplete to Triaged

Chaos 99 wrote:

I want darkroom to behave like any other linux tool: ignore them.

I guess you mean darktable ...

Don't show them in any dialog (except explicitly told so).
Don't import anything from them (except explicitly ordered to do so).

If that's not possible and you somehow have to import them (to have consistent model of the underlying file system for instance),
then don't show them in the folder/filmroll view.

My use case: importing a large folder structure with various sub-folders with the 'import recursively' switch set.
Gives me 400k+ images extra from assorted hidden thumbnail folders set by other programs.

Ok, I see.

Also doesn't help that you can't mark more than one folder to remove from the DB.

Just use the collect module to get all filmrolls starting with a '.', hit ctrl-a and then del. Done.

Also doesn't help that darkroom does no sanity check if it really is an image it is importing. You can skip 0 byte files ...

Probably, yes.

#5 Updated by Chaos 99 about 6 years ago

I guess you mean darktable ...

Yes. Sorry. Got stuck halfway between lightroom and darktable

Also doesn't help that you can't mark more than one folder to remove from the DB.

Just use the collect module to get all filmrolls starting with a '.', hit ctrl-a and then del. Done.

How do I specify "starts with"? Just entering '.' will bring up everything containing a '.'.
Does the filter input follow any standard syntax? RegEx? Bash?

#6 Updated by Tobias Ellinghaus about 6 years ago

  • % Done changed from 20 to 100
  • Status changed from Triaged to Fixed

Chaos 99 wrote:

I guess you mean darktable ...

Yes. Sorry. Got stuck halfway between lightroom and darktable

:)

Also doesn't help that you can't mark more than one folder to remove from the DB.

Just use the collect module to get all filmrolls starting with a '.', hit ctrl-a and then del. Done.

How do I specify "starts with"? Just entering '.' will bring up everything containing a '.'.
Does the filter input follow any standard syntax? RegEx? Bash?

That's SQL. Just "/." works for me.

BTW, this is fixed in git now. Empty files, folders starting with a dot and dot files are ignored.

#7 Updated by Chaos 99 about 6 years ago

Tried the git version. Seems fixed as described.

Although there are still files imported that are not images.
But as the bug was about hidden files, this is off topic here.

Thanks!

#8 Updated by Tobias Ellinghaus about 6 years ago

Oh, what kind of files manages to sneak in?

#9 Updated by Chaos 99 about 6 years ago

Several other tools have access to the (network) file system.
Seems that at least one MacOS tool (iPhoto? Lightroom?) leaves behind files following the naming scheme
":2e_<original_file_name>".

The file ending is still .JPG, the files are non-empty binary files with either 8 Byte or 16 Byte but are not image data.
Darktable can't display them.

#10 Updated by Tobias Ellinghaus about 6 years ago

I guess there is nothing we can do about that. It's named like a regular image file and isn't empty. So the skull is all we can do about it i guess.

#11 Updated by Jérémy Rosen about 6 years ago

in your particular case you can write a small lua script to reject these images at impor time, but I agree with houz that there is no simple way for DT to deal with that...

#12 Updated by Tobias Ellinghaus about 6 years ago

  • % Done changed from 100 to 50
  • Status changed from Fixed to In Progress

Hmm, could you attach one or two of these files to this ticket? Maybe we can find out that we should ignore these files from some magic number. But I need something to play with so I can test my code.

#13 Updated by Tobias Ellinghaus about 6 years ago

  • % Done changed from 50 to 100
  • Status changed from In Progress to Fixed

Applied in changeset darktable|commit:2073d8ba30694ecea354b063ddd1d13596ee1fe0.

#14 Updated by Tobias Ellinghaus about 6 years ago

  • % Done changed from 100 to 20
  • Status changed from Fixed to Incomplete

Redmine, stop doing that.

#15 Updated by Chaos 99 about 6 years ago

Sorry for the long wait.

I've attached one of these files. I've found out that they are files generated by my network storage system when sharing data via afp
to hold the information that would otherwise be in the extended file system record of the original file on a HSF+ file system.

Most image formats have magic numbers that make them unique.
Aren't you better off with a white list instead of a black list? Darktable probably only supports a small number of file types anyway.

Once you've found that you can't decode them (probably by some library code returning nil, whatever you use to decide to display the skull), you could also just offer to remove all those files
from the collection right after the input without worrying about white/black lists at all.

Just my 2 cents ..

Also available in: Atom PDF

Go to top