Project

General

Profile

Feature #12588

File names - display and modify

Added by Claude Jolicoeur 18 days ago. Updated 16 days ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
General
Target version:
-
Start date:
02/05/2019
Due date:
% Done:

0%

Affected Version:
git master branch
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

I am new user of DarkTable (I previously used Canon DPP, but now with a Sony camera, seeking an alternative).
Although I find DarkTable quite nice on many aspects, however the fact that the program completely ignores filenames makes it much less useful. I am really stunted by this lack of functionality from an otherwise quite complete piece of software.
This has been requested many times already (see feature requests #9875 5 years ago, #11795, #12184) and everytime the developers sort of dismissted it with, in my opinion, very weak arguments. Obviously the developers don't want to implement filenames in this program!
Yes the filename is visible in the left panel, but this is not very useful compared to having it directly on the thumbnail image (see #11795 for a nice example)
The developers have also mentioned in #9875: "all other users are plagued with continuously seeing the name without not the slightest benefit of it." - then you simply need to make it optional with the configuration!
Additionally, we need the possibility to rename a file, in a way that the associated .xmp file would also be automatically renamed (would be nice with a right clik on the thumbnail image).
And finally, this program needs a good implementation of a batch rename routine which would let us add parameters like an automatically incrementing index or the date. See for example XnView for an implementation of a powerful batch rename routine.

History

#1 Updated by Pascal Obry 18 days ago

Why do you want to see filenames? You should never go with the explorer to handle them otherwise you'll break the sync with dt database. I keep amazed by the need to have filename... dt is non destructive, so the file is not useful at all. You export the picture in the specific location and then you know where the interesting picture is.

#2 Updated by Claude Jolicoeur 18 days ago

I guess everyone has its own way of keeping a classification of photos. For my part, I rely heavily on filename (and I don't think I am the only one).
I can understand that many people using DarkTable have other ways of keeping order in their photos that doesn't rely on filenames. However, I think a great piece of software should adapt to many different users who may want to use it differently.
For example if I go shooting at a wedding, all the photos will be in a folder named 2019-02-05_Wedding, then the individual photos will be renamed according to the subject, for example Bride01, Bride02..., Kiss01, Kiss02, ChurchExit01, and so on... I have been using this type of sorting for 15 years, and not about to change.
Normally, in all other photo editing software I have seen (and I have seen quite a few), there is a way to interact with filenames, see them, change them...
So if I keep on using DarkTable, this means I have to do all this maintenance in another program like XnView - but then once I import in DarkTable, it becomes very ackward to change the filename again. It means I'll have to remove from DT, rename the raw and the xmp files, and reimport in DT. This is far from being an elegant solution...

#3 Updated by Pascal Obry 18 days ago

For your workflow, it seems that a better solution is to use tags. They get recorded into the database and XMP and permit lot better classification than some words in filenames.

This has been debated many many times, and dt is just not a file-manager and will certainly never be. But if someone come with a patch to rename a filename from the collection, why not. This is certainly a bit tricky as it requires many actions: name of file, name of XMP, name in database, name of duplicates....

#4 Updated by Claude Jolicoeur 17 days ago

Pascal Obry wrote:

For your workflow, it seems that a better solution is to use tags.

Yes, you are suggesting that I change my well proven way of working. I will not change a collection of many 1000s of photos because a program doesn't let me do what I want. I'll rather change the program. As I said, each should have the possibility of working as he desires!

This is certainly a bit tricky as it requires many actions: name of file, name of XMP, name in database, name of duplicates....

Yes, and this is why it would be better if done from inside DarkTable rather than with some other program.

There is already a "move" function for a set of selected images. It wouldn't be much more difficult to implement a "rename" function.
And for the display, it would be so simple to just add an option in the configuration to the effect of displaying filenames in thumbnails.

Any good programmer could implement that in a few hours. I simply don't understand why it hasen't been done yet, as it is so basic. I am sure you have lost a good number of users because of that.

#5 Updated by Pascal Obry 17 days ago

Yes, you are suggesting that I change my well proven way of working.

No, I'm just suggesting to adopt the dt way of working, this has been a design choice for now. If you can't adapt then you can use another software. No problem.

Any good programmer could implement that in a few hours.

You are missing lot of background and I can tell you that this is not few hours. It is quite different to moving a "collection" than renaming a file and I pretty well know the dt internals.

#6 Updated by Claude Jolicoeur 17 days ago

Pascal Obry wrote:

No, I'm just suggesting to adopt the dt way of working, this has been a design choice for now. If you can't adapt then you can use another software. No problem.

I had no big expectations when I decided to raise this point. Obviously the DT design and development team doesn't want users to play with filenames. This really is a philosophical issue much more that a technical one. But open your eyes... It is not because you don't think interaction with filenames is necessary that everyone else should think the same way. You said it: "This has been debated many many times"... plus there has been quite a few requests to this effect through the years. So, clearly, there are many people for which such a functionality would be very useful.

Now I agree that it would take more than just a few hours of work. But I am sure that technically it is very feasable and not so difficult to implement. My opinion (for what it's worth) is that you are loosing many potential users because of that. Maybe you simply don't care?

I would suggest that you add in the web page something like this in order to make things clear for all potential users of DarkTable:
WARNING:
The desing and development team of DarkTable has made a design choice to the effect that a filename is not something a user should be allowded to change. Hence if you decide to use this software, be warned that you will not be able to change a filename and that it is even not so obvious to simply see the filename. So, if filenames are important for you, don't loose your time here and look at some other software for working with your photo files.

#7 Updated by Pascal Obry 17 days ago

  • Affected Version changed from 2.4.4 to git master branch

Indeed I don't care because I don't need this and I prefer to concentrate on features I want/use. That's the Open Source way :) For things to do that I don't like I have my paid day joy :) But as I said I'm not against it if someone come with a good patch.

Loosing users, maybe? But well we all have different expectations and there is different software with different features, I suppose people will just go with the software that fit more their needs, that's liberty and it is nice to have choices.

#8 Updated by Aurélien PIERRE 17 days ago

Claude Jolicoeur wrote:

Pascal Obry wrote:

No, I'm just suggesting to adopt the dt way of working, this has been a design choice for now. If you can't adapt then you can use another software. No problem.

I had no big expectations when I decided to raise this point. Obviously the DT design and development team doesn't want users to play with filenames. This really is a philosophical issue much more that a technical one. But open your eyes... It is not because you don't think interaction with filenames is necessary that everyone else should think the same way. You said it: "This has been debated many many times"... plus there has been quite a few requests to this effect through the years. So, clearly, there are many people for which such a functionality would be very useful.

Now I agree that it would take more than just a few hours of work. But I am sure that technically it is very feasable and not so difficult to implement. My opinion (for what it's worth) is that you are loosing many potential users because of that. Maybe you simply don't care?

I would suggest that you add in the web page something like this in order to make things clear for all potential users of DarkTable:
WARNING:
The desing and development team of DarkTable has made a design choice to the effect that a filename is not something a user should be allowded to change. Hence if you decide to use this software, be warned that you will not be able to change a filename and that it is even not so obvious to simply see the filename. So, if filenames are important for you, don't loose your time here and look at some other software for working with your photo files.

darktable treats the raw file as a master and never touch it anymore after it is imported. The thing is there is a risk of corrupting the file if something happens during the handling, and since Windows, Linux, and Mac OS have different ways of handling filenames and pathes (especially, special characters and such), it is not safe to do it inside darktable, so we discourage it and leave it to your file manager if you really need to do it.

A raw picture can't be displayed out of a raw processing software, so I wonder why you need the file renamed on your system, since you need the raw soft anyway to open it. As Pascal suggested, use tags.

I garantee you will be more angry after us if you lose a raw during renaming than you are, learning you can't do it.

#9 Updated by Nicolas Auffray 17 days ago

Claude Jolicoeur wrote:

I had no big expectations when I decided to raise this point. Obviously the DT design and development team doesn't want users to play with filenames. This really is a philosophical issue much more that a technical one. But open your eyes... It is not because you don't think interaction with filenames is necessary that everyone else should think the same way. You said it: "This has been debated many many times"... plus there has been quite a few requests to this effect through the years. So, clearly, there are many people for which such a functionality would be very useful.

I'm not a developer, just a user who tries to help and never forgot that DT design and development team is made (as a lot of open source software) by people in just their free time. So I'm just surprised on the way many people ask features the same way it was a paid software made by people paid for developing it. If I decide to make something in my free time, as many people, is for things I like and I want, I've decide to make. As you said, open your eyes about this. So, if the development team prefer using their time to develop something else, it's their choise. That's the freedom of open source, also for developer. Yes, it's normal that this cause frustration, but the only way to have a feature wanted is first try to develop it or, if you can't, don't know how, find someone ok to develop it. With the possibility that no one will be ok for that.

So as Pascal said : But well we all have different expectations and there is different software with different features, I suppose people will just go with the software that fit more their needs, that's liberty and it is nice to have choices.

I will just add that none software will totally meet your needs, even paid one. Like a lot of things in life, they have lot of things to do, on their life and on darktable, so they have to prioritize the features/bugs they are ok to use their time.

#10 Updated by Claude Jolicoeur 17 days ago

It is funny, everybody has French names here and we are discussing in English...

Aurélien PIERRE wrote:

darktable treats the raw file as a master and never touch it anymore after it is imported. The thing is there is a risk of corrupting the file if something happens during the handling, and since Windows, Linux, and Mac OS have different ways of handling filenames and pathes (especially, special characters and such), it is not safe to do it inside darktable, so we discourage it and leave it to your file manager if you really need to do it.

I don't follow here. In any case, you would do the rename with a call to the OS. So it is really the OS who does the renaming. There is no more risk than if the rename is requested by the file manager.

A raw picture can't be displayed out of a raw processing software, so I wonder why you need the file renamed on your system, since you need the raw soft anyway to open it. As Pascal suggested, use tags.

Sorry, but I can display raw files with Irfanview, XnView, and even with the file manager when the appropriate driver is installed. Just to name a few options.

I garantee you will be more angry after us if you lose a raw during renaming than you are, learning you can't do it.

I disagree. I always have backups. Plus the original would probably still be on the camera card!

and Pascal wrote:

But as I said I'm not against it if someone come with a good patch.

I can't do it - I haven't done any programming for 25 years, and I am pretty rusted. However I see how it could be implemented.
First, I would take an existing routine as model. There is already a routine to delete a list of files. This could be easily modified to make it a renaming routine.
Then I understand there is a database somewhere, so that each imported photo has a record in that database, and I would then assume there is a field in that record for the filename. And this is probably indexed.

I am pretty sure the Delete routine works as follows:
- Get a list of filenames to delete. // possibly given as an argument
- For each filename in the list, do the following:
- Call OS with a DeleteFile command for "filename"
- Call OS with a deleteFile command for "filename".xmp // for the associated file
- Call Database Engine to seek record corresponding to "filename"
- Call Database Engine to delete that record
- Call Database Engine to reindex // optional, possibly done automatically by DB engine
- Check if all items on the list have been processed
- Done.

Now let's modify that routine to make it a rename routine. It would now go as follows:
- Get list of oldfilenames to rename
- Get list of newfilenames
- For each item in the list of oldfilenames, do the following:
- Call OS with a RenameFile command for "oldfilename" to "newfilename"
- Call OS with a RenameFile command for "oldfilename".xmp to "newfilename".xmp
- Call Database Engine to seek record corresponding to "oldfilename"
- Call Database Engine to replace the content of filename field to "newfilename"
- Call Database Engine to reindex // optional, possibly done automatically by DB engine
- Check if all items on the list have been processed
- Done.

And that would be it. How much time would it take to implement? Probably less than we have spent discussing.
We would still need a dialog to specify the new file name. If someone is ready to help, we can then discuss the dialog.

#11 Updated by Aurélien PIERRE 17 days ago

There is no "call OS" in C. There is only "handle bytes yourself" in C and check that you the OS is happy with it and deal with exceptions yourself. That's what makes things difficult and unreliable. To delete files, darktable has actually 3 different functions doing different things, one for each OS.

For your information, we still have bugs related to the way Windows handles directory pathes (\dir\subdir) because dt defaults to Unix pathes (/dir/subdir). So what reliability do you expect from a file renaming ?

Beside, we would have to handle remote files too (NAS, servers, etc.), meaning backups and checks to handle cases where the connexion is lost or reset and the transaction is aborted. So, all in all, let people do that in their file browser.

Things always look simple until you actually try them.

#12 Updated by Claude Jolicoeur 17 days ago

Aurélien PIERRE wrote:

There is no "call OS" in C. There is only "handle bytes yourself" in C and check that you the OS is happy with it and deal with exceptions yourself. That's what makes things difficult and unreliable. To delete files, darktable has actually 3 different functions doing different things, one for each OS.

You are serious? DarkTable is written in plain C? Wow... I remember having used plain C about 30 years ago, it is a real nightmare, and I wouldn't want to touch it again. I would have thought you used some of those modern development environment... But now aren't there libraries that can handle those calls to OS more easily?

Anyway, yes, I know I was oversimplifying...

So, all in all, let people do that in their file browser.

OK. Let's assume I do it outside of DT. What would be the way to do it without fucking things with the database?
Would I need to remove the files from the library before? (enlever images sélectionnées in the French version)
Then close DT, rename file(s) and associate xmp, reopen DT, and import the file(s) under new name.
Would all this be sufficient/necessary to keep things clean?

#13 Updated by Aurélien PIERRE 17 days ago

C is a real nightmare, but it's also the best we have to make pixel processing fast, work on CPU and GPU, interface them both, possibly use them in parallel, and take advantage of every CPU architecture to vectorize pixels and parallelize loops.

OK. Let's assume I do it outside of DT. What would be the way to do it without fucking things with the database?
Would I need to remove the files from the library before? (enlever images sélectionnées in the French version)
Then close DT, rename file(s) and associate xmp, reopen DT, and import the file(s) under new name.
Would all this be sufficient/necessary to keep things clean?

That sounds right.

#14 Updated by Pascal Obry 17 days ago

The issue is not only the renaming. This part is not difficult indeed. But what has to be done (in case someone want to look at this):

1. rename the RAW file
2. rename the XMP file
3. find and rename the XMP for the duplicate
4. rename the local copy if any
5. make sure the main RAW and XMP are available (that is the external drive is connected)
6. change the filename reference in the database
7. change the reference of the RAW in each XMP (main one and for the duplicates)

Note also that inevitably you've broken the backups done by the user.

And all this should be done atomically, so either all is done correctly or nothing should happen otherwise you desynchronize the data and so break user's precious devs.

You may still think that this is easy? I don't.

#15 Updated by Claude Jolicoeur 16 days ago

Aurélien PIERRE wrote:

C is a real nightmare, but it's also the best we have to make pixel processing fast, work on CPU and GPU, interface them both, possibly use them in parallel, and take advantage of every CPU architecture to vectorize pixels and parallelize loops.

OK! But I am nevertheless quite surprised by this. Yes I can easily understand that plain C is the best for critical pixel calculations. However I would have thought these routines could have been compiled as a library, and then the rest of DT could have been written in a more friendly higher level language (i.e. for the user interface, file manipulation, menus, and so on) that would call these C routines. I am not saying that as a critic (I don't know enough) but it is just that I find this somewhat surprising.

And to Pascal:
Yes I now see that DT handles duplicates, local copies and other details that make this more difficult.
For the backups however they would get automatically updated the next time the user backs-up his data (in a similar way than any other file the user modifies).

Question: When one removes an image from the library, does it clear the database from all records relative to this image? Also I suppose that when the image is removed, all files are closed and handles returned to the OS, right?

#16 Updated by Pascal Obry 16 days ago

For the backups however they would get automatically updated the next time the user backs-up his data

And you keep both old names and new names? "updated" above is a simple word but there is lot of implications as we've seen.

Now, this will be my last message on this thread. I have I think described what is to be done. I have said that this is very tricky to get right. I'm just waiting for the patch :)

Also available in: Atom PDF