Project

General

Profile

Camera Support #12628

exiv2 version 0.26 or later in darktable (under ubuntu)

Added by Claude Jolicoeur 2 months ago. Updated 2 months ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
-
Target version:
-
Start date:
03/14/2019
Due date:
% Done:

0%

Affected Version:
2.4.4

Description

Starting with version 0.26 of exiv2, there is a useful feature that permits to specify your own lens in a configuration file for a given lens ID number that is written in the exif data of a photo. This is mostly useful when exiv2 doesn't identify correctly a lens but lensfun does provide the correction data for it.
Now, Pascal's PPA for ubuntu only provides version 0.25 of exiv2, and this is the version built in darktable.
I can download from exiv2 website the latest version (0.27) but I can't find a way to make darktable work with this newer version of exiv2.

So I am wondering if there is a way to make this exiv2 configuration file work with darktable.
Thanks,
Claude

Bildschirmfoto von 2019-03-14 09-40-15.png - stonehenge, that works fine (65.3 KB) David Schaefer, 03/14/2019 10:41 AM

Bildschirmfoto von 2019-03-14 09-40-32.png - my own shot with a Sigma 24-70mm F2,8 DG OS HSM Art (65.4 KB) David Schaefer, 03/14/2019 10:41 AM

History

#1 Updated by David Schaefer 2 months ago

Hi Claude,

yesterday I tried the same because my new Sigma Art 2.8 24-70 is not recognized inside darktable. I've heard it is a exiv2 problem and quickly found the configuration file described here: http://dev.exiv2.org/projects/exiv2/wiki/Lens_Recognition_in_Exiv2_v026_(and_later)/

I then have compiled the latest exiv2 sourcecode and installed it (/usr/local/bin, /usr/local/include/, /usr/local/lib/). I verified that I am running exiv2 with:

# exiv2 --version

exiv2 0.27.99.0

# which exiv2

/usr/local/bin/exiv2

Like you said it doesnt work for me inside darktable. But it also does not work on the exiv2 level. So can you please assure the following to be working on your side?

Create ~/.exiv2 s.t.:
# cat ~/.exiv2

 
[nikon]
146=Robin's Sigma Lens  <--- The name of your lens

run
# exiv2 -pv --grep lens/i http://clanmills.com/Stonehenge.jpg
and verify this:

# exiv2 -pa --grep lens/i ~/Stonehenge.jpg

Exif.Nikon3.LensType                         Byte        1  D G VR
Exif.Nikon3.Lens                             Rational    4  18-250mm F3.5-6.3
Exif.Nikon3.LensFStops                       Undefined   4  4.58333
Exif.NikonLd3.LensIDNumber                   Byte        1  Robin's Sigma Lens  <--- Will be reported
Exif.NikonLd3.LensFStops                     Byte        1  F4.6

On my side, on which I dont blame dt for not working, I have that result:
# exiv2 -pv --grep lens/i http://clanmills.com/Stonehenge.jpg

0x0083 Nikon3       LensType                    Byte        1  14
0x0084 Nikon3       Lens                        Rational    4  180/10 2500/10 35/10 63/10
0x008b Nikon3       LensFStops                  Undefined   4  55 1 12 0
0x000c NikonLd3     LensIDNumber                Byte        1  146
0x000d NikonLd3     LensFStops                  Byte        1  55

So you see the LensIDNumber was not replaced by the configuration files string.

#2 Updated by Andreas Schneider 2 months ago

I don't see any darktable issue here. If you want to use new features for exiv2 you have to build darktable against it.

If you're distribution doesn't provide the latest version, try to use a 3rdparty repo which normally has newer dependencies:

https://www.darktable.org/install/

#3 Updated by David Schaefer 2 months ago

@Andreas, I don't completely agree. In general you are right but I think there is a bug:

# which exiv2

/usr/local/bin/exiv2

# ldd $(which exiv2)

linux-vdso.so.1 (0x00007ffd4bdb4000)
libexiv2.so.27 => /usr/local/bin/../lib/libexiv2.so.27 (0x00007f8287595000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f82873d7000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f82873bd000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f82871d3000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f82871cd000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f8286fb0000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f8286f71000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8286f50000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8286dc3000)
/lib64/ld-linux-x86-64.so.2 (0x00007f82878b0000)

# cat ~/.exiv2

[nikon]
146=Robin's Sigma Lens  <--- The name of your lens
201=Sigma 24-70mm F2,8 DG OS HSM Art

# exiv2 -pv --grep lens/i ~/Schreibtisch/106ND750/DSC_5154.NEF ~/Downloads/Stonehenge.jpg

/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  0x0083 Nikon3       LensType                    Byte        1  78
/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  0x0084 Nikon3       Lens                        Rational    4  240/10 700/10 28/10 28/10
/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  0x008b Nikon3       LensFStops                  Undefined   4  72 1 12 0
/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  0x000c NikonLd3     LensIDNumber                Byte        1  *201*
/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  0x000d NikonLd3     LensFStops                  Byte        1  72
/home/dave/Downloads/Stonehenge.jpg  0x0083 Nikon3       LensType                    Byte        1  14
/home/dave/Downloads/Stonehenge.jpg  0x0084 Nikon3       Lens                        Rational    4  180/10 2500/10 35/10 63/10
/home/dave/Downloads/Stonehenge.jpg  0x008b Nikon3       LensFStops                  Undefined   4  55 1 12 0
/home/dave/Downloads/Stonehenge.jpg  0x000c NikonLd3     LensIDNumber                Byte        1  *146*
/home/dave/Downloads/Stonehenge.jpg  0x000d NikonLd3     LensFStops                  Byte        1  55

# exiv2 -pa --grep lens/i ~/Schreibtisch/106ND750/DSC_5154.NEF ~/Downloads/Stonehenge.jpg

/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  Exif.Nikon3.LensType                         Byte        1  D G VR
/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  Exif.Nikon3.Lens                             Rational    4  24-70mm F2.8
/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  Exif.Nikon3.LensFStops                       Undefined   4  6
/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  Exif.NikonLd3.LensIDNumber                   Byte        1  *Sigma 24-70mm F2,8 DG OS HSM Art*
/home/dave/Schreibtisch/106ND750/DSC_5154.NEF  Exif.NikonLd3.LensFStops                     Byte        1  F6.0
/home/dave/Downloads/Stonehenge.jpg  Exif.Nikon3.LensType                         Byte        1  D G VR
/home/dave/Downloads/Stonehenge.jpg  Exif.Nikon3.Lens                             Rational    4  18-250mm F3.5-6.3
/home/dave/Downloads/Stonehenge.jpg  Exif.Nikon3.LensFStops                       Undefined   4  4.58333
/home/dave/Downloads/Stonehenge.jpg  Exif.NikonLd3.LensIDNumber                   Byte        1  *Robin's Sigma Lens  <--- The name of your lens*
/home/dave/Downloads/Stonehenge.jpg  Exif.NikonLd3.LensFStops                     Byte        1  F4.6

So you see the ~/.exiv2 file is set up correctly and detecting the lens name if called correctly (pa vs pv). However, in dt the 24-70mm is not recognized, the other one is.

#4 Updated by David Schaefer 2 months ago

So you see the ~/.exiv2 file is set up correctly and detecting the lens name if called correctly (pa vs pv). However, in dt the 24-70mm is not recognized, the other one is.

Please see the attached screenshots.

#5 Updated by Claude Jolicoeur 2 months ago

Andreas Schneider wrote:

I don't see any darktable issue here. If you want to use new features for exiv2 you have to build darktable against it.

If you're distribution doesn't provide the latest version, try to use a 3rdparty repo which normally has newer dependencies:

I would tend to agree with Andreas that this isn't per se a darktable issue, maybe we could put this in a wish list...

As of 3dr party repos, Pascal's PPA I mentioned in my first post is a repo that holds a more recent version of darktable and of exiv2 - but still the version of exiv2 is 0.25, hence too old to incorporate this new feature.

As of building darktable against the newer version of exif2, this has been tried without success. See this discussion:
https://discuss.pixls.us/t/darktable-on-ubuntu-18-04-exiv2-is-obsolete-any-ppa-snap-solving-this/8587/5
and in particular:
"I also tried building darktable, against exiv2 0.26, I could get the build to succeed, ldd confirms exiv2 0.26 was used, it runs (installed via checkinstall), but it doesn’t pick up the new cameras/lens. There is something else that needs to be changed in the source tree I guess other than the exiv2 version dependency."

Claude

#6 Updated by David Schaefer 2 months ago

Well, I also thought it did not work but I had to remove the image from the db and reimport it. Then my own lensname showed up. (Apparently this lens is also not supported in lensfun so I have no benefit of finding my lens name. I am now in discussion with exiv2 and lensfun).

I now completely agree with Andreas as I have an explanation for the behavior on my dt instance.

#7 Updated by Claude Jolicoeur 2 months ago

David,
Could you please describe in more details what you did to make this work?
If I understand correctly, you didn't have to rebuild darktable.
Please keep in mind that I am not familiar with compilations and installations with Linux... so ideally the procedure should be relatively easy for me to be able to reproduce it.
Thanks, Claude

#8 Updated by David Schaefer 2 months ago

Hi Claude,

I did recompile dt from source (git master) and I also compiled exiv2 from source (git master).
1) You can find out if ~/.exiv2 works by calling: exiv2 -pa --grep lens/i image.jpg.
2) Darktable only calls exiv2 for image information on the first import. So after I have updated my eviv2 version I had to delete my images from the library and reimport it again. Now the lens is correctly displayed inside dt. This shows what Andreas said: There is no issue with dt, but with the components its uses.
3) Since my lens is not yet known to exiv2 I have reported the new lens to exiv2 and the lens will be added in the next exiv2 version (0.27.2 in Q2 2019): https://github.com/Exiv2/exiv2/issues/743#issuecomment-473096468
4) I am also in discussion with liblensfun because my lens is also not known there.
5) In the end is the ~/.exiv2 file not helping much because even though I can see a correct lens name I will miss the functionality until the dependent modules are updated.

#9 Updated by Claude Jolicoeur 2 months ago

David Schaefer wrote:

I did recompile dt from source (git master) and I also compiled exiv2 from source (git master).
1) You can find out if ~/.exiv2 works by calling: exiv2 -pa --grep lens/i image.jpg.

Yes, for me that part went well. I was able to compile exiv2 using this procedure
http://dev.exiv2.org/projects/exiv2/wiki/Building_on_Ubuntu_using_Cmake
and in effect the terminal command for exiv2 reports the right lens.

However this isn't sufficient and I would need to build darktable with the newer version of exiv2. This part however looks daunting for me; I have looked at
https://www.darktable.org/install/
https://redmine.darktable.org/projects/darktable/wiki/Building_darktable_22
and honestly I can't figure how to start!

3) Since my lens is not yet known to exiv2 I have reported the new lens to exiv2 and the lens will be added in the next exiv2 version (0.27.2 in Q2 2019): https://github.com/Exiv2/exiv2/issues/743#issuecomment-473096468
4) I am also in discussion with liblensfun because my lens is also not known there.
5) In the end is the ~/.exiv2 file not helping much because even though I can see a correct lens name I will miss the functionality until the dependent modules are updated.

This part for me is easier, as my lens is an old Sigma 24mm f/2.8 prime lens (dating back from the 1990s, but still working fine). In DT, this lens is identified as a Canon 28mm because both lenses share the same ID number in the exif. Correction coefficients for the Sigma are in lensfun, and the solution I use for the moment is to patch the lensfun data: insert the Sigma data in place of the Canon 28mm data. This way, lensfun thinks this is a Canon 28mm, but applies the correction of the Sigma 24mm! Works, but lacks elegance...

In your case, from what I understand, you would have to provide some photos to the lensfun team, so that they will be able to compute the correction coefficients needed for your particular lens.
It would be so much easier if the lens manufacturers would make this sort of information available! I am sure Sigma knows very well the correction factors needed for the lenses they make - why don't they make this public?

Also available in: Atom PDF