Project

General

Profile

Bug #11130

Export to JPEG issues - missing metadata

Added by Andrius Wild about 2 years ago. Updated almost 2 years ago.

Status:
Fixed
Priority:
Low
Assignee:
-
Category:
-
Target version:
Start date:
08/25/2016
Due date:
% Done:

100%

Affected Version:
git master branch
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Hello, none of the JPEGs created by darktable have metadata from the original file. Camera, date taken, shutter speed, aperture, exposure, ISO, etc. everything is gone. I have looked into metadata with darktable (by importing the new JPEG back to DT) ,exiftool, Geequie, ristretto, showPhoto in Linux as well as File Explorer in Windows 10.

Associated revisions

Revision b74374d5
Added by Roman Lebedev about 2 years ago

Merge remote-tracking branch 'upstream/pr/1255'

Refs #11130

  • upstream/pr/1255:
    When reading exif, ensure that initialy, the buffer ptr is NULL, and ensure it will be freed and reset to NULL if there is an error during the call.
    Initialize exif ptr to NULL, in case the exif reading function do not return a valid pointer.
    Fix allocation size to take into account the 6 bytes header.
    Change how Exif buffers are managed: since we could have buffer of length >= 65356 bytes when adding new tags, allocate them with malloc() and let them be returned by the read function. Then, in the jpeg format, remove the test and let exiv2 manage things: it will display a nice help message if data is too big to fit.
    Display a warning message when there is too much exif data to be read/written.

History

#1 Updated by Roman Lebedev about 2 years ago

  • % Done changed from 0 to 20
  • Status changed from New to Incomplete

Please upload raw+XMP+JPG that show the problem
I would suspect it is because the metadata is too big and don't fit into jpg.
File upload in redmine seems to be broken, so you may need to use dropbox or something.

#2 Updated by Andrius Wild about 2 years ago

Roman Lebedev wrote:

Please upload raw+XMP+JPG that show the problem
I would suspect it is because the metadata is too big and don't fit into jpg.
File upload in redmine seems to be broken, so you may need to use dropbox or something.

Raw - https://drive.google.com/file/d/0B5_iknSPeSNBMVBIT0wxNXowRE0/view?usp=sharing
JPEG with the problem - https://drive.google.com/file/d/0B5_iknSPeSNBUkJ0VmNadmhMT1k/view?usp=sharing
XMP - https://drive.google.com/file/d/0B5_iknSPeSNBTkcteUlsV2ZHM2M/view?usp=sharing

#3 Updated by Gerd Mann about 2 years ago

I can confirm the issue. Also for PNGs no metadata are exported.
No XMP sidecar file is created. The bug seems to be independent from the settings concerning XMP compression.

#4 Updated by Tobias Ellinghaus about 2 years ago

Are there any error messages when starting darktable from a shell?

#5 Updated by Roman Lebedev about 2 years ago

Which version of exiv2 you have? (show dpkg -l | grep -i exiv2)
Are you using pmjdebruijn's ppa or not? (you should)

That file was generated by the Adobe DNG Converter, it's not an original camera RAW.
Do you have the same issue with the original raw file?

#6 Updated by Gerd Mann about 2 years ago

dpkg -l | grep -i exiv2

ii libexiv2-14:amd64 0.25-xenial~ppa2 amd64 EXIF/IPTC metadata manipulation library
ii libgexiv2-2:amd64 0.10.3-2 amd64 GObject-based wrapper around the Exiv2 library
ii libkexiv2-11v5 4:15.08.2-xenial~ppa2 amd64 Qt like interface for the libexiv2 library
ii libkexiv2-data 4:15.08.2-xenial~ppa2 all Qt-like interface for the libexiv2 library -- data files

Are you using pmjdebruijn's ppa or not?
I am using pmjdebruijn's ppa

I am using Canon raw (CR2) and calling Darktable via DigiKam 5.0.1

#7 Updated by Roman Lebedev about 2 years ago

Gerd Mann wrote:

dpkg -l | grep -i exiv2

ii libexiv2-14:amd64 0.25-xenial~ppa2 amd64 EXIF/IPTC metadata manipulation library
ii libgexiv2-2:amd64 0.10.3-2 amd64 GObject-based wrapper around the Exiv2 library
ii libkexiv2-11v5 4:15.08.2-xenial~ppa2 amd64 Qt like interface for the libexiv2 library
ii libkexiv2-data 4:15.08.2-xenial~ppa2 all Qt-like interface for the libexiv2 library -- data files

Are you using pmjdebruijn's ppa or not?
I am using pmjdebruijn's ppa

I am using Canon raw (CR2) and calling Darktable via DigiKam 5.0.1

Does that canon write DNG files all of the sudden?
Because that raw is DNG. Which means it was converted from CR2.
Please try on the original raw file.

#8 Updated by Pascal de Bruijn about 2 years ago

Please keep in mind that "libexiv2-14:amd64 0.25-xenial~ppa2" is not the original Exiv2 in Ubuntu, neither is it my patched Exiv2 version.

Please install Exiv2 and Darktable from:

https://launchpad.net/~pmjdebruijn/+archive/ubuntu/darktable-release

#9 Updated by Gerd Mann about 2 years ago

Now, it seem that I am in a pickle.

exiv2 0.25-xenial~ppa2 comes with DigiKam 5.1.0 via
https://launchpad.net/~philip5/+ppa-packages

Synaptic refuses to install your version.

#10 Updated by Roman Lebedev about 2 years ago

So what was the conclusion?
Using proper libexif fixed it?
Or it only happens with dng and not original raw files?

#11 Updated by Andrius Wild about 2 years ago

Roman Lebedev wrote:

So what was the conclusion?
Using proper libexif fixed it?
Or it only happens with dng and not original raw files?

Hello,

Sorry I haven't tryied installing a different version of libexiv yet. However I do have digikam installed as well so maybe digikam is the one that pulls a buggy version of libexiv. In fact, I have stopped using digikam because of similar issue - it does not save metadata to files properly.

As for the second question, I will have to process DNG files created by camera itself not by Adobe DNG Converter soon. I will keep you posted.

#12 Updated by Roman Lebedev about 2 years ago

Andrey Goreev wrote:

Roman Lebedev wrote:

So what was the conclusion?
Using proper libexif fixed it?
Or it only happens with dng and not original raw files?

Hello,

Sorry I haven't tryied installing a different version of libexiv yet. However I do have digikam installed as well so maybe digikam is the one that pulls a buggy version of libexiv. In fact, I have stopped using digikam because of similar issue - it does not save metadata to files properly.

As for the second question,

I will have to process DNG files created by camera itself not by Adobe DNG Converter soon.

The only question here is:
Does the original raw file that was converted to DNG also exhibit the same metadata issue?

I will keep you posted.

#13 Updated by Andrius Wild about 2 years ago

Roman Lebedev wrote:

The only question here is:
Does the original raw file that was converted to DNG also exhibit the same metadata issue?

I can't answer this since I converted CR2 and PEF files to DNG and deleted the originals.

#14 Updated by Andrius Wild about 2 years ago

Roman Lebedev wrote:

Which version of exiv2 you have? (show dpkg -l | grep -i exiv2)
Are you using pmjdebruijn's ppa or not? (you should)

dpkg -l | grep -i exiv2
ii exiv2 0.25-xenial~ppa2 amd64 EXIF/IPTC metadata manipulation tool
ii libexiv2-14:amd64 0.25-xenial~ppa2 amd64 EXIF/IPTC metadata manipulation library
ii libgexiv2-2:amd64 0.10.3-2 amd64 GObject-based wrapper around the Exiv2 library
ii python-pyexiv2 0.3.2-5ubuntu4build2 amd64 Python binding to Exiv2
ii python-pyexiv2-doc 0.3.2-5ubuntu4build2 all Documentation for Python binding to Exiv2

#15 Updated by Andrius Wild about 2 years ago

Well, I just deleted digikam and related PPAs, reinstalled exiv2 to have these versions of the packages (see below) but it did not solve the problem. Metadata are still missing.
Next step I will try a DNG created by the camera not by Adobe DNG conveter.

dpkg -l | grep -i exiv2
ii exiv2 0.25-2.1pmjdebruijn1~xenial amd64 EXIF/IPTC metadata manipulation tool
ii libexiv2-14:amd64 0.25-2.1pmjdebruijn1~xenial amd64 EXIF/IPTC metadata manipulation library
ii libgexiv2-2:amd64 0.10.3-2 amd64 GObject-based wrapper around the Exiv2 library
ii python-pyexiv2 0.3.2-5ubuntu4build2 amd64 Python binding to Exiv2
ii python-pyexiv2-doc 0.3.2-5ubuntu4build2 all Documentation for Python binding to Exiv2

#16 Updated by Andrius Wild about 2 years ago

Andrey Goreev wrote:

Next step I will try a DNG created by the camera not by Adobe DNG conveter.

Well, this trick did not work either. So no matter how DNG file has been created darktable loses all metadata on export to JPEG.

#17 Updated by Roman Lebedev about 2 years ago

  • % Done changed from 20 to 10
  • Affected Version changed from 2.0.5 to git master branch
  • System changed from Ubuntu to all
  • Category deleted (Lighttable)
  • Status changed from Incomplete to Confirmed

When exporting to jpeg, i see the following message in the console:
[exiv2] /tmp/Downloads/drive.google.com/darktable_exported/IMGP0385.jpg: This does not look like a TIFF image

I'd recommend to report this to http://dev.exiv2.org/

#18 Updated by Andrius Wild about 2 years ago

Roman Lebedev wrote:

When exporting to jpeg, i see the following message in the console:
[exiv2] /tmp/Downloads/drive.google.com/darktable_exported/IMGP0385.jpg: This does not look like a TIFF image

I'd recommend to report this to http://dev.exiv2.org/

I see only "[export_job] exported to (path)" in the terminal, no errors

#19 Updated by Andrius Wild about 2 years ago

Roman Lebedev wrote:

I'd recommend to report this to http://dev.exiv2.org/

This is done. http://dev.exiv2.org/issues/1221

#20 Updated by Andrius Wild about 2 years ago

Andrey Goreev wrote:

Roman Lebedev wrote:

I'd recommend to report this to http://dev.exiv2.org/

This is done. http://dev.exiv2.org/issues/1221

Please see the reply from Exi2 team:
http://dev.exiv2.org/issues/1221#note-1

Please let me know if anything could be done in DT on this.
If there was a setting on what tags are to be kept / cut off that would help.

#21 Updated by Roman Lebedev about 2 years ago

  • % Done changed from 10 to 100
  • Status changed from Confirmed to Fixed

#22 Updated by Roman Lebedev almost 2 years ago

  • Target version set to 2.2.0

Also available in: Atom PDF