Project

General

Profile

Bug #8882

Basecurve for Ricoh GR Digital IV camera broken

Added by Michal Thoma over 6 years ago. Updated over 3 years ago.

Status:
Fixed
Priority:
Low
Assignee:
-
Category:
Darkroom
Start date:
08/22/2012
Due date:
% Done:

100%

Affected Version:
git development version
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Just tried to open DNG raw from Ricoh GR Digital IV in v1.0.5 and image was open but display broken. Just the narrow strip of image appears in the darkroom mode. The thumnails are read ok. With the older version of Darktable, the files were open. Sample file for camera can be found here: http://www.photographyblog.com/reviews/ricoh_gr_digital_iv_review/sample_images/

History

#1 Updated by Boris Simovic over 6 years ago

I have the same problem with DNG files from my Ricoh GXR camera! This bug is in both the Linux and Mac versions of Darktable...

#2 Updated by Boris Simovic over 6 years ago

Possibly a problem with Rawspeed, not libraw?

http://www.libraw.org/node/1667

#3 Updated by Pascal de Bruijn over 6 years ago

This is likely a problem with RawSpeed. I'll give Klaus Post a bump.

On a sidenote: Reading that link, I'm not sure what you meant by "trying to recompress it", it's never a good idea to modify original camera files. These formats tend to be rather fragile. So I wouldn't modify them even if everything "appears" to be working.

#4 Updated by Pascal de Bruijn over 6 years ago

  • Status changed from New to Confirmed
  • Target version set to Candidate for next major release
  • % Done changed from 0 to 10

#5 Updated by Michal Thoma over 6 years ago

Pascal de Bruijn wrote:

On a sidenote: Reading that link, I'm not sure what you meant by "trying to recompress it", it's never a good idea to modify original camera files. These formats tend to be rather fragile. So I wouldn't modify them even if everything "appears" to be working.

OT: Well camera produces DNG which is uncompressed and with half size preview. In my archiving/cataloging scenario, I want to keep losslessly compressed and full preview files so I want to process camera's DNG through DNG converter / and get the DNG I like. In fact it's the same as to process any other raw through DNG Converter. Yeah, some people treat camera's produced RAW as something what should not be touched at any cost. I think with good reason it can be done.

The issue with digikam/kipi plugin is being discussed here: https://bugs.kde.org/show_bug.cgi?id=304242

#6 Updated by Pascal de Bruijn over 6 years ago

"In fact it's the same as to process any other raw through DNG Converter."
Which is ill advised too. As there is little tangible benefit in most cases, and non-negligible risk of breakage (sometimes not even immediately obvious).

"Yeah, some people treat camera's produced RAW as something what should not be touched at any cost. I think with good reason it can be done."
For good reason, RAW formats and EXIF are terribly fragile, with likely many implementation "bugs" both on the camera side and exif reader/writer side.

But whatever the case, just know there is a real risk involved. Beyond that it's your choice of course.

#7 Updated by Pascal de Bruijn over 6 years ago

So, Klaus Post from RawSpeed got back to me, and he fixed the cropping problem. I imported the updated RawSpeed code into Darktable git master and it seems to be working just fine.

Next I noticed there were two more issues for Ricoh camera's:

1. Darktable currently doesn't read the D65 matrix from the DNG. I manually extracted these for a range of Ricoh camera's, and put these into our lookup table:
https://github.com/darktable-org/darktable/commit/1c84e22715e964d77b776cbc0c71bde1abe47b8e

2. Darktable doesn't have a basecurve for Ricoh yet... However the Pentax basecurve seems to work very well for the Ricoh samples I have, so I defaulted to that for the time being.

Please give Darktable from git master a try as we're unlikely to do another 1.0.x release, so it will be some time before these changes hit an official released version of Darktable.

If you are using Ubuntu Precise by any chance, I just pushed a new build to my Unstable PPA:
https://launchpad.net/~pmjdebruijn/+archive/darktable-unstable
The build farm is a bit congested at the moment, so the updated build will probably be available in 24 hours or so.

Also, I'd love to get various DNG+JPG samples, so I can verify if the Pentax basecurve is fine.

#8 Updated by Simon Spannagel over 6 years ago

  • Subject changed from DNG file from Ricoh GR Digital IV camera broken to Basecurve for Ricoh GR Digital IV camera broken

#9 Updated by Michal Thoma over 6 years ago

> If you are using Ubuntu Precise by any chance, I just pushed a new build to my Unstable PPA:

https://launchpad.net/~pmjdebruijn/+archive/darktable-unstable
The build farm is a bit congested at the moment, so the updated build will probably be available in 24 hours or so.

By chance I do. :-) Will give it a try tomorrow.

Also, I'd love to get various DNG+JPG samples, so I can verify if the Pentax basecurve is fine.

I think this JPEG+DNG should be just enough to asses baseline curve: http://download.thoma.cz/ricoh-dng-jpg.zip (not good for profiling though, can be affected by color cast from reflected light).

#10 Updated by Michal Thoma over 6 years ago

I tried the unstable Darktable from ppa and can confirm the cropping bug is gone.

But color matrix seems wrong. Please find developed files here: http://download.thoma.cz/darktable-colors.zip . One is made with darktable, second with RAWTherapee. The latter's result seems natural and correth (while maybe bit oversaturated), even more then incamera JPEG from file linked in previous post. In camera jpeg is bit orangish and not so deepblue (probably by design to prevent color clipping).

Regarding the basecurve it's almost prefect in highlight and mishadows though seems to steep in deep shadows. If you point me to methodology, I can try to determine correct curve myself.

Also please mind that at least for Ricoh GR Digital III and IV, the green equlibration should be set as defaul. Local averaging seems to work just fine.

#11 Updated by Pascal de Bruijn over 6 years ago

@ColorMatrix, it looks right over here. Did you discard the history stack on that file, to make sure the new defaults are being properly applied?

@BaseCurve, there is no set procedure. It's just a lot of experimentation using various samples from various camera's from a single brand. If you can supply me with more DNG+JPG samples, I would take another look. Please make sure you include some slightly overexposed and underexposed shots too.

#12 Updated by Michal Thoma over 6 years ago

Pascal de Bruijn wrote:

@ColorMatrix, it looks right over here. Did you discard the history stack on that file, to make sure the new defaults are being properly applied?

Yes I did. Will look at that further. Darktable color rendering seems to be quite congruous with that of digikam, so probably the matrix is good after all.

@BaseCurve, there is no set procedure. It's just a lot of experimentation using various samples from various camera's from a single brand. If you can supply me with more DNG+JPG samples, I would take another look. Please make sure you include some slightly overexposed and underexposed shots too.

Will copy more DNG+JPG samples as you like... I'm quite new to Darktable so I have not much idea about the basic designs. Though would feel that basecurve should be that resulting in normal (N) gradation according to zone system instead of mimicking often dubious JPEG rendering.

#13 Updated by Pascal de Bruijn over 6 years ago

With the default behavior of Darktable we try to get fairly close to the default behavior of the camera.

If you want high accuracy, you should disable the basecurve, and do custom ICC profiles.

#14 Updated by Michal Thoma over 6 years ago

Please find some DNG+JPG samples here: http://download.thoma.cz/grd4samples.zip I tried to give all possible exposure scenarios. Please let me know if this is enough.

I will produce custom ICC soon.

#15 Updated by Pascal de Bruijn over 6 years ago

Ah, thanks for the samples, they're useful to me. I've done a bit of experimenting and indeed these is a bit of a discrepancy in the shadows. I'll try to readjust it a bit when I have time.

#16 Updated by Tobias Ellinghaus over 5 years ago

  • Status changed from Confirmed to Incomplete
  • System set to all
  • % Done changed from 10 to 20

Pascal, any updates?

#17 Updated by Pedro CĂ´rte-Real over 3 years ago

  • % Done changed from 20 to 100
  • Status changed from Incomplete to Fixed
  • bitness set to 64-bit

As far as we know this is fixed. The two raw samples at rawsamples.ch work fine too:

http://rawsamples.ch/index.php/en/ricoh

Please let us know if anything is still missing.

Also available in: Atom PDF