Project

General

Profile

Bug #10320

Local Contrast IOP turns saturated blues to magenta (patch enclosed)

Added by Ralf Brown about 5 years ago. Updated almost 5 years ago.

Status:
Fixed
Priority:
Low
Assignee:
Category:
Darkroom
Start date:
02/10/2015
Due date:
% Done:

100%

Estimated time:
0.00 h
Affected Version:
git development version
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

I've been having trouble with blue LED lighting showing up as magenta in images ever since 1.5.1. Enabling gamut clipping in the Input Color Profile module has only been a partial workaround (sometimes I also need to set the profile to Rec2020, which messes with all the colors in the image).

I've finally tracked down a minimal history stack causing the problem. It requires the highly-saturated blue in the image, a tungsten-like white balance (high blue multiplier), and the local contrast module enabled. Note that in the white balance module, you can set any red multiplier you like without affecting whether or not the bug occurs, and you can also set any BLUE multiplier as long as the GREEN multiplier is approximately equal to the blue multiplier.

In the enclosed sample image and history stack, toggling local contrast on and off will flip the highlights on the keyboard and keyboard player's jacket between deep blue and bright magenta.

I tracked the color change to a statement that clips the L channel. In file src/common/bilateral.h, function dt_bilateral_slice(), changing the line (#278)

   out[index] = MAX(0.0f, Lout);

to
   out[index] = Lout;

fixes the problem. If you run a version of DT with that change in place, toggling local contrast on/off in the attached example changes the local contrast of the saturated blues without turning them magenta. I see that the function dt_bilateral_slice() is called from a few different iops, but haven't yet double-checked whether the patch causes any issues elsewhere.

Version 1.7.0+926~gd05d0fc.

DSE64854.nef.xmp (4.64 KB) DSE64854.nef.xmp History stack showing bug Ralf Brown, 02/10/2015 06:10 AM
DSE64854.nef (16.2 MB) DSE64854.nef Ralf Brown, 02/10/2015 06:14 AM
DSE64854_01.nef.xmp (6.04 KB) DSE64854_01.nef.xmp Ralf Brown, 03/05/2015 09:45 AM
DSE64854-lcontrast=off.jpg (95.1 KB) DSE64854-lcontrast=off.jpg Too-dark blues resulting from problem with enhanced color matrix Ralf Brown, 03/23/2015 12:21 AM
DSE64854-lcontrast=on.jpg (108 KB) DSE64854-lcontrast=on.jpg Magenta highlights due to L-channel clipping in bilateral.h Ralf Brown, 03/23/2015 12:21 AM
DSE64854-ICCprofile=D7000_neutral.jpg (106 KB) DSE64854-ICCprofile=D7000_neutral.jpg Correct appearance using Nikon's own input profile Ralf Brown, 03/23/2015 12:21 AM
DSE64854-inprofile=Rec2020.jpg (120 KB) DSE64854-inprofile=Rec2020.jpg Appearance using Linear Rec2020 as input profile instead of camera matrix Ralf Brown, 03/23/2015 12:28 AM

Associated revisions

Revision 414b7a2c (diff)
Added by Roman Lebedev almost 5 years ago

Bilateral: dt_bilateral_slice(): do not clamp L. Refs #10320
Syncs native and OpenCL codepaths. (slice kernel was not clamping)

Revision da862192 (diff)
Added by Roman Lebedev almost 5 years ago

Bilateral: dt_bilateral_slice(): do not clamp L. Refs #10320
Syncs native and OpenCL codepaths. (slice kernel was not clamping)

(cherry picked from commit 414b7a2c868cada039d14c9d251006d5bcb2626f)

History

#1 Updated by Ralf Brown almost 5 years ago

Well, it looks like the above patch is treating the symptom, not the underlying cause. After way too much time fiddling with colorin and temperature iops, it looks like either the Nikon D7000 camera matrices themselves or the way colorin handles them is broken (the standard matrix less so than the enhanced matrix).

I've now processed a number of photos where very highly saturated bright blues get a much lower luminance than they should if colorin is using either camera matrix, but not for any of the other profiles -- in every other profile, those blues are still properly brighter than the surrounding areas not lit directly by the blue LED lighting (though Lab does take a huge amount of brightening to even see the image as anything but pure black). I've attached another history stack on the same image as before to demonstrate; so that profile-caused color shifts and system display profiles don't confound the issue, I've set colorout to use sRGB for the display profile and have completely desaturated the image.

On initially opening the image, colorin is set to Rec2020, and you see nice highlights on the person's forehead, bridge of nose, lower lip, left shoulder, and left lapel. Stepping down one operation to #6, colorin is set to the standard color matrix, and the highlights essentially disapper -- the blue light is not contributing to the L channel at all. Dropping down to #5, colorin is now set to the default of enhanced color matrix, and the hightlights turn dark. On this particular image, the color is in the range where the standard matrix is less broken than the enhanced matrix.

If you turn off the white balance iop (or equivalently set r/g/b all to 1.00), the effect becomes much less pronounced -- standard color matrix still shows a bit of highlights and enhanced color matrix doesn't go dark, but looks like standard color matrix with the camera white balance.

The above is all with version 1.7.0+1092~g66bb25f, pulled a few hours ago. I spent hours trying to bypass various points where channels get clipped by colorin (on a version pulled a couple of days ago) without producing any noticeable difference in appearance.

#2 Updated by Ralf Brown almost 5 years ago

I just had a chance to compare against Darktable 1.4.1. The reduction of the L channel in the blue-lit areas when using standard/enhanced color matrix versus one of the other profiles is much less pronounced than in current git master, but still present. 1.4.1 does not, however, produce the near-black pixels of the current version.

#3 Updated by Ralf Brown almost 5 years ago

I added some print statements to current master to look at the actual values generating negative luminances. A few extreme examples:

 cam= 0.186938  0.219031  0.990601

yields
 xyz= 0.099599 -0.113522  0.875484

which converts to
 Lab=-102.543945 607.636108 -353.217834

(and if I understand the code correctly, a should also always be between -500 and +500 and b between -200 and +200)
 cam= 0.174936  0.234026  1.000000
 xyz= 0.096693 -0.101663  0.883971
 Lab=-91.831757 559.158508 -335.405121
 cam= 0.200858  0.217677  0.961503
 xyz= 0.111730 -0.100041  0.848783
 Lab=-90.366615 564.309509 -330.128418

Printing out the actual values of piece->data->cmatrix passed to colorin.c:process() shows the following conversions for Rec2020:

  RGB (1,0,0) -> xyz(0.6735 0.2790 -0.0019)
  RGB (0,1,0) -> xyz(0.1657 0.6754 0.0300)
  RGB (0,0,1) -> xyz(0.1250 0.0456 0.7969)

(If you read the right-hand side as a 3x3 matrix, those are the values passed in the cmatrix parameter, and the center column gives the coefficients for the luminance function L = aR + bG + cB.) Presumably that -0.0019 is a result of rounding errors during matrix inversion; Rec709, Linear Infrared BGR, sRGB, and AdobeRGB matrices are all entirely non-negative.

In contrast to the above matrices, the D7000 standard matrix gives

  RGB (1,0,0) -> xyz(0.68740 0.26750 0.01505)
  RGB (0,1,0) -> xyz(0.30827 1.03427 -0.18022)
  RGB (0,0,1) -> xyz(-0.03147 -0.30178 0.99007)

and the enhanced matrix is even more extreme with
  RGB (1,0,0) -> xyz(0.66743 0.20374 -0.04249)
  RGB (0,1,0) -> xyz(0.41363 1.21879 -0.03135)
  RGB (0,0,1) -> xyz(-0.11687 -0.42253 0.89874)

This is saying that increasing blue darkens the pixel by twice as much as the equal amount of additional red brightens the image!

While I can conceive of a case with poor channel separation where that might actually be mathematically correct, it would only be correct for a single white balance (set of channel multipliers).

#4 Updated by Roman Lebedev almost 5 years ago

  • % Done changed from 0 to 20
  • Status changed from New to Incomplete

I have looked at that sample, and i must say, i do not see where on sample saturated blues turns magenta.
Maybe some screenshots?

Are you sure you display profile is not broken?

#5 Updated by Ralf Brown almost 5 years ago

The problem is visible even in JPEG exports using sRGB, and even after stripping out all ExIF to ensure that the display program isn't using embedded profile information. I'm attaching three such exported files (using version 1.7.0+1150~g87f7321). The first two use the history stack from the original post, with the local contrast turned on (magenta highlights) and off (blue, but much too dark). The third one is using an external profile that I found online as the input profile; this profile was extracted from ViewNX, and shows the color of the lighting as it should appear -- and it is quite similar to the appearance you get when choosing Rec2020 as the input profile rather than one of the camera color matrices.

#6 Updated by Roman Lebedev almost 5 years ago

  • % Done changed from 20 to 10
  • Status changed from Incomplete to Confirmed

Hmm, okay, now i confirm that saturated blues turns magenta.
But only when CPU codepath is used - opencl is fine... That is definitely shouldn't happen.

#7 Updated by Roman Lebedev almost 5 years ago

Though, setting gamut clipping to "linear Rec2020 RGB" totally fixes it, which, honestly is our current solution.

#8 Updated by Ralf Brown almost 5 years ago

While the gamut clipping keeps the blues from flipping to magenta, they still come out much too dark -- just as in DSE64854-lcontrast=off.jpg, compared to the correct brightness in DSE64854-ICCprofile=D7000_neutral.jpg.

And yes, I'm using CPU-only because I have a low-end graphics card and high-end CPU, and with OpenCL enabled the GPU alone takes more elapsed time than total CPU time without OpenCL, never mind elapsed time....

#9 Updated by Roman Lebedev almost 5 years ago

  • System changed from other GNU/Linux to all
  • Estimated time changed from 0.50 h to 0.00 h
  • % Done changed from 10 to 100
  • Assignee set to Roman Lebedev
  • Status changed from Confirmed to Fixed

I have dropped that clamp.
In this particular case, that might be the fix, however we predict that it might come back to us.

#10 Updated by Ralf Brown almost 5 years ago

In this particular case, that might be the fix, however we predict that it might come back to us.

See #10374, which describes multiple other iops where the negative L values spit out by colorin cause clamping that turns deep blues into magenta. Which ultimately seems to come from the color matrix having a negative value in cmatrix [5] (the coefficient for the blue contribution to luma).

Also available in: Atom PDF

Go to top