Project

General

Profile

Bug #11340

Demosaic outputs negative values

Added by Francisco Cribari over 1 year ago. Updated 4 months ago.

Status:
In Progress
Priority:
Medium
Assignee:
-
Category:
Darkroom
Target version:
Start date:
11/29/2016
Due date:
% Done:

50%

Affected Version:
git master branch
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

I am running the Darktable git version, more specifically 2.2.0rc1+35~g56998df. When I open a RAW file in darkroom, go to the exposure module and click on the color picker to get an auto exposure suggestion DT considerably reduces the blacks and the image's contrast is greatly reduced. That happens with RAW files produced with Fujifim X-100T and X-Pro2 and Nikon D800 RAW files. (I initially thought the problem was Fuji-specific and I later noticed that it also affects Nikon RAW files.) See

https://dl.dropboxusercontent.com/u/2171814/darktable_fuji_exposure-2016-11-27_18.45.01.mp4

RAW file produced with Fujifilm X-Pro2, if needed to repoduce the issue: https://dl.dropboxusercontent.com/u/2171814/_DSF1080.RAF


Related issues

Related to darktable - Bug #11838: Demosaic match greens = "local average" trashes the image Confirmed 11/28/2017
Duplicated by darktable - Bug #11418: Exposure module calculates wrong correction for some Canon RAWs Duplicate 01/02/2017

Associated revisions

Revision 517622ca
Added by Dan Torop over 1 year ago

demosaic: hack to interpolate previews

Right now for previews, roi_out->scale is always 1, hence
full_scale_demosaicing will generally occur. This can produce negative
values in the results of demosaic for images with very dark pixels,
perhaps because the good demosaicing algorithms produce negative values
in response to noisy low-quality downscaled data? Auto exposure will set
a negative black level in order to compensate for these outlying
negative pixels, making the image low-contrast, see #11340.

This change is a total hack which ignores the value of roi_out->scale if
demosaic is working on a preview. Please see #11167, as there are other
problems here -- MIP_F's coming out of demosaic in the preview pipe
should be smaller 720x450, rather than, as now, approx. 1440x900.

History

#1 Updated by Pascal Obry over 1 year ago

  • % Done changed from 0 to 10
  • Status changed from New to Confirmed

Indeed, something really broken here.

#2 Updated by Roman Lebedev over 1 year ago

  • Subject changed from Exposure module problem to Markestein outputs negative values
  • Priority changed from Low to Medium
  • System changed from other GNU/Linux to all

I'm afraid we have another X-Trans issue here.
Assuming that -d perf -d nan do not lie:

[dev] took 0.000 secs (0.000 CPU) to load the image.
[dev_pixelpipe] took 0.000 secs (0.000 CPU) initing base buffer [preview]
[dev_pixelpipe] took 0.003 secs (0.008 CPU) processed `raw black/white point' on CPU, blended on CPU [preview]
[dev_pixelpipe] module `raw black/white point' min: (0.001237) max: (1.000065) [preview]
[dev_pixelpipe] took 0.002 secs (0.008 CPU) processed `white balance' on CPU, blended on CPU [preview]
[dev_pixelpipe] module `white balance' min: (0.001888) max: (1.910720) [preview]
[dev_pixelpipe] took 0.026 secs (0.112 CPU) initing base buffer [full]
[dev_pixelpipe] took 0.002 secs (0.016 CPU) processed `highlight reconstruction' on CPU, blended on CPU [preview]
[dev_pixelpipe] took 0.013 secs (0.072 CPU) processed `raw black/white point' on CPU, blended on CPU [full]
[dev_pixelpipe] module `highlight reconstruction' min: (0.001888) max: (1.000000) [preview]
[dev_pixelpipe] took 0.067 secs (0.380 CPU) processed `demosaic' on CPU, blended on CPU [preview]
[dev_pixelpipe] module `demosaic' min: (-0.152224; 0.000000; -0.129049) max: (1.242378; 1.000000; 1.367112) [preview]
[dev_pixelpipe] took 0.004 secs (0.024 CPU) processed `exposure' on CPU, blended on CPU [preview]
[dev_pixelpipe] module `exposure' min: (-0.152224; 0.000000; -0.129049) max: (1.242378; 1.000000; 1.367112) [preview]
[dev_pixelpipe] took 0.005 secs (0.024 CPU) processed `base curve' on CPU, blended on CPU [preview]
[dev_pixelpipe] module `base curve' min: (0.000000; 0.000000; 0.000000) max: (1.022307; 0.999985; 1.032306) [preview]
[dev_pixelpipe] took 0.005 secs (0.028 CPU) processed `input color profile' on CPU, blended on CPU [preview]
[dev_pixelpipe] module `raw black/white point' min: (-0.000586) max: (1.000065) [full]
[dev_pixelpipe] module `input color profile' min: (-24.330225; -74.532745; -155.622345) max: (100.409744; 193.666733; 278.781891) [preview]
[dev_pixelpipe] took 0.031 secs (0.200 CPU) processed `white balance' on CPU, blended on CPU [full]
[dev_pixelpipe] took 0.025 secs (0.128 CPU) processed `sharpen' on CPU, blended on CPU [preview]
[dev_pixelpipe] module `sharpen' min: (-31.518848; -74.532745; -155.622345) max: (101.844185; 193.666733; 278.781891) [preview]
[dev_pixelpipe] took 0.075 secs (0.452 CPU) processed `output color profile' on CPU, blended on CPU [preview]
[dev_pixelpipe] module `output color profile' min: (0.013016; 0.000000; 0.000000) max: (1.000000; 0.999985; 1.000000) [preview]
[dev_pixelpipe] took 0.003 secs (0.020 CPU) processed `gamma' on CPU, blended on CPU [preview]
[dev_process_preview] pixel pipeline processing took 0.468 secs (1.752 CPU)
[dev_pixelpipe] module `white balance' min: (-0.001120) max: (1.910720) [full]
[dev_pixelpipe] took 0.017 secs (0.072 CPU) processed `highlight reconstruction' on CPU, blended on CPU [full]
[dev_pixelpipe] module `highlight reconstruction' min: (-0.001120) max: (1.000000) [full]
[dev_pixelpipe] took 0.506 secs (2.448 CPU) processed `demosaic' on CPU, blended on CPU [full]
[dev_pixelpipe] module `demosaic' min: (-0.024458; -0.036580; -0.033161) max: (1.112538; 1.114310; 1.119035) [full]
[dev_pixelpipe] took 0.002 secs (0.012 CPU) processed `exposure' on CPU, blended on CPU [full]
[dev_pixelpipe] module `exposure' min: (-0.024458; -0.036580; -0.033161) max: (1.112538; 1.114310; 1.119035) [full]
[dev_pixelpipe] took 0.004 secs (0.016 CPU) processed `base curve' on CPU, blended on CPU [full]
[dev_pixelpipe] module `base curve' min: (0.000000; 0.000000; 0.000000) max: (1.010892; 1.011056; 1.011491) [full]
[dev_pixelpipe] took 0.003 secs (0.016 CPU) processed `input color profile' on CPU, blended on CPU [full]
[dev_pixelpipe] module `input color profile' min: (-5.520893; -38.563103; -76.696312) max: (100.425064; 94.778625; 136.431274) [full]
[dev_pixelpipe] took 0.008 secs (0.028 CPU) processed `sharpen' on CPU, blended on CPU [full]
[dev_pixelpipe] module `sharpen' min: (-7.147652; -38.563103; -76.696312) max: (100.425064; 94.778625; 136.431274) [full]
[dev_pixelpipe] took 0.050 secs (0.260 CPU) processed `output color profile' on CPU, blended on CPU [full]
[dev_pixelpipe] module `output color profile' min: (0.010773; 0.000000; 0.000000) max: (1.000000; 0.999985; 1.000000) [full]
[dev_pixelpipe] took 0.002 secs (0.008 CPU) processed `gamma' on CPU, blended on CPU [full]
[dev_process_image] pixel pipeline processing took 1.361 secs (4.872 CPU)

I read that as if the demosaic was the one to output negative values.
This is with default dt history stack, Markestein 1-pass.

#3 Updated by Roman Lebedev over 1 year ago

Same with VNG and Markestein 3-pass.

#4 Updated by Roman Lebedev over 1 year ago

  • Subject changed from Markestein outputs negative values to Demosaic outputs negative values

#5 Updated by Ulrich Pegelow over 1 year ago

Seems to affect the preview and seems to only affect the CPU codepath:

preview GPU:
[dev_pixelpipe] module `Raw-Schwarz-/Weißpunkt' min: (0,001237) max: (1,000000) [preview]
[dev_pixelpipe] module `Weißabgleich' min: (0,001888) max: (1,910596) [preview]
[dev_pixelpipe] module `Spitzlicht-Rekonstruktion' min: (0,001888) max: (1,000000) [preview]
[dev_pixelpipe] module `Entrastern' min: (0,000000; 0,000000; 0,000000) max: (1,242363; 1,000000; 1,303427) [preview]
[dev_pixelpipe] module `Basiskurve' min: (0,000000; 0,000000; 0,000000) max: (1,022306; 0,999985; 1,027308) [preview]
[dev_pixelpipe] module `Eingabefarbprofil' min: (-52,109722; -72,328392; -225,739151) max: (101,309540; 311,487183; 278,876526) [preview]
[dev_pixelpipe] module `Schärfen' min: (-64,910759; -72,328392; -225,739151) max: (101,845024; 311,487183; 278,876526) [preview]
[dev_pixelpipe] module `Ausgabefarbprofil' min: (0,000000; 0,000000; 0,000000) max: (1,036194; 1,149735; 1,085034) [preview]

preview CPU:
[dev_pixelpipe] module `Raw-Schwarz-/Weißpunkt' min: (0,001237) max: (1,000000) [preview]
[dev_pixelpipe] module `Weißabgleich' min: (0,001888) max: (1,910596) [preview]
[dev_pixelpipe] module `Spitzlicht-Rekonstruktion' min: (0,001888) max: (1,000000) [preview]
[dev_pixelpipe] module `Entrastern' min: (-0,152214; 0,000000; -0,129032) max: (1,242363; 0,999935; 1,367065) [preview]
[dev_pixelpipe] module `Basiskurve' min: (0,000000; 0,000000; 0,000000) max: (1,022306; 0,999969; 1,032302) [preview]
[dev_pixelpipe] module `Eingabefarbprofil' min: (-24,335974; -74,530273; -155,637115) max: (100,409233; 193,690933; 278,769104) [preview]
[dev_pixelpipe] module `Schärfen' min: (-31,524202; -74,530273; -155,637115) max: (101,844353; 193,690933; 278,769104) [preview]
[dev_pixelpipe] module `Ausgabefarbprofil' min: (0,000000; 0,000000; 0,000000) max: (1,023960; 1,111137; 1,055443) [preview]

full GPU:
[dev_pixelpipe] module `Raw-Schwarz-/Weißpunkt' min: (-0,000586) max: (1,000000) [full]
[dev_pixelpipe] module `Weißabgleich' min: (-0,001120) max: (1,910596) [full]
[dev_pixelpipe] module `Spitzlicht-Rekonstruktion' min: (-0,001120) max: (1,000000) [full]
[dev_pixelpipe] module `Entrastern' min: (-0,045683; -0,051669; -0,068285) max: (1,125497; 1,120922; 1,154931) [full]
[dev_pixelpipe] module `Basiskurve' min: (0,000000; 0,000000; 0,000000) max: (1,012084; 1,011665; 1,014745) [full]
[dev_pixelpipe] module `Eingabefarbprofil' min: (-5,811750; -36,408783; -83,738441) max: (100,450455; 88,767799; 158,333420) [full]
[dev_pixelpipe] module `Schärfen' min: (-6,002073; -36,408783; -83,738441) max: (100,450455; 88,767799; 158,333420) [full]
[dev_pixelpipe] module `Ausgabefarbprofil' min: (0,000000; 0,000000; 0,000000) max: (1,016048; 1,030010; 1,019253) [full]

full CPU:
[dev_pixelpipe] module `Raw-Schwarz-/Weißpunkt' min: (-0,000586) max: (1,000000) [full]
[dev_pixelpipe] module `Weißabgleich' min: (-0,001120) max: (1,910596) [full]
[dev_pixelpipe] module `Spitzlicht-Rekonstruktion' min: (-0,001120) max: (1,000000) [full]
[dev_pixelpipe] module `Entrastern' min: (-0,045683; -0,051669; -0,068285) max: (1,125497; 1,120922; 1,154931) [full]
[dev_pixelpipe] module `Basiskurve' min: (0,000000; 0,000000; 0,000000) max: (1,012084; 1,011665; 1,014745) [full]
[dev_pixelpipe] module `Eingabefarbprofil' min: (-5,811779; -36,407681; -83,706055) max: (100,449760; 88,769348; 158,294266) [full]
[dev_pixelpipe] module `Schärfen' min: (-6,002103; -36,407681; -83,706055) max: (100,449760; 88,769348; 158,294266) [full]
[dev_pixelpipe] module `Ausgabefarbprofil' min: (0,000000; 0,000000; 0,000000) max: (1,016103; 1,030010; 1,018783) [full]

#6 Updated by Ulrich Pegelow over 1 year ago

I had suspected that the issue might be related to border handling, which differs between CPU and GPU path, but that doesn't seem to be the case. For the following output I had changed the nan/min/max statistics so that it disregards the borders of the image (outermost 20px). No principle difference versus the full image:

preview GPU:
[dev_pixelpipe] module `Raw-Schwarz-/Weißpunkt' min: (0,001237) max: (1,000000) [preview]
[dev_pixelpipe] module `Weißabgleich' min: (0,001888) max: (1,910596) [preview]
[dev_pixelpipe] module `Spitzlicht-Rekonstruktion' min: (0,001888) max: (1,000000) [preview]
[dev_pixelpipe] module `Entrastern' min: (0,000000; 0,000000; 0,000000) max: (1,242363; 0,999935; 1,303427) [preview]
[dev_pixelpipe] module `Basiskurve' min: (0,000000; 0,000000; 0,000000) max: (1,022306; 0,999969; 1,027308) [preview]
[dev_pixelpipe] module `Eingabefarbprofil' min: (-16,232780; -71,305931; -134,194473) max: (100,410034; 141,212723; 278,876526) [preview]
[dev_pixelpipe] module `Schärfen' min: (-22,138666; -71,305931; -134,194473) max: (101,845024; 141,212723; 278,876526) [preview]
[dev_pixelpipe] module `Ausgabefarbprofil' min: (0,000000; 0,000000; 0,000000) max: (1,023864; 1,111148; 1,055936) [preview]

preview CPU:
[dev_pixelpipe] module `Raw-Schwarz-/Weißpunkt' min: (0,001237) max: (1,000000) [preview]
[dev_pixelpipe] module `Weißabgleich' min: (0,001888) max: (1,910596) [preview]
[dev_pixelpipe] module `Spitzlicht-Rekonstruktion' min: (0,001888) max: (1,000000) [preview]
[dev_pixelpipe] module `Entrastern' min: (-0,152214; 0,000000; -0,129032) max: (1,242363; 0,999935; 1,303427) [preview]
[dev_pixelpipe] module `Basiskurve' min: (0,000000; 0,000000; 0,000000) max: (1,022306; 0,999969; 1,027308) [preview]
[dev_pixelpipe] module `Eingabefarbprofil' min: (-16,232859; -71,305542; -134,155518) max: (100,409233; 141,212814; 278,769104) [preview]
[dev_pixelpipe] module `Schärfen' min: (-22,138771; -71,305542; -134,155518) max: (101,844353; 141,212814; 278,769104) [preview]
[dev_pixelpipe] module `Ausgabefarbprofil' min: (0,000000; 0,000000; 0,000000) max: (1,023960; 1,111137; 1,055443) [preview]

#7 Updated by Ulrich Pegelow over 1 year ago

Forget all what was said on the differences between CPU and GPU. That was simply due to some code divergence between the two. Has been fixed now in master. Remaining differences are due to the different border handling in the two cases. The problem described in this ticket thus remains unsolved so far and is valid for the GPU codepath as well.

SIDE NOTE: I would really appreciate if changes to process() in any module are not just simply applied to the CPU code without further notice. At least you need to drop me a note so that I have a chance to look after the corresponding OpenCL code. In this case reason for the divergence has been commit d1f9bce571e421ec22c4da83c0864a72e8bc97cb. It took me the better part of this evening to figure out :(

#8 Updated by Dan Torop over 1 year ago

Yes, seeing it too as with both your results (CPU only here, no GPU). I'll spend a bit of time on this over the weekend. More, with luck, if I can find a result.

#9 Updated by Dan Torop over 1 year ago

The negative values are scattered in the red/blue channel of dark areas when Markesteijn demosaic operates on an image downscaled by dt_iop_clip_and_zoom_mosaic_third_size_xtrans(). Currently, full_scale_demosaicing is always true for previews (see #11167), hence this is the usual case. Auto exposure gets confused by the outlying negatives values and sets a negative black point, destroying image contrast. Note that negative values also occur in thumbnails which go through the full demosaic path due to uhq_thumb being true, but this seems to have no bad repercussions. I don't see negative values when demosaic processes a full image buffer.

If I force full_scale_demosaicing to be false for previews, as in https://github.com/dtorop/darktable/tree/preview-interpolate-hack, the negative values go away and auto exposure works.

In Markesteijn demosaic, the negative values occur in the "Interpolate red and blue values for solitary green pixels" stanza for very dark areas of the image. This pass looks horizontally/vertically by one and two pixels and uses those red/blue pixels to guess the red/blue value of the current pixel. It also uses some roughly interpolated green values for the neighboring red/blue pixels, which seems to lead to the results going negative. The original dcraw code clips the results of this pass, but my commit b528051f9a5e2f28636f959ec4d59c3f8d44d9ef removed this for dt's version.

My guess is that the low-quality downscaling for @MIP_F@s produces noisy results which expose flaws in demosaic. (One could think about this as demosaic actually doing the right thing, by producing red/blue values which in the case of very dark noisy areas are so noisy that the dip into the negative? Regardless, it confuses auto exposure.)

I haven't looked at VNG or a Bayer image.

#10 Updated by Dan Torop over 1 year ago

Also, it goes without saying (with regards to Ulrich's side note particularly) that even if the preview-interpolate-hack fix is helpful, it doesn't yet change behavior of the OpenCL path.

#11 Updated by Dan Torop over 1 year ago

PR #1382 has this code. OpenCL path should be changed appropriately to match to CPU path. The PR does fix this bug, but I'm still not entirely comfortable with whether this a solution or a quick hack.

#12 Updated by Roman Lebedev over 1 year ago

  • Target version changed from 2.2.0 to 2.4.0

#13 Updated by Roman Lebedev over 1 year ago

  • Duplicated by Bug #11418: Exposure module calculates wrong correction for some Canon RAWs added

#14 Updated by Alexander Steffen over 1 year ago

The first commit 517622caa420f955d3336951f9154d58aab001f5 (demosaic: hack to interpolate previews) of PR #1382 is sufficient to fix the issue I saw in #11418. Is there anything wrong with using that as a bugfix?

#15 Updated by Dan Torop over 1 year ago

I'd hoped the code in PR 1397 would help this (for X-Trans), as my guess was that artifacts in downsample were the problem. But no go there. Good to know 517622caa420f955d3336951f9154d58aab001f5 works for Bayer, and that http://www.rawsamples.ch/raws/canon/RAW_CANON_EOS_5DS.CR2 is a good test image.

Regarding PR 1382, it's been worrying me that identical logic to determine full_scale_demosaicing, xtrans_full_markesteijn_demosaicing, etc. is spread out amongst CPU/OpenCL paths. Adding yet another element to that logic (quick downscaling for previews) only exacerbates this. I would like to amend the PR by adding a function in demosaic iop akin to get_quality() which would in one place set a bit field corresponding to whether various conditions are met for processing the image. I'll also revert the shrinking of mosaiced MIP_Fs, which is wrong.

#16 Updated by Dan Torop over 1 year ago

I updated PR 1382 as suggested in my prior note. It's an invasive change to demosaic, but I think there's an argument that it's a cleanup. The current state of the PR fixes auto-exposure for test Bayer and X-Trans images. I'm still not sure if this gets to the root of the problem, or is a coincident fix.

#17 Updated by Roman Lebedev over 1 year ago

  • % Done changed from 10 to 100
  • Status changed from Confirmed to Fixed

#18 Updated by Roman Lebedev over 1 year ago

  • % Done changed from 100 to 50
  • Status changed from Fixed to In Progress

#19 Updated by Peter Budai 11 months ago

Sorry for the naive question: Has this been not fixed by PR #1382: https://github.com/darktable-org/darktable/pull/1382 ? Status says in progress.

#20 Updated by Roman Lebedev 11 months ago

Peter Budai wrote:

Sorry for the naive question: Has this been not fixed by PR #1382: https://github.com/darktable-org/darktable/pull/1382 ? Status says in progress.

It was fixed, but the next changes essentially unfixed it.
At least i'm pretty sure it is still reproducible in git master.

#21 Updated by Dan Torop 11 months ago

Roman Lebedev wrote:

It was fixed, but the next changes essentially unfixed it.
At least i'm pretty sure it is still reproducible in git master.

I think 517622caa420f955d3336951f9154d58aab001f5 in particular fixed this bug. Then PR 1411 unfixed it. I've gotten too distracted but will give attention to this in the upcoming weeks. It still seems that the real problem is fixing mosaiced->mosaiced downscaling, but that has its own problems (see bug #11167).

#22 Updated by Roman Lebedev 4 months ago

  • Related to Bug #11838: Demosaic match greens = "local average" trashes the image added

#23 Updated by Roman Lebedev 4 months ago

  • Target version changed from 2.4.0 to 2.6.0

Also available in: Atom PDF