Project

General

Profile

Bug #10704

green atifacts with denoise profiled

Added by Pascal Obry over 2 years ago. Updated over 1 year ago.

Status:
Confirmed
Priority:
Medium
Assignee:
-
Category:
Darkroom
Target version:
-
Start date:
11/08/2015
Due date:
% Done:

10%

Affected Version:
git development version
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Attached is a CR2 with the corresponding .xmp.

The image has some noise. When a wavelet denoise profile is activated there is lot of green appearing on the fur of the gorilla.

Don't think this is expected. Looks like a bug to me.

It has been reported by a friend using 1.6.8 and I have been able to reproduce with the Git master version.

Note that I have deactivate the 7D denoise data and recompiled dt to use the generic profile and I have the same issue. So it does not look like an issue with the specific profile for the 7D.

IMG_4365.CR2.xmp (3.2 KB) Pascal Obry, 11/08/2015 04:22 PM

IMG_4365.CR2 (26.7 MB) Pascal Obry, 11/08/2015 04:24 PM

Screenshot - 12162015 - 03_56_08 PM.png (1.4 MB) Joe Giampaoli, 12/17/2015 12:01 AM

Screenshot - 12222015 - 03_30_49 PM.png - My Core Options (36.8 KB) Joe Giampaoli, 12/22/2015 11:31 PM

ProfiledDenoiseOff.png (1.45 MB) Pedro Côrte-Real, 12/31/2015 01:02 PM

ProfiledDenoiseOn.png (1.41 MB) Pedro Côrte-Real, 12/31/2015 01:02 PM

DSC01698.ARW (20.3 MB) Pedro Côrte-Real, 12/31/2015 01:08 PM

DSC01698.ARW.xmp (8.77 KB) Pedro Côrte-Real, 12/31/2015 01:09 PM

CRW_0815.DNG (18.3 MB) Joe Giampaoli, 12/31/2015 01:52 PM

CRW_0815.DNG.xmp (3.31 KB) Joe Giampaoli, 12/31/2015 01:52 PM

ProfiledDenoiseOff2.png (551 KB) Pedro Côrte-Real, 12/31/2015 09:00 PM

ProfiledDenoiseOn2.png (561 KB) Pedro Côrte-Real, 12/31/2015 09:00 PM

D81_5386.NEF.xmp (2.49 KB) Joe Giampaoli, 01/01/2016 03:02 AM

Screenshot - 12312015 - 06_58_02 PM.png (542 KB) Joe Giampaoli, 01/01/2016 03:02 AM

img_0001_02.jpg (15.2 MB) Johannes Hanika, 04/27/2016 10:57 PM

Screenshot - 10172016 - 03_15_17 PM.png (1.46 MB) Joe Giampaoli, 10/17/2016 11:23 PM

CRW_1724.DNG.xmp (4.35 KB) Joe Giampaoli, 10/17/2016 11:38 PM

CRW_1724.DNG (18.3 MB) Joe Giampaoli, 10/17/2016 11:39 PM

Capture d_écran de 2016-10-18 14-23-47.png (1.62 MB) Pascal Obry, 10/18/2016 03:25 PM

Screenshot - 10182016 - 03_14_08 PM.png (1.48 MB) Joe Giampaoli, 10/18/2016 11:19 PM

History

#1 Updated by Joe Giampaoli over 2 years ago

I have this very same issue with self made profiles on my Canon SX510. I noticed it at first on the fur of a Squirrel. I just thought maybe it's my profile, not perfect. My profiles denoise very well actually, but now that I saw this bug report it got me thinking. One small work around is that if I choose "linearRec 709" on the input color profile these green artifacts seem to diminish a bit, then I can increase the saturation and colors because that input profile is very flat.

And yes, this only happens with wavelets denoising.

Attaching screenshot of what I see. Just above the eye of the squirrel and on the head you can see that greenish tint artifacts.

Thanks!

#2 Updated by Pedro Côrte-Real over 2 years ago

Do you happen to have opencl enabled? Does it change if you disable it?

#3 Updated by Pascal Obry over 2 years ago

No, I have the same result with and without OpenCL.

#4 Updated by Joe Giampaoli over 2 years ago

Pedro Côrte-Real wrote:

Do you happen to have opencl enabled? Does it change if you disable it?

In my case it seems it's enabled but I can't disable it. The check-mark is like greyed out but active. I believe I have openCL installed right in Debian, but maybe the openCL packages are too old?

#5 Updated by Pascal Obry over 2 years ago

If it's gray out and ticked it means that it has been activated at some point but it is not activated now. Meaning that you're missing some OpenCL libraries now.

#6 Updated by Joe Giampaoli over 2 years ago

Pascal Obry wrote:

If it's gray out and ticked it means that it has been activated at some point but it is not activated now. Meaning that you're missing some OpenCL libraries now.

OK, I see, well I tried starting DT with -d opencl -d perf and got the following:

[opencl_init] opencl related configuration options:
[opencl_init] 
[opencl_init] opencl: 1
[opencl_init] opencl_library: ''
[opencl_init] opencl_memory_requirement: 768
[opencl_init] opencl_memory_headroom: 300
[opencl_init] opencl_device_priority: '*/!0,*/*/*'
[opencl_init] opencl_size_roundup: 16
[opencl_init] opencl_async_pixelpipe: 0
[opencl_init] opencl_synch_cache: 0
[opencl_init] opencl_number_event_handles: 25
[opencl_init] opencl_micro_nap: 1000
[opencl_init] opencl_use_pinned_memory: 0
[opencl_init] opencl_use_cpu_devices: 0
[opencl_init] opencl_avoid_atomics: 0
[opencl_init] opencl_omit_whitebalance: 0
[opencl_init] 
[opencl_init] found opencl runtime library 'libOpenCL'
[opencl_init] opencl library 'libOpenCL' found on your system and loaded
[opencl_init] found 2 platforms
[opencl_init] found 2 devices
[opencl_init] device 0 `GeForce 210' doesn't have sm_20 support.
[opencl_init] discarding device 0 `GeForce 210' due to insufficient global memory (511MB).
[opencl_init] discarding CPU device 1 `AMD Athlon(tm) II X3 450 Processor'.
[opencl_init] no suitable devices found.
[opencl_init] FINALLY: opencl is NOT AVAILABLE on this system.
[opencl_init] initial status of opencl enabled flag is OFF.

It does seem to work with 3d rendering program I use, "luxrender" I used to get an error about not finding openCL, when I installed the Debian libraries it stopped showing that error but really have not tested fully render times in it now that I installed it.

Doesn't really matter, it would be nice to have that extra speed during export but I guess I can live without it, I think it has to do with my card's sm_20 support, I'll figure it out, I don't want to drive this report to another direction....

Thanks anyway, I appreciate it!

#7 Updated by Pascal Obry over 2 years ago

  • Status changed from New to Closed: won't fix

Ok, closing then. The card has not enough memory.

#8 Updated by Joe Giampaoli over 2 years ago

Pascal Obry wrote:

Ok, closing then. The card has not enough memory.

Hmmm, does this memory limit in openCL affect the issue with the green artifacts then?

#9 Updated by Pedro Côrte-Real over 2 years ago

  • Status changed from Closed: won't fix to New

The bug isn't about an opencl issue, it's about green artifacts with denoise. The fact that opencl doesn't work isn't really a resolution, it just means that if there's a bug it's in the CPU code path or possibly in both.

#10 Updated by Pascal Obry over 2 years ago

Sorry, my mistake I should not have closed this one :) This has nothing to do with OpenCL indeed. The confusion is that the OpenCL issue has been raised here but it is a completely different issue.

#11 Updated by Joe Giampaoli over 2 years ago

AH! That's good to know! I was a bit worried there for a second! :)

Thanks!

#12 Updated by Pedro Côrte-Real over 2 years ago

I've reproduced this with one of my images. This time it's the grey coat of a donkey :) It seems easy to trigger in a very visible way by using wavelet mode and HSV color blending which is the suggested way of eliminating chroma noise (and usually works quite well).

#13 Updated by Joe Giampaoli over 2 years ago

By looking at our previous examples, it seems to me that when there's more green value in an image these artifacts become more noticeable. All previous shots have green grass in the background. Here's another test with a dog in an interior of a pharmacy, you can see that the artifacts aren't as exaggerated as the previous ones, and this one is shot at a reasonably high ISO. Attaching both RAW and sidecard.

#14 Updated by Pedro Côrte-Real over 2 years ago

Found a pretty extreme case when testing out a Nikon D810 ISO 12800 file. It's almost as if the green channel noise isn't being eliminated or is even being enhanced.

#15 Updated by Joe Giampaoli over 2 years ago

Pedro Côrte-Real wrote:

Found a pretty extreme case when testing out a Nikon D810 ISO 12800 file. It's almost as if the green channel noise isn't being eliminated or is even being enhanced.

Pedro, may I try something with your original RAW?

#16 Updated by Pedro Côrte-Real over 2 years ago

Joe Giampaoli wrote:

Pedro, may I try something with your original RAW?

Sure. It's too big to attach here but here's a link:

http://scratch.corujas.net/D81_5386.NEF

#17 Updated by Joe Giampaoli over 2 years ago

Thanks Pedro, just wanted to compare with my RAW's. Nice shot BTW.

This is what I get when purging the history stack to original and just basic corrections. Using linearRec709 as input color profile does seem to help a little, even adding extra saturation later, but not 100% effective. This is pretty much how I have been doing it since I saw this green artifacts, as I mentioned before, I thought my denoising profiles or using a CHDK RAW made file had something to do with it. I have also tried exporting to 16bit tiff with no denoising at all and then try to apply the denoising to the tiff but unfortunately it's not as effective as doing it directly on the RAW. I'll attach my xmp version of your RAW if you want to see pretty much what I do. Later I apply something like a low-pass filter with extra saturation in multiply mode to bring out those colors...

Cheers

#18 Updated by Maximilian Trescher about 2 years ago

I experienced the same issues with shots from a Canon 60D.

Curiously there are no green artifacts when I use the equalizer module and apply some chroma denoise.

(I can provide the Raw and xmp files if needed, right now I don"t have access to them).

Cheers
Max

#19 Updated by Johannes Hanika almost 2 years ago

here's my jpg of the green artifact wall. it is really touchy at these noise levels.. using a second instance of profiled denoising with lightness doesn't work, but using HSV lightness performs a lot better. i think maybe the numbers are so small that the colour space transforms become unstable? given this interesting variation in outcome based on the blend mode maybe the culprit is blending and not denoising?

#20 Updated by Pascal Obry over 1 year ago

  • Status changed from Confirmed to Fixed
  • % Done changed from 10 to 100

I think this issue is fixed. I suppose this is due to Roman's work on the low level sensor data fixes done recently. Can someone confirm this is also fixed?

I cannot reproduce with attached DSC01698.ARW nor with my original sample IMG_4365.CR2.

#21 Updated by Joe Giampaoli over 1 year ago

Hmmmm, I still get green artifacts in stable 2.0.6

Are we supposed to try with development branch?

Thanks!

#22 Updated by Pascal Obry over 1 year ago

No you are not supposed to test the dev branch if you are not a developer and not ready to have some issues from time to time.

But yes I was talking about testing the dev version has the fixes have been integrated there only.

#23 Updated by Roman Lebedev over 1 year ago

Pascal Obry wrote:

I think this issue is fixed. I suppose this is due to Roman's work on the low level sensor data fixes done recently.

Hm?

#24 Updated by Pascal Obry over 1 year ago

I think it could be due to the fix on the white level you've done in many places? Couldn't this be it?

#25 Updated by Roman Lebedev over 1 year ago

  • Status changed from Fixed to Confirmed
  • % Done changed from 100 to 10

Not really, those white level fixes mostly only touch nikon.

#26 Updated by Pascal Obry over 1 year ago

Ok, yet this is fixed on my side. A good news... But then we really don't know at this point what has fixed this!

#27 Updated by Joe Giampaoli over 1 year ago

Here's a screenshot from today, same DNG with the embedded color profile of the file, maybe it's a bit less than my older screenshot, but if you look carefully, you will see some greenish artifacts on the head and part of the tail. This is now in latest 2.0.6.

The only de-noising at the moment on this sample is the one expanded on the screenshot which is wavelet on the color channel, HSV color gives me pretty much same results, also you will see that I purged out any automatic applied curves from the start on my history stack, I always do that...

Thanks!

#28 Updated by Joe Giampaoli over 1 year ago

Hmmm, didn't upload screenshot? Let's try again...

#29 Updated by Pascal Obry over 1 year ago

Which is expected as the fix is only in the development version not yet released.

#30 Updated by Joe Giampaoli over 1 year ago

Oh! OK! That's good news!

If you want my DNG just to test let me know.

Will this fix be released in next stable then?

This is really great!

Thanks!

#31 Updated by Pascal Obry over 1 year ago

Yes please make available the DNG and the .xmp I'll double check with my version. Thanks.

#32 Updated by Joe Giampaoli over 1 year ago

OK, here are both, since it's a CHDK DNG it shouldn't be to big to upload here, I hope...

The XMP is just a basic edit to make those artifacts appear, not much processing but basic corrections.

Thanks again!

#33 Updated by Pascal Obry over 1 year ago

With the development version.

#34 Updated by Joe Giampaoli over 1 year ago

OK, yes, it's a bit better. I can still see a little bit of it but not as bad as before with the embedded color profile. Still if I change the input color profile it always gives me better results here on latest stable, I usually use linear rec 709 or linear rec 2020, I think I will get better results now with these profiles and the results you get.

Here's a sample of what I get with linear rec 709 and saturation bumped to 100% so it's almost similar to your screenshot, but like I said, yours is with the embedded profile, which in my case always gives me a lot more of those artifacts.

So to keep it simple, not precisely fixed I think, but better!

Thanks for trying my RAW with your version

Also available in: Atom PDF