Project

General

Profile

Bug #10498

Image color incorrect on wide gamut display

Added by David Wang almost 5 years ago. Updated over 3 years ago.

Status:
Fixed
Priority:
Low
Assignee:
Category:
General
Target version:
Start date:
05/29/2015
Due date:
% Done:

100%

Estimated time:
Affected Version:
1.6.6
System:
Mac OS X
bitness:
64-bit
hardware architecture:
amd64/x86

Description

Darktable 1.6.6
OS X 10.10.3
Display Monitor: Dell 2407WFP-HC Wide Gamut Display

A screenshot is attached. One the left side is the image showen in Darktable.
On the right-hand side is the same image exported to JPEG and showen in OS X Preview. You can see the color difference.
I actually have another monitor with standard gamut attached to the same computer. OS X preview shows consistent color on both displays, while darktable displays correctly on the display with standard gamut, but incorrectly on wide gamut display.

This should be a display color management issue.

History

#1 Updated by David Wang almost 5 years ago

Screenshot attached.

#2 Updated by Tobias Ellinghaus almost 5 years ago

File upload is broken at the moment, please upload the screenshot somewhere else.

#3 Updated by David Wang almost 5 years ago

The screen shot is here. Please see if you can see it.
https://drive.google.com/file/d/0B__iLgQAy_l-eG15YWNkdHFtOXc/view?usp=sharing

#4 Updated by Tobias Ellinghaus almost 5 years ago

  • % Done changed from 0 to 20
  • Status changed from New to Incomplete
Ok, I would like to know two things:
  • what display profile is selected in the "output color profile" module?
  • what do you get when running darktable from a terminal like this?
    darktable -d control | grep "color profile" 
    

I don't know too much about how OSX handles screen profiles, but I am sure we can somehow work this out.

#5 Updated by David Wang almost 5 years ago

what display profile is selected in the "output color profile" module?
- System Display Profile
what do you get when running darktable from a terminal like this?
- See below
----------------------
./darktable -d control | grep "color profile"

(process:649): Gtk-WARNING **: Locale not supported by C library.
Using the fallback 'C' locale.

(darktable-bin:649): GLib-GObject-WARNING **: invalid cast from 'GtkMenuBar' to 'GtkWindow'

(darktable-bin:649): Gtk-CRITICAL **: gtk_window_add_accel_group: assertion 'GTK_IS_WINDOW (window)' failed
[color profile] we got a new screen profile `Dell U2312HM - Custom Color v4.icc' from the osx color profile api (size: 4176)
[color profile] we got a new screen profile `DELL2407WFPHC Calibrated - Gamma 2.2' from the osx color profile api (size: 10620)
May 30 08:56:36 Rodgers-Mac-Pro.local darktable-bin649 <Error>: The function ‘CGFontGetGlyphPath’ is obsolete and will be removed in an upcoming update. Unfortunately, this application, or a library it uses, is using this obsolete function, and is thereby contributing to an overall degradation of system performance.
May 30 08:56:36 Rodgers-Mac-Pro.local darktable-bin649 <Error>: The function ‘CGFontGetGlyphPaths’ is obsolete and will be removed in an upcoming update. Unfortunately, this application, or a library it uses, is using this obsolete function, and is thereby contributing to an overall degradation of system performance.


#6 Updated by David Wang almost 5 years ago

Also attach two new screen shots.
This one is screenshot of darktable showing the photo on normal gamut display
https://drive.google.com/file/d/0B__iLgQAy_l-ang1Skc3Zkp2Qkk/view?usp=sharing

This one is the screenshot of darktable showing the same photo on wide gamut display.
https://drive.google.com/file/d/0B__iLgQAy_l-ZmNYY1hGTWNYVFk/view?usp=sharing

#7 Updated by Tobias Ellinghaus almost 5 years ago

  • Does darktable mention the correct display profile with -d control | ... whenever you move the window to another monitor? Or are they interchanged?
  • Does darktable's display look right when you copy your display profiles to ~/.config/darktable/color/out/ (you probably have to create that directory) and select the display profile manually in "output color profile"? Or does it look exactly the same?

#8 Updated by David Wang almost 5 years ago

Does darktable mention the correct display profile with -d control | ... whenever you move the window to another monitor? Or are they interchanged?

--Yes. Whenever I drag Darktable window from one screen to another, Darktable log output prompt that it gets new screen profile from OSX. And, the profile name is correct.

Does darktable's display look right when you copy your display profiles to ~/.config/darktable/color/out/ (you probably have to create that directory) and select the display profile manually in "output color profile"? Or does it look exactly the same?

--I did exactly as what you told above, but it looks exactly the same. No change.

#9 Updated by Tobias Ellinghaus almost 5 years ago

Did you try those display profiles in some other application to see if they are valid at all?

#10 Updated by Pascal de Bruijn almost 5 years ago

When taking a look at normal_Gamut.jpg and wide_gamut.jpg this is exactly what I'd expect.

Wide gamut displays inherently oversaturate everything, thus a profile for a wide gamut display essentially desaturates the image data, before it's passed to the display, so everything looks "normal". And this is exactly what's happening.

What I'm wondering is, how the associated profiles have been generated? Are these auto-generated by OSX? Or are these proper profiles generated using a proper colorimeter? (if so, which one particularly). (Please upload both ICC profiles somewhere, so I can take a look).

If it's the autogenerated by OSX, it's likely derived from the display's EDID (which is often wrong), but even if that's mostly right, many wide gamut display offer a sRGB emulation option (by which you essentially crippling your pricy display capabilities to a much cheaper one's). When sRGB emulation is enabled, it essentially no longer functions as a wide gamut display, but the display EDID may still report it's native wide gamut capabilities. Thus creating this mismatch. So I highly recommend checking the display's OSD.

#11 Updated by David Wang almost 5 years ago

Here is the same NEF image shown by OS X Preview on both displays
Normal Gamut: https://drive.google.com/file/d/0B__iLgQAy_l-cXNfZkVVVUZPTEU/view?usp=sharing
Wide Gamut: https://drive.google.com/file/d/0B__iLgQAy_l-dXhWQzREVl9NZm8/view?usp=sharing

Blow is the same NEF image shown by Nikon View NX-i on both displays
Normal Gamut: https://drive.google.com/file/d/0B__iLgQAy_l-c1lOdTRNWGFZNWc/view?usp=sharing
Wide Gamut: https://drive.google.com/file/d/0B__iLgQAy_l-NXRsZmYwcEFVTzQ/view?usp=sharing

To my naked eye, they look identical. So I come to a wild guess: in OS X, the application itself doesn't really need to care about display profile, the system will handle it automatically.
A bit Google gives me this link:
https://developer.apple.com/library/mac/technotes/tn2313/_index.html#//apple_ref/doc/uid/DTS40014694-CH1-COLOR_MANAGEMENT_WORKFLOWS
Look at the section “Example - What happens in an Application using Color Managed Frameworks”. My guess is proved. OS X applications don't need to know target device color profiles, ColorSync will handle the conversion for them.
So I go ahead and did an experiment: manually set the display profile of Darktable to sRGB, and drag the main window between the two displays. Oh, dear. This time no color shift any more. They look similar enough, albeit not exactly the same. (no two display look exactly the same. I know that)

I am not a color management expert, but I understand it to some extent. I hope my additional guess below could be of any help to the developers.
I guess the flaw in Darktable's screen display output logic, at least in OS X, is when it calls Quartz or CoreImage, it does not attach its own output color profile to the output image bitmap data. In this circumstance OS X can only assume the output color profile of the image bitmap data is sRGB, and then does a second round of color conversion(remember Darktable has already done a round of conversion to the system display profile). That's why the result is incorrect, especially obvious on wide gamut display.

The solution could be:
1. Just like what I did in the experiment, set the display output profile to sRGB manually.
It works, but theoretically this incurs two rounds of color conversion: RAW color --> sRGB --> Display profile
First conversion happens in Darktable, and the second round happens in ColorSync. This is a waste of time and may incur precision loss.
2. Modify darktable code to attach it's output color profile to the image data to be displayed when it calls OS X Quartz(or CoreImage)
This approach OS X knows Darktable has already done the conversion. So only one conversion occurs from RAW color space to display profile in darktable. This looks like a better approach.
This link has good explanation, including code examples:
https://developer.apple.com/library/ios/technotes/tn2313/_index.html

As an additional step, I am also going to check if it's similar in Ubuntu.

#12 Updated by David Wang almost 5 years ago

Gents, Situation in Ubuntu 14.04 seems similar.

#13 Updated by Tobias Ellinghaus almost 5 years ago

David Wang wrote:

Gents, Situation in Ubuntu 14.04 seems similar.

What do you mean? Linux doesn't have any desktop wide color management (by default).

#14 Updated by Tobias Ellinghaus almost 5 years ago

Oh, and please answer Pascal's questions:
  • how have the profiles been created?
  • could you upload the profiles for us to inspect?
  • could you check if your display is set to emulate sRGB in it's settings?

#15 Updated by Igor Kuzmin almost 5 years ago

  • % Done changed from 20 to 10
  • Assignee set to Igor Kuzmin
  • Status changed from Incomplete to Confirmed

Indeed it seems that on OS X 10.10 DT applies the profile twice. I'm pretty sure it wasn't the case with OS X 10.8 (where actual behavior contradicted the documentation, see https://bugzilla.gnome.org/show_bug.cgi?id=681784), so Apple must have changed this either in OS X 10.9 or 10.10. Because of that there's no easy fix and code examples in the documentation you linked doesn't do us much good since we use GTK/cairo for rendering, not Quartz directly. For the time being as a workaround set sRGB as display profile.

#16 Updated by Pascal de Bruijn almost 5 years ago

So the obvious and ideal approach (long-term), if possible, would be to opt-out of the fullscreen color management. And only do CM ourselves.

Short term, it might be nice, to stop querying OSX for display profiles by default, for certain versions of OSX, so at least users don't have to deal with this consciously. But this will sadly cripple us to sRGB :(

#17 Updated by Igor Kuzmin almost 5 years ago

Things are much worse.
Firstly I don't know a way to opt-out from CM. Secondly if we just disable system profile querying it will cripple current OS X versions to sRGB only, but older versions would be become unmanaged at all. Thirdly Apple changed documentation (by removing the note about CGColorSpaceCreateDeviceRGB being actually a generic profile), so if before documentation stated that we are doing the wrong thing but actually it worked, right now we do the right thing but it doesn't work. This is totally fucked up and I don't know how to correctly fix this. Cairo doesn't have a way to set colorspace, so only way I see is to disable getting color profile depending on runtime OS X version. The problem here is that I don't know from which OS X version CM stuff broke, was it 10.9 or 10.10? We need someone with OS X 10.9 installed.

#18 Updated by Igor Kuzmin almost 5 years ago

  • % Done changed from 10 to 50
  • Status changed from Confirmed to In Progress

Committed a workaround limited to OS X 10.10, so next release should "fix" this issue. Sadly though it will limit colorspace to sRGB. Also we need to check which OS X versions are affected.

#19 Updated by David Wang almost 5 years ago

Igor, I also have Rawtherapee installed on the same system which does not have this issue. In fact, Rawtherapee doesn't even allow you to specify display output profile, not even on Linux platform. Maybe it does no harm to take a look at how they do it?
That said, I really love the UI and local adjustment of DT. :-)

Moreover, shall we open a new ticket to discuss Linux scenario?

#20 Updated by David Wang almost 5 years ago

One more finding: I am surprised to find out that changing display color profile in DT actually changes the histogram at the same time. I can see, at least, dark point and white point shift.
This sounds confusing: do I have to develop my photo according to each individual display monitor? I thought histogram was only affected by input and output color profiles, not display color profile. Am I wrong?

#21 Updated by Igor Kuzmin almost 5 years ago

Linux scenario is known to work, so what you're observing there has to be configuration error.
Histogram issue has been known for a long time, there are plans to fix it.
Rawtherapee just outputs sRGB on OS X, just we will do now: http://50.87.144.65/~rt/w/index.php?title=Preferences#Color_Management_Tab

#22 Updated by Igor Kuzmin over 4 years ago

  • % Done changed from 50 to 100
  • Status changed from In Progress to Fixed

Switched to sRGB on all OS X versions which is the best we can do for now (until cairo gets support for specifying surface colorspace). See https://bugzilla.gnome.org/show_bug.cgi?id=681784#c12 for more technical details.

#23 Updated by Roman Lebedev over 3 years ago

  • Target version changed from Candidate for next minor release to 2.2.0

Also available in: Atom PDF

Go to top