Project

General

Profile

Bug #9591

Local contrast module produces strong artifacts only visible in exported TIFFs/JPEGs

Added by Frank Stefani over 6 years ago. Updated over 6 years ago.

Status:
Fixed
Priority:
Medium
Assignee:
Category:
Darkroom
Target version:
-
Start date:
09/24/2013
Due date:
% Done:

100%

Estimated time:
Affected Version:
git stable branch
System:
Ubuntu
bitness:
64-bit
hardware architecture:
amd64/x86

Description

[Darktable-1.2.3]
[Ubuntu-12.04.3 LTS]
[Intel i7, 8 cores, 16 GB RAM]

FILES USED: Canon CR2 raw files from Canon EOS 1Ds Mark III and Canon EOS 7D

SCREENSHOT:
https://secure.flickr.com/photos/67091795@N02/9916437903/in/pool-darktable

right image: what I intended to get (100% preview)
left image: what darktable exported (100% preview)

  • these artifacts are visible on paper prints and in all image viewers
  • standard darktable settings were used to reproduce and track down this error

DISCUSSION:
https://secure.flickr.com/groups/darktable/discuss/72157635796808093/

Thanks for your investigation

- slowpic

DT-Local.Contrast-1-20-1.000-Export.Error.png (1.9 MB) DT-Local.Contrast-1-20-1.000-Export.Error.png Screenshot of local contrast export error Frank Stefani, 09/24/2013 06:47 PM
Siebeneicher-13-075.cr2 (24.7 MB) Siebeneicher-13-075.cr2 Original RAW file for screen shot demonstration Frank Stefani, 09/24/2013 06:47 PM
Siebeneicher-13-075.cr2.xmp (16.2 KB) Siebeneicher-13-075.cr2.xmp Associated XMP file for screen shot demonstration Frank Stefani, 09/24/2013 06:47 PM
DT-local-contrast-1-20-0.900.png (1.62 MB) DT-local-contrast-1-20-0.900.png 100% comparison of DT settings (right) and what was exported (left) Frank Stefani, 09/26/2013 11:15 AM
DT-local-contrast-20-20-0.900.png (1.6 MB) DT-local-contrast-20-20-0.900.png 100% comparison of DT settings (right) and what was exported (left) Frank Stefani, 09/26/2013 11:15 AM
local-contrast-20-20-0.900.jpg (17.9 MB) local-contrast-20-20-0.900.jpg First parameter "20": original and export look very similar Frank Stefani, 09/26/2013 11:15 AM
local-contrast-1-20-0.900.jpg (18 MB) local-contrast-1-20-0.900.jpg First parameter "1": original and export differ harshly Frank Stefani, 09/26/2013 11:15 AM

History

#1 Updated by Ulrich Pegelow over 6 years ago

  • % Done changed from 0 to 20
  • Status changed from New to Incomplete

I checked your example with current git version. Taking the example raw and XMP you supplied. Activating local contrast has the expected effect. Exporting that images (JPEG or TIFF) and comparing exported version (in Gimp) versus darktable's display does not show any significant differences. Tested with CPU and GPU codepath.

Two requests:

1) is there any terminal output during export on your side? (start darktable from a terminal then export as you did and watch for the output)

2) can you try the latest git version? (make sure to backup your database in .config/darktable/library.db before)

#2 Updated by Ulrich Pegelow over 6 years ago

Forgot one point:

3) please double-check that there is no style applied during export (export panel in lighttable mode, combobox "style" should say "none")

#3 Updated by Frank Stefani over 6 years ago

I double checked no style was applied during export.
The exported image looks as bad when opened with Gimp.

On direct export to 16-bit TIFF from the darkroom (Ctrl+E) the following lines (darktable -d all) appeared:

----------
[sql] prepare "delete from history where imgid = ?1" 
[sql] prepare "select num from history where imgid = ?1 and num = ?2" 
[sql] prepare "insert into history (imgid, num) values (?1, ?2)" 
[sql] prepare "update history set operation = ?1, op_params = ?2, module = ?3, enabled = ?4, blendop_params = ?7, blendop_version = ?8, multi_priority = ?9, multi_name = ?10 where imgid = ?5 and num = ?6" 
[sql] prepare "SELECT id FROM tags WHERE name = ?1" 
[sql] prepare "INSERT OR REPLACE INTO tagged_images (imgid, tagid) VALUES (?1, ?2)" 
[sql] prepare "UPDATE tagxtag SET count = count + 1 WHERE (id1 = ?1 AND id2 IN (SELECT tagid FROM tagged_images WHERE imgid = ?2)) OR (id2 = ?1 AND id1 IN (SELECT tagid FROM tagged_images WHERE imgid = ?2))" 
[sql] prepare "select distinct id from images where id in (select imgid from selected_images) order by filename" 
[add_job] 0 export
[run_job+] 08 89501967.225392 export
[sql] prepare "SELECT id FROM tags WHERE name = ?1" 
[sql] prepare "SELECT id FROM tags WHERE name = ?1" 
[sql] prepare "UPDATE tagxtag SET count = count - 1 WHERE (id1 = ?1 AND id2 IN (SELECT tagid FROM tagged_images WHERE imgid = ?2)) OR (id2 = ?1 AND id1 IN (SELECT tagid FROM tagged_images WHERE imgid = ?2))" 
[sql] prepare "DELETE FROM tagged_images WHERE tagid = ?1 AND imgid = ?2" 
[sql] prepare "INSERT OR REPLACE INTO tagged_images (imgid, tagid) VALUES (?1, ?2)" 
[sql] prepare "UPDATE tagxtag SET count = count + 1 WHERE (id1 = ?1 AND id2 IN (SELECT tagid FROM tagged_images WHERE imgid = ?2)) OR (id2 = ?1 AND id1 IN (SELECT tagid FROM tagged_images WHERE imgid = ?2))" 
[sql] prepare "select folder || '/' || filename from images, film_rolls where images.film_id = film_rolls.id and images.id = ?1" 
[sql] prepare "select folder || '/' || filename from images, film_rolls where images.film_id = film_rolls.id and images.id = ?1" 
[sql] prepare "select folder || '/' || filename from images, film_rolls where images.film_id = film_rolls.id and images.id = ?1" 
[sql] prepare "select folder || '/' || filename from images, film_rolls where images.film_id = film_rolls.id and images.id = ?1" 
[sql] prepare "select imgid, num, module, operation, op_params, enabled, blendop_params, blendop_version, multi_priority, multi_name from history where imgid = ?1 order by num" 
[export] creating pixelpipe took 0.058 secs (0.060 CPU)
[pixelpipe_process] [export] using device 0
[memory] before pixelpipe process
[memory] max address space (vmpeak):    33886064 kB
[memory] cur address space (vmsize):    33873816 kB
[memory] max used memory   (vmhwm ):     1259148 kB
[memory] cur used memory   (vmrss ):      685212 kB
pixelpipe cacheline 0 used 0 by 18446744073709551615
pixelpipe cacheline 1 used 0 by 18446744073709551615
cache hit rate so far: -nan
[dev_pixelpipe] took 0.000 secs (0.000 CPU) initing base buffer [export]
[dev_pixelpipe] took 0.011 secs (0.012 CPU) processing `Weißabgleich' [export]
[dev_pixelpipe] took 0.004 secs (0.004 CPU) processing `Spitzlicht-Rekonstruktion' [export]
[dev_pixelpipe] took 0.023 secs (0.000 CPU) processing `Entrastern' [export]
[dev_pixelpipe] module `Entrastern' min: (-0.000147; 0.000000; -0.015737) max: (1.089860; 1.000000; 1.067321) [export]
[dev_pixelpipe] took 0.009 secs (0.008 CPU) processing `Eingabefarbprofil' [export]
[dev_pixelpipe] module `Eingabefarbprofil' min: (0.632485; -21.235556; -31.693195) max: (101.328789; 63.217014; 96.924347) [export]
[dev_pixelpipe] took 0.070 secs (0.044 CPU) processing `Lokaler Kontrast' [export]
[dev_pixelpipe] module `Lokaler Kontrast' min: (-8.721398; -21.235556; -31.693195) max: (111.109955; 63.217014; 96.924347) [export]
[dev_pixelpipe] took 0.022 secs (0.008 CPU) processing `Ausgabefarbprofil' [export]
[dev_pixelpipe] module `Ausgabefarbprofil' min: (0.000000; 0.000000; 0.000000) max: (1.161822; 1.132581; 1.163418) [export]
[opencl_profiling] spent  0.0069 seconds in [Write Image (from host to device)]
[opencl_profiling] spent  0.0028 seconds in whitebalance_1ui
[opencl_profiling] spent  0.0031 seconds in highlights_1f
[opencl_profiling] spent  0.0077 seconds in ppg_demosaic_green
[opencl_profiling] spent  0.0117 seconds in ppg_demosaic_redblue
[opencl_profiling] spent  0.0026 seconds in border_interpolate
[opencl_profiling] spent  0.4285 seconds in [Read Image (from device to host)]
[opencl_profiling] spent  0.0066 seconds in colorin
[opencl_profiling] spent  0.0002 seconds in zero
[opencl_profiling] spent  0.0479 seconds in splat
[opencl_profiling] spent  0.0003 seconds in [Copy Buffer to Buffer (on device)]
[opencl_profiling] spent  0.0112 seconds in blur_line
[opencl_profiling] spent  0.0003 seconds in blur_line_z
[opencl_profiling] spent  0.0079 seconds in slice
[opencl_profiling] spent  0.0198 seconds in colorout
[opencl_profiling] spent  0.5575 seconds totally in command queue (with 0 events missing)
[dev_process_export] pixel pipeline processing took 3.119 secs (3.068 CPU)
[sql] prepare "select folder || '/' || filename from images, film_rolls where images.film_id = film_rolls.id and images.id = ?1" 
[sql] prepare "select value from meta_data where id = ?1 and key = ?2 order by value" 
[sql] prepare "select value from meta_data where id = ?1 and key = ?2 order by value" 
[sql] prepare "select value from meta_data where id = ?1 and key = ?2 order by value" 
[sql] prepare "select value from meta_data where id = ?1 and key = ?2 order by value" 
[sql] prepare "select flags from images where id = ?1" 
[sql] prepare "select folder || '/' || filename from images, film_rolls where images.film_id = film_rolls.id and images.id = ?1" 
[sql] prepare "select filename, flags, raw_parameters, longitude, latitude from images where id = ?1" 
[sql] prepare "select key, value from meta_data where id = ?1" 
[sql] prepare "SELECT DISTINCT T.id, T.name FROM tagged_images JOIN tags T on T.id = tagged_images.tagid WHERE tagged_images.imgid = 474" 
[sql] prepare "SELECT DISTINCT T.id, T.name FROM tagged_images JOIN tags T on T.id = tagged_images.tagid WHERE tagged_images.imgid = 474" 
[sql] prepare "select color from color_labels where imgid=?1" 
[sql] prepare "select imgid, num, module, operation, op_params, enabled, blendop_params, blendop_version, multi_priority, multi_name from history where imgid = ?1 order by num" 
[export_job] exported to `/foto/2013/09/14.09.13-Hochzeit-Eichinger/result.darktable/local.contrast-Siebeneicher-13-075_01.tif'
[run_job-] 08 89501974.553859 export
----------

#4 Updated by Ulrich Pegelow over 6 years ago

Seeing that your pixelpipe runs on GPU, I'd like to ask you to test with OpenCL disabled (in preferences->core options). If your problem goes away with disabled OpenCL, please as a second test go into .config/darktable/darktablerc, look for config varibable opencl_avoid_atomics, set it to TRUE and run again with OpenCL enabled.

#5 Updated by Frank Stefani over 6 years ago

I tried what you finally suggested and compiled the git version 1.3+1044-gcd0b294 with all dev libraries in the required version as documented in the Wiki. I moved my original ~/.config/darktable away and cleaned the cache. Then, I started the git version and opened the image I referred to earlier and applied nothing but local contrast with the mentioned values, ie. "1", "20", "1.000". Evereything else, esp. the export color management and style option, I left untouched. I only selected 16-bit TIFF as the file format.

Unfortunately, the result is as bad as it is in the current stable "plus" version 1.2.3 which I used up to now.

What else can I try to narrow the problem?

Thanks for your investigation.

#6 Updated by Frank Stefani over 6 years ago

@Ulrich: The openCL flag is set but the whole line in the settings dialog ist dark grayed out. How can I disable it?

#7 Updated by Ulrich Pegelow over 6 years ago

That's strange. A greyed out "OpenCL" field in core options means that darktable does not detect OpenCL on your system. The debug output you posted earlier has lines that say "[opencl_profiling] ..." which can only be emitted if you have OpenCL active.

Anyhow, please leave the OpenCL field greyed out. Does the problem still persist? Please also make available an exported JPEG so we can have a closer look at the problem.

#8 Updated by Frank Stefani over 6 years ago

A greyed out "OpenCL" field in core options means that darktable does not detect OpenCL on your system.

Probably I did not emphasize that the flag OpenCL=1 is set by default. It is also grayed out, ie. unchangeable. This affects only the git version 1.3+1044-gcd0b294. I can switch the flag in the stable plus version 1.2.3

The problem persists in both git and stable version no matter whether OpenCL is set enabled or not.

--------------------------------

On request I attach more screenshots and JPEGs:

  • DT-local-contrast-20-20-0.900.png: With these settings (20,20,0.900) the exported image looks very similar to the darkroom preview.
  • DT-local-contrast-1-20-0.900.png: Obviously a much smaller first parameter (1,20,0.900) makes a big difference on export.

I attach the left side JPEGs also. They are saved with 100% quality. Please note that both exported JPEGs look very similar to each other though their darkroom previews differ strongly.

#9 Updated by Ulrich Pegelow over 6 years ago

  • % Done changed from 20 to 10
  • Status changed from Incomplete to Confirmed

With the proposed settings (1, 20, 0.900) I can confirm your findings - there is a marked difference between preview and exported image. I suspect a problem in the scaling of the sigma_s parameter.

@Jo: could you have a look at that issue. I am not sure how the scaling is expected to work.

#10 Updated by Frank Stefani over 6 years ago

With the proposed settings (1, 20, 0.900) I can confirm your findings

Glad you could reproduce it. It felt too silly being the only one able to reproduce such an error :)

#11 Updated by Tobias Ellinghaus over 6 years ago

  • % Done changed from 10 to 20
  • Status changed from Confirmed to Triaged

#12 Updated by Johannes Hanika over 6 years ago

should be fixed in dee303a, please test.

#13 Updated by Ulrich Pegelow over 6 years ago

  • bitness set to 64-bit

@Frank: any feedback to Jo? Has the issue been fixed?

#14 Updated by Frank Stefani over 6 years ago

Sorry for the delay ...

I downloaded and tried darktable-1.3+1347~g22e4c22 yesterday,
glad to tell that the problem has been resolved.

GREAT THANKS TO A GREAT TEAM!

Would be nice to see this fix in the next stable release
as I don't want to use this development verion for intense
production work, though.

#15 Updated by Ulrich Pegelow over 6 years ago

  • % Done changed from 20 to 100
  • Status changed from Triaged to Fixed

Also available in: Atom PDF

Go to top