Project

General

Profile

Bug #11279

Black image exported with lens distortion correction and opencl

Added by Emilian Bogatu over 3 years ago. Updated over 3 years ago.

Status:
Fixed
Priority:
Low
Category:
Darkroom
Target version:
Start date:
10/31/2016
Due date:
% Done:

100%

Estimated time:
Affected Version:
2.0.7
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

On particular raw files, image exported is black (files have small size, 1/10 of "normal" exported) if lens distortion correction (with opencl) is activated.
Some raw files are partially exported: there are visible black rectangles in corners(on one file left upper and bottom part of image).

Less than 10% of raw files are affected (completely or partial black images), I cannot find a rule about. However at least with DT 2.0.6 and 2.0.7, the same raw files cannot be exported with opencl and lens distortion correction applied. One sample attached and the exported black jpg file.

Lens/camera is automatically detected in lens correction module. Image can be exported via CPU path with all corrections from lens correction module (opencl disabled). No issue with TCA & vignetting, only distortion correction seems to trigger the black image result with opencl activated.

Nothing abnormal visible in console during export (output attached for the attached raw test file).

Nvidia drivers updated to last available in Ubuntu 16.04 with no effect. Same for lensfun (purged liblensfun* packages, installed latest lensfun version available from source and DT 2.0.7)

darktable_opencl.txt (23.8 KB) darktable_opencl.txt Emilian Bogatu, 10/31/2016 09:38 PM
ITAL6492.PEF.xmp (17.4 KB) ITAL6492.PEF.xmp Emilian Bogatu, 10/31/2016 09:38 PM
ITAL6492.PEF (28 MB) ITAL6492.PEF Emilian Bogatu, 10/31/2016 09:38 PM
ITAL6492.jpg (341 KB) ITAL6492.jpg Emilian Bogatu, 10/31/2016 09:38 PM

Associated revisions

Revision 26eaec44 (diff)
Added by Ulrich Pegelow over 3 years ago

lens: disable tiling in the lens correction module

This is a quick fix for bug #11279. The underlying cause of that bug is a
region of interest mismatch during tiling, thus affecting mostly file export with
OpenCL and low to medium memory GPUs. A real fix would require a rework of
the interplay of lens.c:modify_roi_in() and the corresponding tiling routines
and could not be implemented without heavy testing. As we are shortly before
a release the quick fix disables tiling in that module. This will give a
certain slow-down on the mentioned low to medium memory GPUs but only during
export which is tolerable.

Revision d45fb971 (diff)
Added by Ulrich Pegelow over 3 years ago

lens: disable tiling in the lens correction module

This is a quick fix for bug #11279. The underlying cause of that bug is a
region of interest mismatch during tiling, thus affecting mostly file export with
OpenCL and low to medium memory GPUs. A real fix would require a rework of
the interplay of lens.c:modify_roi_in() and the corresponding tiling routines
and could not be implemented without heavy testing. As we are shortly before
a release the quick fix disables tiling in that module. This will give a
certain slow-down on the mentioned low to medium memory GPUs but only during
export which is tolerable.

(cherry picked from commit 26eaec446015c99a04e4af383de09a39f1575a5b)

History

#1 Updated by Ulrich Pegelow over 3 years ago

  • % Done changed from 0 to 20
  • Status changed from New to Triaged

I can confirm that there is an issue with lens correction and OpenCL. I don't get a black output, instead in my case darktable crashes during export when running on an NVIDIA GPU.

#2 Updated by Ulrich Pegelow over 3 years ago

  • System changed from Ubuntu to all
  • Assignee set to Ulrich Pegelow
  • Category changed from Lighttable to Darkroom

#3 Updated by Emilian Bogatu over 3 years ago

Until this issue can be corrected, is there a quick/simple way to run only specific modules (lens correction in this case) on CPU while opencl is still globally active?

#4 Updated by Ulrich Pegelow over 3 years ago

If you compile darktable by yourself, you can rename process_cl() in src/iop/lens.c to some other name like xprocess_cl(). This way the lens correction module would not be processed via OpenCL.

#5 Updated by Ulrich Pegelow over 3 years ago

  • % Done changed from 20 to 100
  • Status changed from Triaged to Fixed

#6 Updated by Emilian Bogatu over 3 years ago

Got the git version with changeset applied (after a purge of dt 2.0.7).
With the initial attached raw file I have the same issue - black image with opencl and distortion correction applied. Since I had no crash (GTX960/4GB) and only some files are affected (as reported), there might be a second issue still undetected.
This is low priority and I know that you have a lot of work before release. Further more, there is a good chance that something is still wrong on my side.
Before closing this issue, can you confirm that with this commit and initial attached files you don't get a black image export?
Thank you!

#7 Updated by Ulrich Pegelow over 3 years ago

No black image here.

Please do one additional test on your side. Go into the lens correction module and press on the the "auto scale" button (the circle-like button right to the "scale" slider). This should bring the scale value to something like 1.003. Then please export again. Still a black output?

#8 Updated by Emilian Bogatu over 3 years ago

auto scale -> scale is on 1.003, still black output.
Ok, seems to be something on my side; I'll try to figure why ..
Thank you!

#9 Updated by Ulrich Pegelow over 3 years ago

A few tips.

You can run 'darktable -d nan' do the export and post the output here. This could help to narrow down the problem.

Another idea: your settings of opencl_event_handles is extremely high (500). I would recommend to reduce this to 50 or even 25 and see what happens.

#10 Updated by Emilian Bogatu over 3 years ago

25 or 50 did the trick and image is properly exported. Nice!
Only in the initial report was 500 (event missing signalled in console due to atrous -> memory headroom had to be increased to 7-800), in rest default value.
Is there a way to find the optimum value (default seems too high)? Or rise debug level to catch in console some messages when wrong values are used?

#11 Updated by Roman Lebedev over 3 years ago

  • Target version set to 2.2.0

#12 Updated by Ulrich Pegelow over 3 years ago

Good to hear that it works now!

For your information: the default value for opencl_event_handles is 25 in darktable. So it looks like you had changed it at some time. The default has proven to be a good choice in many cases. Some hints on optimization can be seen in the usermanual ([[http://www.darktable.org/usermanual/ch10s02s07.html.php]]). The maximum number of handles supported by the device cannot be found out - at least there is no documented option. Therefore we need to make an assumption and that's what the parameter reflects. If we are wrong and the chosen number is too high we expect to see an OpenCL error -5 (CL_OUT_OF_RESOURCES) at some point, which darktable would recognize and continue processing on the CPU. Interestingly, your device/driver does not report this error and so the problem is only noticed by the black output. Maybe a bug in your driver version. Is that version already published as stable by NVIDIA? AFAIK the latest stable series is still 367.xx.

Also available in: Atom PDF

Go to top