Project

General

Profile

Bug #11626

lens correction faulty for arw from Sony DSC RX100M3

Added by G L about 2 years ago. Updated almost 2 years ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
General
Start date:
05/28/2017
Due date:
% Done:

0%

Estimated time:
Affected Version:
2.2.4
System:
Ubuntu
bitness:
64-bit
hardware architecture:
amd64/x86

Description

I am trying to setup a small workflow to generate jpgs from the raw files which should involve among others the lens correction for the camera. Luckily I found that darktable is in principle able to do this, however some controls showed me that the lens correction is somehow done but differs from the in camera processing and the digikam/lensfun method.

Please find attached the corresponding jpegs with self-explanatory filenames.

My ultimate goal is to setup a batch script which takes the raws and makes a little processing to generate full-size jpegs involving rotation, lens correction, denoising and base curve transformation. These jpegs should be similar in quality to the out of camera jpegs and would allow me to shoot raw only (instead of raw + jpegs) and give me to possibility to process the most interesting raws manually. I tried to use ufraw but the programme fails to do the lens correction at all.

History

#1 Updated by Roman Lebedev about 2 years ago

G L wrote:

I am trying to setup a small workflow to generate jpgs from the raw files which should involve among others the lens correction for the camera. Luckily I found that darktable is in principle able to do this, however some controls showed me that the lens correction is somehow done but differs from the in camera processing and the digikam/lensfun method.

Please find attached the corresponding jpegs with self-explanatory filenames.

My ultimate goal is to setup a batch script which takes the raws and makes a little processing to generate full-size jpegs involving rotation, lens correction, denoising and base curve transformation. These jpegs should be similar in quality to the out of camera jpegs and would allow me to shoot raw only (instead of raw + jpegs) and give me to possibility to process the most interesting raws manually. I tried to use ufraw but the programme fails to do the lens correction at all.

This is sony, which does stupid shit in terms of applying lens correction to the raw image data. Are you sure that ALL in-camera corrections are disabled?

#2 Updated by G L about 2 years ago

The in-camera corrections are basically default values. As I understand these settings change the way the jpeg is generated but do not affect the raw picture itself. The raw should be correct as using digikam/lensfun lens corrections yields a jpeg very similar to the OOC jpeg.

#3 Updated by Roman Lebedev about 2 years ago

G L wrote:

The in-camera corrections are basically default values. As I understand these settings change the way the jpeg is generated but do not affect the raw picture itself. The raw should be correct as using digikam/lensfun lens corrections yields a jpeg very similar to the OOC jpeg.

Hint: if you want to get help, do follow the questions we are asking. If i'm saying that sony modifies raw data, then that is the sad truth.

#4 Updated by G L about 2 years ago

I have the following settings:

size: 20M
ratio: 3/2
quality: raw & & jpeg
red eye reduction: On (just for flashing ?)
exposure correction: 0
ISO: auto
ND filter: off
white balance: auto
DRO: auto
creative mode: standard
picture effect: off
high ISO noise reduction: normal
soft-skin effect: off
color profile: sRGB

I did not find anything connected to the lens itself. Hope this helps !

#5 Updated by Roman Lebedev about 2 years ago

G L wrote:

Please find attached the corresponding jpegs with self-explanatory filenames.

More seriously though, If you are going to be processing raw images, then for everyone's sanity it is best to forget about in-camera jpegs, and embedded thumbnails.
Processing results (between OOC jpeg, some program, and other programs) will always differ.

So, what is the problem, specifically?

#6 Updated by G L about 2 years ago

The problem is the lens correction in darktable. RX100 lens is quite poor optically and the barrel should be corrected during raw processing. In order to judge the lens correction I use as benchmark the Ooc resp.the embedded jpeg. The correction in digikam/lensfun seems to be correct, but digikam is not so good for raw processing. Darktable recognizes the camera and does lens correction (compare both dt versions) but both outputs do not match with Ooc,embedded nor the digikam / lensfun jpeg. As darktable is also using lensfun I wonder whether the implementation is correct. Does it also takes into account the focal length for example?

#7 Updated by Roman Lebedev about 2 years ago

G L wrote:

The problem is the lens correction in darktable. RX100 lens is quite poor optically and the barrel should be corrected during raw processing.

I still did not got a straight answer about in-camera corrections.
If put camera on tripod, enabled in-camera lens corrections, take a raw image, disable all in-camera corrections, and take another raw, how does these two raws look like, when loaded in darktable? Do they differ?

In order to judge the lens correction I use as benchmark the Ooc resp.the embedded jpeg. The correction in digikam/lensfun seems to be correct, but digikam is not so good for raw processing. Darktable recognizes the camera and does lens correction (compare both dt versions) but

both outputs do not match with Ooc,embedded nor the digikam / lensfun jpeg.

Like i said, outputs will never match.

As darktable is also using lensfun I wonder whether the implementation is correct.

Does it also takes into account the focal length for example?

Sure it does :)
Perhaps it is the lensfun's lens profile that is broken, especially since this is sony.

Before posting any new comments, do the test from the beginning of this my comment.

#8 Updated by G L about 2 years ago

I still did not got a straight answer about in-camera corrections.
If put camera on tripod, enabled in-camera lens corrections, take a raw image, disable all in-camera corrections, and take another raw, how does these two raws look like, when loaded in darktable? Do they differ?

Sorry there is no such setting to switch on/off lens correction. I read somewhere that the lens correction is stored in parameters in the raw file. Could not find the page anymore, but I found this: https://redmine.darktable.org/issues/9886 also stating this.

Greetings

#9 Updated by G L about 2 years ago

I repeated the experiment with a chequered picture. My settings for lens correction were not correct.
Default settings is: Thoby fisheye, when using linear ("geradlinig") the lens correction is perfect (see pictures attached).

#10 Updated by Roman Lebedev about 2 years ago

G L wrote:

I repeated the experiment with a chequered picture. My settings for lens correction were not correct.

Default settings is: Thoby fisheye

Ugh.
Where did you get that dt from, and from where is the lensfun?

, when using linear ("geradlinig") the lens correction is perfect (see pictures attached).

Good to hear that!

#12 Updated by Roman Lebedev about 2 years ago

G L wrote:

dt (2.4.2): http://ppa.launchpad.net/pmjdebruijn/darktable-release/ubuntu

either purge (downgrade) the lensfun, or rebuild darktable from sources.

lensfun (0.3.2): http://ppa.launchpad.net/philip5/extra/ubuntu

Lensfun has broken their ABI in <= 0.3.2, without bumping the API version.
The symptoms - "Default settings is: Thoby fisheye"

#13 Updated by Roman Lebedev about 2 years ago

So with the default geometry issue fixed, are there remaining issues here?

#14 Updated by G L about 2 years ago

I now defined a style including the lens correction step and applied this to jpeg export from raw. This seems to work but I have to test with additional pictures whether pictures taken at different focal length get correctly corrected or whether the same focal length is applied regardless of metadata of the raw.

#15 Updated by Roman Lebedev about 2 years ago

G L wrote:

I now defined a style including the lens correction step and applied this to jpeg export from raw. This seems to work but I have to test with additional pictures whether pictures taken at different focal length get correctly corrected or whether the same focal length is applied regardless of metadata of the raw.

I have explained more than once how to do that in the mailing lists, google it.

#16 Updated by G L almost 2 years ago

Still struggling with the lens correction. I cannot downgrade lensfun since digikam depends on 0.3.1.

Under default preset the lens correction correctly loads camera model, lens, focal length and aperture but the geometry is Thoby Fisheye (but should be linear). To correct for this I change the geometry to linear and save this new preset while defining that the preset is applied only with Sony DSC-RX100M3 and for all ISOs, all exposure times, all focal lengths and all apertures.

When opening a new raw and applying lens correction the new preset are automatically taken BUT the preset uses the focal length and aperture NOT from the metadata of the current raw but from the raw when I defined the preset. It appears to me that the newly defined preset includes fixed settings for aperture and focal length in contrast to the original default preset.

How can I solve this problem? Define a preset for every focal length (as this is the most important parameter for the barrel distortion) or is there a more generic solution?

#17 Updated by Roman Lebedev almost 2 years ago

G L wrote:

Under default preset the lens correction correctly loads camera model, lens, focal length and aperture but the geometry is Thoby Fisheye (but should be linear).

The problem is that lensfun broke abi (that wrong geometry you see) without bumping soname.
That lensfun is installed from some third-party PPA? If yes, then remove the PPA and make sure that lensfun and darktable are from pmjdebruijn's PPA.

#18 Updated by G L almost 2 years ago

I downgraded to 2.8.3 via ppa:pmjdebruijn/darktable-release. Together with dt 2.2.5 there are issues with the drop-down menu (as observed by others). I will try to do the work around as described in my last post. At low focal length there is a very strong barrel distortion and lens correction is absolutely required for this camera.

#19 Updated by G L almost 2 years ago

Workaround works quite good.

Also available in: Atom PDF

Go to top