Project

General

Profile

Camera Support #10523

Raw import for NX500 seem wrong

Added by Josef Wells over 4 years ago. Updated about 4 years ago.

Status:
Fixed
Priority:
Low
Category:
-
Target version:
-
Start date:
06/11/2015
Due date:
% Done:

100%

Estimated time:
Affected Version:
git development version

Description

The Red and Blue channels are crazy high in this SRW file.

Am I just processing this incorrectly?

Raw and Jpg version:
https://www.dropbox.com/sh/l30xtpwawyqh7l0/AADvWdnLFNgYGYNWG9z5AoUNa?dl=0

History

#1 Updated by Pascal de Bruijn over 4 years ago

You're right there is something quite off...

In cameras.xml it should probably say something like this:

<Sensor black="64" white="4096"/>

The above values I just tested manually, so they are probably not quite optimal either.

To verify/properly fix this, we'd need a DNG (using the latest Adobe DNG Converter) converted SRW (preferably shot with a blown out sky), for each ISO level, preferably including third stop ISOs.

#2 Updated by Josef Wells over 4 years ago

Alright, I took the blown-out shots at 1/3 ISO steps and used Adobe DNG Converter to get the dng versions.

I uploaded all of them SRW/DNG to dropbox, same link as above, in the SRW-DNG directory.

Anything else I can do to help?

#3 Updated by Pascal de Bruijn over 4 years ago

Actually, I noticed we don't have wb_presets yet for the NX500, while unrelated to your issue, it would be nice to add them.

Could you shoot a single SRW for each in-camera white balance preset (image content doesn't matter), and upload those as well.

#4 Updated by Josef Wells over 4 years ago

I'll do the wb-presets.

I'm going to do the Enhanced Color Matrix as well, but I've been waiting on a day where I am a) Home around noon and b) it is sunny and clear. Amazingly this hasn't really happened since I got this camera, despite it being summer in Austin, TX.

#5 Updated by Josef Wells over 4 years ago

WB shots added in the same dropbox link under WB_presets.

I just did the defaults, no tweaks, as there is a crazy matrix of adjustments available.

#6 Updated by Josef Wells over 4 years ago

Added Color matrix shots. I didn't do the processing work to figure out which shot is good because the raw handling in darktable is still off. Same dropbox link, under "ColorChart"

I also added some old noise-profile shots I took.. I thought I did them wrong, but maybe it was just the broken raw support. Same dropbox link, under "NoiseProfile"

Anything else I can do? I can build with patches or from git if it helps.

#7 Updated by Josef Wells over 4 years ago

I looked at cameras.xml, and while the blacklevel iso list is incomplete, it has the values that this says it should:
http://www.darktable.org/2012/10/whats-involved-with-adding-support-for-new-cameras/

The whitelevel is also correct.

Is something else going on?

#8 Updated by Pascal de Bruijn over 4 years ago

So I just figured it out, and it was kinda obvious...

$ exiv2 -pt 2015-05-30-19.54.31.SRW | grep -ai bit
Exif.SubImage2.BitsPerSample Short 1 12

$ exiv2 -pt SAM_1703.SRW | grep -ai bit
Exif.SubImage2.BitsPerSample Short 1 14

We're only handling the 14 bit images now, and we are mishandling the 12bit ones. Presumably you can configure this in-camera?

#9 Updated by Josef Wells over 4 years ago

Ahh yeah, Single shot mode is apparerntly 14 bit, Continious is 12.

I've been back and forth there all willy-nilly, so no telling what my samples have been.

Also there is an option for RAW compression lossy/lossless..

How does this affect all of the shots I've uploaded so far? Should I redo all of them using both 12/14 bit modes? How about lossy vs lossless RAW?

#10 Updated by Pascal de Bruijn over 4 years ago

Having a full set of all ISOs for all mode combinations would probably be handy in trying to expand support

#11 Updated by Josef Wells over 4 years ago

Sorry for the delay. I grabbed blown out shots for the Black/White level for single (14bit) and cont (12bit) modes, as well as Lossless vs Lossy RAW modes.

Uploaded to the dropbox site. I've got color-matrix shots as well, when the black/white is correct I'll go through that procedure as well.

#12 Updated by Josef Wells over 4 years ago

I wrote a script to process a directory of .dng and .srw files and write out this report:

Bits Per Sample: 12
 ISO: 100     WL: 4000   BL: 32
 ISO: 125     WL: 4000   BL: 32
 ISO: 160     WL: 4000   BL: 32
 ISO: 200     WL: 4000   BL: 32
 ISO: 250     WL: 4000   BL: 32
 ISO: 320     WL: 4000   BL: 32
 ISO: 400     WL: 4000   BL: 32
 ISO: 500     WL: 4000   BL: 32
 ISO: 640     WL: 4000   BL: 32
 ISO: 800     WL: 4000   BL: 32
 ISO: 1000    WL: 4000   BL: 32
 ISO: 1250    WL: 4000   BL: 32
 ISO: 1600    WL: 4000   BL: 32
 ISO: 2000    WL: 4000   BL: 32
 ISO: 2500    WL: 4000   BL: 32
 ISO: 3200    WL: 4000   BL: 32
 ISO: 4000    WL: 4000   BL: 32
 ISO: 5000    WL: 4000   BL: 32
 ISO: 6400    WL: 4000   BL: 32
 ISO: 8000    WL: 4000   BL: 128
 ISO: 10000   WL: 4000   BL: 128
 ISO: 12800   WL: 4000   BL: 128
 ISO: 16000   WL: 4000   BL: 128
 ISO: 20000   WL: 4000   BL: 128
 ISO: 25600   WL: 4000   BL: 128
Bits Per Sample: 14
 ISO: 100     WL: 16100   BL: 128
 ISO: 125     WL: 16100   BL: 128
 ISO: 160     WL: 16100   BL: 128
 ISO: 200     WL: 16100   BL: 128
 ISO: 250     WL: 16100   BL: 128
 ISO: 320     WL: 16100   BL: 128
 ISO: 400     WL: 16100   BL: 128
 ISO: 500     WL: 16100   BL: 128
 ISO: 640     WL: 16100   BL: 128
 ISO: 800     WL: 16100   BL: 128
 ISO: 1000    WL: 16100   BL: 128
 ISO: 1250    WL: 16100   BL: 128
 ISO: 1600    WL: 16100   BL: 128
 ISO: 2000    WL: 16100   BL: 128
 ISO: 2500    WL: 16100   BL: 128
 ISO: 3200    WL: 16100   BL: 128
 ISO: 4000    WL: 16100   BL: 128
 ISO: 5000    WL: 16100   BL: 128
 ISO: 6400    WL: 16100   BL: 128
 ISO: 8000    WL: 16100   BL: 512
 ISO: 10000   WL: 16100   BL: 512
 ISO: 12800   WL: 16100   BL: 512
 ISO: 16000   WL: 16100   BL: 512
 ISO: 20000   WL: 16100   BL: 512
 ISO: 25600   WL: 16100   BL: 512
 ISO: 51200   WL: 16100   BL: 1024

I don't see how to handle the 12 vs. 14 bits per sample in the cameras.xml, some form of Hints I guess.

#13 Updated by Pedro Côrte-Real over 4 years ago

  • % Done changed from 0 to 100
  • Assignee set to Pedro Côrte-Real
  • Status changed from New to Fixed

Thanks, this should now be fixed in this commit:

https://github.com/darktable-org/darktable/commit/70a60f14581ff66448e0584a70def8a439e38883

Unfortunately it seems at least some of these files are currently broken. This camera uses the latest samsung braindead decoding scheme and apparently there's some bug in my implementation of it. That's a separate issue though so please confirm if this now works well for you.

#14 Updated by Pedro Côrte-Real over 4 years ago

  • % Done changed from 100 to 50
  • Status changed from Fixed to In Progress

The rabbit hole is deeper than expected. It turns out this hits some format bits that weren't implemented yet. I'm not sure but I think that once those are fixed the different white and black points for 12/14bit are no longer needed.

#15 Updated by Pedro Côrte-Real about 4 years ago

  • % Done changed from 50 to 100
  • Status changed from In Progress to Fixed
  • Category deleted (Darkroom)
  • Tracker changed from Bug to Camera Support

#16 Updated by Josef Wells about 4 years ago

Everything looks good. I'll submit this set of files to rawsamples.ch

I'll also get processing on my profiled denoise and enhanced color matrix shots.

Thank you SO much for your hard work on this!

Also available in: Atom PDF

Go to top