Project

General

Profile

Bug #10340

profiled denoise no longer working

Added by Dirk Sagwitz about 5 years ago. Updated almost 5 years ago.

Status:
Fixed
Priority:
Low
Assignee:
-
Category:
Darkroom
Target version:
-
Start date:
02/22/2015
Due date:
% Done:

100%

Estimated time:
Affected Version:
git development version
System:
all
bitness:
64-bit
hardware architecture:
amd64/x86

Description

One of the commits on Friday / Saturday broke the profiled denoise module. Camera profiles are no longer available, only generic profile is shown / used :-(

noiseprofiles.json (551 KB) noiseprofiles.json Dirk Sagwitz, 02/22/2015 10:19 PM
Photo20120827T183948_5339.nef (8.77 MB) Photo20120827T183948_5339.nef D80-sample Dirk Sagwitz, 02/24/2015 06:38 PM
DSC_0646.NEF (27.6 MB) DSC_0646.NEF D7100-sample Dirk Sagwitz, 02/24/2015 06:43 PM
noiseprofiles.c (11.5 KB) noiseprofiles.c Dirk Sagwitz, 02/26/2015 08:28 PM

Associated revisions

Revision 6e7eaacc (diff)
Added by Tobias Ellinghaus about 5 years ago

Revert to substring matching for noiseprofile makers

Fixes #10340.

Revision c8e48f31 (diff)
Added by Tobias Ellinghaus almost 5 years ago

Fix #10340: Code around some old Json-Glib bug

Apparently some old versions of Ubuntu ship a buggy version of
json-glib. Credit for finding the problem and coming up with a fix goes
to Dirk Sagwitz.

History

#1 Updated by Tobias Ellinghaus about 5 years ago

  • % Done changed from 0 to 20
  • Status changed from New to Incomplete

Could you be a little more precise? What camera? What iso? Do you have a sample file?

#2 Updated by Roman Lebedev about 5 years ago

Have you tried removing build/ and /opt/darktable and testing with clean install?

#3 Updated by Dirk Sagwitz about 5 years ago

Yes, I tried with a clean install and the the behaviour is independently from camera / iso. The only entry I have in the profile is "generic poisson distribution". I am using Nikon D80 and Nikon D7100.

Even with "old" images having already applied an iso profile before it's removed.

#4 Updated by Pascal Obry about 5 years ago

I don't have this issue, with a D90 RAW, 550D RAW but I can reproduce with a D800 RAW. Maybe a recent issue due to the migration to JSON format?

#5 Updated by Tobias Ellinghaus about 5 years ago

Do you see something helpful when running like this: darktable -d control | grep noiseprofile?

#6 Updated by Tobias Ellinghaus about 5 years ago

I just looked at a sample from rawsamples.ch, there the maker is "NIKON CORPORATION" while our noise profiles have just "NIKON" (and that didn't change since the old .h format). Could you try if that camera works in 1.6.2?

#7 Updated by Tobias Ellinghaus about 5 years ago

  • Status changed from Incomplete to Triaged

Ok, I found it. We used to look for the maker substring in the one from the profile and now we look for an exact match. What makes more sense?

#8 Updated by Dirk Sagwitz about 5 years ago

darktable -d control | grep noiseprofile

gives me the following:

[noiseprofile] found 13 makers
[noiseprofile] found NIKON CORPORATION
[noiseprofile] found 29 models
[noiseprofile] found NIKON D80
[noiseprofile] found 13 profiles
[noiseprofile] error: missing `name`
[noiseprofile] found 13 makers
[noiseprofile] error: missing `name`
[noiseprofile] found NIKON CORPORATION
[noiseprofile] found 29 models
[noiseprofile] found NIKON D80
[noiseprofile] found 13 profiles
...

#9 Updated by Dirk Sagwitz about 5 years ago

even with the latest commit it's still not working :-(

[noiseprofile] found 13 makers
[noiseprofile] found `NIKON CORPORATION' as `NIKON'
[noiseprofile] found 6 models
[noiseprofile] found `NIKON CORPORATION' as `NIKON CORPORATION'
[noiseprofile] found 29 models
[noiseprofile] found NIKON D80
[noiseprofile] found 13 profiles
[noiseprofile] error: missing `name`

#10 Updated by Tobias Ellinghaus about 5 years ago

  • % Done changed from 20 to 100
  • Status changed from Triaged to Fixed

#11 Updated by Dirk Sagwitz about 5 years ago

to make it work with Nikon D80 I had to change the noiseprofile.json:

{"name": "D80 ...", ...} instead of {"name": "Nikon D80 ...", ....}

#12 Updated by Tobias Ellinghaus about 5 years ago

The only thing you changed are the names? That shouldn't change anything, those are just for displaying, they are never used for anything.

#13 Updated by Dirk Sagwitz about 5 years ago

I thought so, but it was a false positive. If I go back to a commit from 2 days ago denoising is working, the new denosing with json format is not working for me. Neigther D80 nor D7100.

I am still getting those
...
[noiseprofile] error: missing `name`
...

Probably this evening I will build a version with some additional debuging messages. For sure it's a name matching problem ...

#14 Updated by Dirk Sagwitz about 5 years ago

Still no success. I added some additional log prints and it seems to me as if the profiles were found but as soon as the first profile is being read it's bailing out with the message "error: missing `name`" :-(

#15 Updated by Tobias Ellinghaus about 5 years ago

  • % Done changed from 100 to 20
  • Status changed from Fixed to Incomplete

Please provide a sample raw file.

#16 Updated by Pascal Obry about 5 years ago

Working on my side now for the D800.

#17 Updated by Dirk Sagwitz about 5 years ago

Here you an find two RAW files from my cameras, one from the D80, one from the D7100.

I am wondering whether it might be also an environmental problem. I am compiling / running darktable on Ubuntu 14.04, libjson-glib-1.0-common is installed

#18 Updated by Tobias Ellinghaus about 5 years ago

Both files work for me. What version of libjson-glib do you have?

#19 Updated by Dirk Sagwitz about 5 years ago

I am a step further now: Looks as if the comparison / return value of the json_reader_read_member returns "false" instead of "true", at least I was able to read "name", "iso", "a" and "b" elements from the json file when I inverted the expression.

May that be related to the libjson-glib - version installed? What versions are you using?

#20 Updated by Dirk Sagwitz about 5 years ago

I have libjson-glib-1.0.0 in the version 0.16.2-1ubuntu1 (standard for ubuntu 14.04 / 64bit) installed on my system.

#21 Updated by Dirk Sagwitz about 5 years ago

Fixed!

I compiled the libjson-glib from the sources (version 1.0.3) and installed it on /usr/local/lib. After changing LD_LIBRARY_PATH to first include the /usr/local/lib directory and starting darktable the denoise is working. So it's definitely a library version problem.

#22 Updated by Tobias Ellinghaus about 5 years ago

That is bad news. So we either have to find out if that was a bug in old json-glib and have Ubuntu ship a fixed version, or look into something else than json for noise profiles. Or ignore Ubuntu 14.04 for the upcoming dartkable 2.0.

#23 Updated by Dirk Sagwitz about 5 years ago

Most likely a bug in the json-glib version from Ubuntu 14.04, but probably there is a chance to get it running even with the buggy version. During my tests I did the check for availability / unavailability of members in a different way: I've read the list of members and then I accessed directly to those members without checking the return code. With this I was able to access and to read the from the jason, too. Probably this is one way to go. If I'll find the time I will have a look at this this evening.

#24 Updated by Dirk Sagwitz about 5 years ago

Please find attached a version of noiseprofiles.c which is working with the old version of the library, too. It needs some additional error handling, but should show you the way you can go to be compatible with the old and the new version.

#25 Updated by Tobias Ellinghaus almost 5 years ago

  • % Done changed from 20 to 100
  • Status changed from Incomplete to Fixed

Also available in: Atom PDF

Go to top