Pages: 1 ... 61 62 [63] 64 65 ... 78   Go Down

Author Topic: DCamProf - a new camera profiling tool  (Read 767364 times)

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1240 on: May 25, 2016, 10:42:56 am »

Oops, the 8 bit thing is an error in my docs, forgot the -6, it should be:

dcraw -v -r 1 1 1 1 -o 0 -H 0 -T -6 -W -g 1 1 <rawfile>

8 bit or not should not be a difference between success or failure though as far as I know, so it's a mystery. I've always used RawTherapee myself so I haven't run that particular workflow but I don't see why it wouldn't work...

Thanks.  This is for the Leica SL, which generates DNGs natively.

I converted to TIFF copying and pasting the dcraw command line that you have on the tutorial page, I think, so not sure why it's not 16-bit.

I have then taken the resulting TIFF into Photoshop (unmanaged) and used the warp tool to correct perspective, cropped and run scanin as above.
« Last Edit: May 25, 2016, 10:49:23 am by torger »
Logged

jrp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 322
Re: DCamProf - a new camera profiling tool
« Reply #1241 on: May 25, 2016, 11:41:06 am »

Do you need to resize the file?

The scanin docs say that it doesn't like more than 1200 on the longest side, which seems a bit small.  I think that your workflow gives a large file, unless you film the target from a great distance.   (Neither small nor large sizes worked for me.)
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1242 on: May 25, 2016, 02:42:38 pm »

I have never needed to resize from what I remember. If you make the file available I can scan it for you.
Logged

jrp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 322
Re: DCamProf - a new camera profiling tool
« Reply #1243 on: May 25, 2016, 03:06:35 pm »

Thanks Anders.  I have shared a dropbox folder with you.
Logged

GWGill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 608
  • Author of ArgyllCMS & ArgyllPRO ColorMeter
    • ArgyllCMS
Re: DCamProf - a new camera profiling tool
« Reply #1244 on: May 25, 2016, 07:53:17 pm »

Thanks.  The previous 2 posts give those diagnostics.
The previous posts do not have diagnostics for a run using the -a flag, and the suggestion was for your own benefit - i.e. given it is not working normally, don't just blindly use the output even if output is produced using the -a flag.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1245 on: May 26, 2016, 03:07:27 am »

I've tried your file and here's a scan (attached).

However I also had much difficulties scanning it, and I can't guarantee that the attached file is 100% perfect as the matches sits on the borders of the patches. Even if the image looks square and alright to the eye after a perspective correction some tiny residual scale differences between patches throw scanin out of balance and it can't match. It's extremely sensitive, I hadn't really realized how much as I've always used straight perpendicular shots and not needed to perspective correct.

I shall update the manual to stress the importance of perpendicular shots for scanin, as perspective corrected shots rarely seems to be good enough for scanin. This tool is originally designed to read patch sets from scanners (where perspective distortion is zero and rotation is zero) so it does not give much margin for error for camera test target shots.

If you need more forgiving patch scan software you need to turn to one of the commercial softwares like RawDigger.
« Last Edit: May 26, 2016, 03:26:23 am by torger »
Logged

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: DCamProf - a new camera profiling tool
« Reply #1246 on: May 26, 2016, 03:44:34 am »

I shall update the manual to stress the importance of perpendicular shots for scanin, as perspective corrected shots rarely seems to be good enough for scanin. This tool is originally designed to read patch sets from scanners (where perspective distortion is zero and rotation is zero) so it does not give much margin for error for camera test target shots.

Hi Anders,

I understand the compatibility with scanning, but I find it a bit strange that perspective corrected shots cause 'scanin' to trip over the input. I do not understand why it has to be so sensitive to geometry (if that's the issue). Afterall, there is a '-p' flag to adjust for perspective correction. Maybe there is something that Graeme could redesign in his utility?

There is quite a number of situations when not shooting perpendicular to the target can be a common scenario, and sometimes even preferable (to avoid reflections). One can always square the image before feeding it to 'scanin', and/or use the -p flag, but then it should be accepted.

I do understand that resampling changes the grain/noise structure, but since we are averaging over a larger patch area, maybe the changes in noise weigh in too much?

Cheers,
Bart
Logged
== If you do what you did, you'll get what you got. ==

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1247 on: May 26, 2016, 04:38:44 am »

Hi Anders,

I understand the compatibility with scanning, but I find it a bit strange that perspective corrected shots cause 'scanin' to trip over the input. I do not understand why it has to be so sensitive to geometry (if that's the issue). Afterall, there is a '-p' flag to adjust for perspective correction. Maybe there is something that Graeme could redesign in his utility?

There is quite a number of situations when not shooting perpendicular to the target can be a common scenario, and sometimes even preferable (to avoid reflections). One can always square the image before feeding it to 'scanin', and/or use the -p flag, but then it should be accepted.

I do understand that resampling changes the grain/noise structure, but since we are averaging over a larger patch area, maybe the changes in noise weigh in too much?

I haven't had much success with the -p parameter. You'd have to ask Graeme what the limits are supposed to be. From the tests I just did all I can conclude is that if your shot aren't perpendicular you're in for some trouble. I did small small adjustments of perspective correction in jrp's image and suddenly it succeeded matching, but changing a tiny bit (into something that looked even more regular to my eye) it failed again, so it appears pretty random. For perpendicular shots when the only correction required if any is rotation I haven't had issues though. You could still have issues if there's significant distortion in the lens.

I agree that for outdoor shots it can be an advantage to shoot at an angle to minimize reflections, but then it seems like scanin is almost impossible to use.
« Last Edit: May 26, 2016, 04:43:25 am by torger »
Logged

jrp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 322
Re: DCamProf - a new camera profiling tool
« Reply #1248 on: May 26, 2016, 08:36:31 am »

The versions of the files that I tried used the Photoshop Warp tool to transform the target into a rectangular rectangle.

Perhaps it's an exposure rather than a geometry issue?

I tried it with and without -a, but it made no difference.

I did fiddle with RawTherapee but did not pursue it much further as there are, apparently, settings that you have to switch off manually before exporting.

Ideally, you want to mechanical stuff in the workflow to happen mechanically, so that you can concentrate on tuning.

This sensitivity to alignment does seem to be a forbidding weak link in the workflow for dcamprof.
Logged

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: DCamProf - a new camera profiling tool
« Reply #1249 on: May 26, 2016, 09:00:16 am »

I agree that for outdoor shots it can be an advantage to shoot at an angle to minimize reflections, but then it seems like scanin is almost impossible to use.

Yes, and even indoors in a partially bright surrounding. It's no fun (although the most robust profiling workflow) having to shield the entire ambiance with black, including the T-shirt and jeans of the photographer, to avoid potential reflections on the surface of the target.

I hope Graeme will chime in on this potential showstopper. Is it geometry, is it noise (what if we add blur or defocus), or is it ... ?

Cheers,
Bart
Logged
== If you do what you did, you'll get what you got. ==

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1250 on: May 26, 2016, 09:53:49 am »

I've thought about making an own scan tool, it's an interesting programming challenge with pattern recognition. The most robust thing would be to make a GUI where you have a patch grid wireframe that you can manually stretch and bend in place, but then DCamProf would get lots of GUI dependencies. It would be nicer with a command that could search and find a target in a picture and automatically map without any GUI, but it's no easy task.
Logged

howardm

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1984
Re: DCamProf - a new camera profiling tool
« Reply #1251 on: May 26, 2016, 10:24:26 am »

what about opencv or a number of other similar packages?

howardm

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1984
Re: DCamProf - a new camera profiling tool
« Reply #1252 on: May 26, 2016, 10:26:04 am »

what about opencv or a number of other similar packages?

AlterEgo

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1995
Re: DCamProf - a new camera profiling tool
« Reply #1253 on: May 26, 2016, 10:38:52 am »

I've thought about making an own scan tool, it's an interesting programming challenge with pattern recognition. The most robust thing would be to make a GUI where you have a patch grid wireframe that you can manually stretch and bend in place, but then DCamProf would get lots of GUI dependencies. It would be nicer with a command that could search and find a target in a picture and automatically map without any GUI, but it's no easy task.

http://campus.udayton.edu/~ISSL/index.php/research/ccfind/  -> http://campus.udayton.edu/~ISSL/index.php/software/
Logged

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: DCamProf - a new camera profiling tool
« Reply #1254 on: May 26, 2016, 11:14:41 am »

I've thought about making an own scan tool, it's an interesting programming challenge with pattern recognition.

Yes, assuming that the issue with 'scanin' is geometry based.

There are different degrees of sophistication possible, but one could start as scanin does, by assuming a more or less properly squared input. It doesn't have to be perfectly distortion free either, as long as one can get enough of the centers of the patches for a statistically significant number of samples to average.

Strictly speaking, one doesn't even have to use perspective corrected images, just the locations of each of the 4 patch corners from which to take the non-resampled pixel values, and a zone within those coordinates to avoid along the patch edges (to avoid shadows and/or edge blur).  It can also avoid the need for resampling of the source image, just sample and average the pixels within the patch inner boundaries.

A simplification would be to use full pixel row shifts (or only offsets) to align vertical edges a bit better, and a full pixel column shift to align horizontal edges a bit better. Such a trapezoidal de-skewing avoids potential resampling issues (by using only full pixel shifts), and produces somewhat more rectangular ROIs in case a target is rotated.

Quote
The most robust thing would be to make a GUI where you have a patch grid wireframe that you can manually stretch and bend in place, but then DCamProf would get lots of GUI dependencies. It would be nicer with a command that could search and find a target in a picture and automatically map without any GUI, but it's no easy task.

That's more complex (although nice), and not trivial to do for multiple OS environments. But let's see if Graeme can (and is willing to) improve the usability of scanin, by changing whatever is causing the refusal of certain target inputs.
EDIT: Maybe AlterEgo's link gives some more ideas, I haven't read it yet.

Cheers,
Bart
« Last Edit: May 26, 2016, 11:25:15 am by BartvanderWolf »
Logged
== If you do what you did, you'll get what you got. ==

jrp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 322
Re: DCamProf - a new camera profiling tool
« Reply #1255 on: May 26, 2016, 03:13:41 pm »

I've tried your file and here's a scan (attached).

Many thanks, Anders.  I have now tried to generate Natural and Natural+ profiles.

The result is indeed natural looking in Lightroom.  The result is quite desaturated and lower contrast, relative to the regular Xrite or Adobe-generated profiles (and indeed Adobe Standard). The biggest difference is in the Blue and the Red patch.  This is not necessarily a bad thing; I need to test out some more images.

Quote
../dcamprof make-dcp -n "LEICA SL (Typ 601)" -d "Natural+" -t acr -g srgb -o neutral-plus.json my-profile.json natural-plus.dcp
Generating 2.5D HueSatMap with 90x30 = 2700 entries...done!
The tone curve's contrast value is 1.30 (=> auto chroma scaling value 1.121)
Generating 3D LookTable with 90x30x30 = 81000 entries for the neutral tone reproduction operator...
  0%..10%..20%..30%..40%..50%..60%..70%..80%..90%..100%
Writing output to "natural-plus.dcp"...
Complete!

PS: gnuplot on OS X does not like -background gray.

Quote
sage: gnuplot [OPTION] ... [FILE]
  -V, --version
  -h, --help
  -p  --persist
  -d  --default-settings
  -c  scriptfile ARG1 ARG2 ...
  -e  "command1; command2; ..."
gnuplot 5.0 patchlevel 3
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1256 on: May 26, 2016, 03:44:28 pm »

PS: gnuplot on OS X does not like -background gray.

Yeah, it's a mess I've recently found out that it's only working on the X11 terminal :/ as far as I know there's no good standard way to change background color in the others. I've seen hacks like plotting a gray rectangle in the background and stuff like that.
Logged

GWGill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 608
  • Author of ArgyllCMS & ArgyllPRO ColorMeter
    • ArgyllCMS
Re: DCamProf - a new camera profiling tool
« Reply #1257 on: May 26, 2016, 08:40:17 pm »

Even if the image looks square and alright to the eye after a perspective correction some tiny residual scale differences between patches throw scanin out of balance and it can't match. It's extremely sensitive, I hadn't really realized how much as I've always used straight perpendicular shots and not needed to perspective correct.
No, it isn't intended to be this way - quite the contrary.
If someone would like to make the image available to me, then I will look into it.
Logged

GWGill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 608
  • Author of ArgyllCMS & ArgyllPRO ColorMeter
    • ArgyllCMS
Re: DCamProf - a new camera profiling tool
« Reply #1258 on: May 26, 2016, 08:42:30 pm »

The most robust thing would be to make a GUI where you have a patch grid wireframe that you can manually stretch and bend in place, but then DCamProf would get lots of GUI dependencies.
The scanin -F parameters let it be used as a back end to extract patch data from a manually aligned chart.
Logged

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: DCamProf - a new camera profiling tool
« Reply #1259 on: May 27, 2016, 08:13:16 am »

No, it isn't intended to be this way - quite the contrary.
If someone would like to make the image available to me, then I will look into it.

I hope jrp will share it with you. I got the file but as it's his photo I'd like him to share it.
Logged
Pages: 1 ... 61 62 [63] 64 65 ... 78   Go Up