Please help with Argyll scanin

[Hening Bettermann]

I try to make an ICC camera profile for my a7r using Argyll. Scanin does not recognize the reference file, I keep getting the error
"scanin: Error - Scanin failed with code 0x10000004, read_elists: error opening match reference file 'ColorChecker.cht' "

I have tried with the Passport both in full size and reduced to 25 % (this helped in an earlier case, where Graeme told me that the noise from the image broke the pattern). I also tried to just use the part of the image that corresponds to the CC24 and rotate it (like in the error report quoted), but no avail. I have tried to contact both Graeme and the Argyll list, but no response this time. I'm stuck... :-(

[howardm]

do you see that file 'ColorChecker.cht' in the 'ref' folder of the Argyll installation?  Can you open/read it?

Are you on Mac or Win?

[AlterEgo]

I try to make an ICC camera profile for my a7r using Argyll. Scanin does not recognize the reference file, I keep getting the error
"scanin: Error - Scanin failed with code 0x10000004, read_elists: error opening match reference file 'ColorChecker.cht' "

I have tried with the Passport both in full size and reduced to 25 % (this helped in an earlier case, where Graeme told me that the noise from the image broke the pattern). I also tried to just use the part of the image that corresponds to the CC24 and rotate it (like in the error report quoted), but no avail. I have tried to contact both Graeme and the Argyll list, but no response this time. I'm stuck... :-(

if you can share the raw file that might be helpful (somebody can try on their system and may be provide a hint)... otherwise you can use rawdigger / makeinputicc argyll frontend workflow... that shall bypass your issue at all.

PS: http://comments.gmane.org/gmane.comp.graphics.argyllcms/11201

[Hening Bettermann]

do you see that file 'ColorChecker.cht' in the 'ref' folder of the Argyll installation?  Can you open/read it?

Are you on Mac or Win?

I'm on Mac 10.9.5, sorry for not specifying that. The .cht file is in the ref folder, and I can open and read it.

[Hening Bettermann]

if you can share the raw file that might be helpful (somebody can try on their system and may be provide a hint)... otherwise you can use rawdigger / makeinputicc argyll frontend workflow... that shall bypass your issue at all.

PS: http://comments.gmane.org/gmane.comp.graphics.argyllcms/11201

Here is the raw file as well as the TIF for FTP download:
server: landshape.net; user name: ad_hoc@landshape.net; password: hereinspaziert! (including the exclamation mark)

I will have to look at the Rawdigger alternative later - special thanks for the tip.

[AlterEgo]

Here is the raw file as well as the TIF for FTP download:
server: landshape.net; user name: ad_hoc@landshape.net; password: hereinspaziert! (including the exclamation mark)

I will have to look at the Rawdigger alternative later - special thanks for the tip.

here are 3 cgats files from rawdigger (with gamma = 1.0, 1.8, 2.2 - I am not sure which raw converter you want the profile for and scaled to 242 8-bit value) and .cie file to use with makeinputicc GUI frontend for argyll... so you just need to download makeinputicc, update argyll binaries there to v1.7 (released several days ago), and feed the proper (for your converter) cgats file + .cie to it... that's it.

https://app.box.com/s/eycaxf4t2y0ea4n9u8660z4k4jpdtj2a

PS: I do not have issues with scanin finding and using .cht file on my PC/Win8.1x64 - I put the full/absolute path to the file (may be you have issues with access rights in your case on Mac or something like this ?), however scanin has issues like "Error - Scanin failed with code 0x3, Pattern match wasn't good enough"... so that's why I did rawdigger way for you, thus avoiding the hassle with argyll recognition of the target in the file.

[GWGill]

I have tried to contact both Graeme and the Argyll list, but no response this time. I'm stuck... :-(

And I replied  on the ArgyllCMS mailing list here.

I'm not sure why you haven't received it.

[Hening Bettermann]

And I replied  on the ArgyllCMS mailing list here.

I'm not sure why you haven't received it.

Ooops! I must have overlooked it in my In box, or maybe I was too impatient w. r. to the time frame, before I put the list in vacancy mode. Anyway, thank you for chiming in here, and sorry for the extra bother.
I have now put dups of the .cht and .cie files in the same folder as the .tifs, and moved the whole folder from an external drive to the Desktop. I run Argyll by using ShellHere, which opens a Terminal window 'in' the folder where it is clicked. I tried both the full frame and the 1/4 version of the .tif, but still get the same error. 

[Hening Bettermann]

Hi AlterEgo,

thank you for your effort. - The file downloaded from your app.box... link is called 'Hening Bettermann.rar.' When I open it, it says '/Users/hening/Desktop/Hening Bettermann.rar is not RAR archive. No files to extract'
Also, I only have the Exposure edition of Raw Digger. Anyway, I should be able to run Argyll properly, I have done it before, even if I had the same problem before and obviously forgot the solution (as your comments.gmane... link shows). The strange thing is I made a profile for RawTherapee recently, and that worked.
The pattern match problem was the one Graeme solved by advising me to downsample the .tif file. 

[AlterEgo]

thank you for your effort. - The file downloaded from your app.box... link is called 'Hening Bettermann.rar.' When I open it, it says '/Users/hening/Desktop/Hening Bettermann.rar is not RAR archive. No files to extract'

I am using WinRAR 5.xx - it works... I will reupload plain files

try this one = https://app.box.com/s/u9ii8zmk00u27bo2w5i0r0d8vpy5muv9

[Hening Bettermann]

AlterEgo,

thank you, this one works! -
I understand that if I want to go this route, I have to purchase RawDigger Profile version to make the cgats file myself. There is no upgrade path, so that is 90 $. Before I go for that, I hope I can make Argyll work via the command line. I was so proud believing that I had managed to handle that, basically... ;-)

Thanks for your effort!

Do I understand this correctly so that the point is that MakeInputICC makes .ti3 files and profiles from the raw file, independent of raw converter? Is that an advantage, since you will need to convert the raw in the end?

[AlterEgo]

I understand that if I want to go this route

how often do you make profiles ? are you getting better in terms of time & effort results from that route vs just argyll ? may be wait for promotion, they do sometimes... or if once in a while there will be always somebody who can do that for you for free, just for fun - like I did, I never before assembled the data from several grids with RD - so it was just also an exercise for me.

There is no upgrade path, so that is 90 $.

RawDigger Profile Edition + FastRawViewer = $90 both

Do I understand this correctly so that the point is that MakeInputICC

makeinputicc is a free program that automates a lot of argyll work and makes it less painful... there are also some other gui frontends... for example RPP raw converter (donationware) is such front end too... or roughprofiler or CoCoa - if I am not mistaken about these 2 - never used them myself, but I guess they are GUI frontends automating manual tasks

Is that an advantage, since you will need to convert the raw in the end?

there are some advantages - for example rawdigger can do flat fielding , that alone might be worth a lot for a perfectionist who still has to go with targets (and not - see the other topic - with SSFs)

[Hening Bettermann]

After a long Odyssee, I finally figured it out. - I could not understand why scanin did not recognise the tif after I put scanin in the same folder. I suspected scanin was corrupt, and this seems to have been the case, since installing the latest version of Argyll fixed it.

However, the resulting profile had a peak error of 78.996748 and an average error of 23.109304. After much wondering, the diag tif gave a hint: The tif was downscaled to 1/4 the size, and Argyll placed the frames (or what they're called)  partly outside the actual color patches.

OTOH, using the full scale tif  resulted in error "pattern match not good enough", as exspected, which is because the noise of the black frame breaks up the pattern. So downsizing the input tif to 1/2 was the proper compromise. Maybe it also helped to lower the resolution from 360 to 72 dpi.

The resulting profile has a peak error of 8.118006 and an average error of 2.438826. That's acceptable.

AlterEgo, in case it has your interest for your own profiles: MakeInputICC displayed a neutrality check for the profile it made from the raw. The graph shows a divergence between the R, G and B curves, regardless if I chose White Point unscaled or auto-scaled.
Opening the profile in ColorSync and looking at the R, G and B response 'curves', they all have slightly different gamma, and all slightly different from 1.
I'm afraid this would translate to a red cast in a real image.

Braggin' ;-) : The profile I finally managed to produce has gamma=1.0 for all three channels.
I used the following options:
scanin: -G1.0: approximate gamma encoding of image
colprof: -am: creating a matrix profile with gamma=1
-u: If input profile, auto scale WP to allow extrapolation; this means the profile can reproduce whites which are whiter than the whitest patch in the target, here the ColorChecker. This is unlike the standard behavior of profiling.
Furthermore, I manipulated the .ti3 file to allow for the production of pure black:
I added a line 'NEU00' with all values 00.0000, and accordingly increased the number of data sets from 50 to 51. (Tip from Brian Griffith, author of Iridient Developer)
MakeInputICC does not give me all these (and other) choices. There is a price for the convenience of a GUI...

Thanks to you who chimed in!

[AlterEgo]

After a long Odyssee, I finally figured it out. - I could not understand why scanin did not recognise the tif after I put scanin in the same folder. I suspected scanin was corrupt, and this seems to have been the case, since installing the latest version of Argyll fixed it.

However, the resulting profile had a peak error of 78.996748 and an average error of 23.109304. After much wondering, the diag tif gave a hint: The tif was downscaled to 1/4 the size, and Argyll placed the frames (or what they're called)  partly outside the actual color patches.

OTOH, using the full scale tif  resulted in error "pattern match not good enough", as exspected, which is because the noise of the black frame breaks up the pattern. So downsizing the input tif to 1/2 was the proper compromise. Maybe it also helped to lower the resolution from 360 to 72 dpi.

The resulting profile has a peak error of 8.118006 and an average error of 2.438826. That's acceptable.

AlterEgo, in case it has your interest for your own profiles: MakeInputICC displayed a neutrality check for the profile it made from the raw. The graph shows a divergence between the R, G and B curves, regardless if I chose White Point unscaled or auto-scaled.
Opening the profile in ColorSync and looking at the R, G and B response 'curves', they all have slightly different gamma, and all slightly different from 1.
I'm afraid this would translate to a red cast in a real image.

Braggin' ;-) : The profile I finally managed to produce has gamma=1.0 for all three channels.
I used the following options:
scanin: -G1.0: approximate gamma encoding of image
colprof: -am: creating a matrix profile with gamma=1
-u: If input profile, auto scale WP to allow extrapolation; this means the profile can reproduce whites which are whiter than the whitest patch in the target, here the ColorChecker. This is unlike the standard behavior of profiling.
Furthermore, I manipulated the .ti3 file to allow for the production of pure black:
I added a line 'NEU00' with all values 00.0000, and accordingly increased the number of data sets from 50 to 51. (Tip from Brian Griffith, author of Iridient Developer)
MakeInputICC does not give me all these (and other) choices. There is a price for the convenience of a GUI...

Thanks to you who chimed in!

if you want g = 1 profile - you can select pure matrix option in makeinputicc - there you will see that trc tags in icc profile as exactly gamma 1.0 because with gamma = 1.0 you just have only matrix doing anything with your data ... see what makeinputicc is doing = ""D:/TEMP/MakeInputICC_Win/Argyll_bin_Win/colprof" -v  -D "P1030200_RW2.icc"  -qu -am -u -O "Z:/_DSC1039_ARW=G1.0.icc" "Z:/_DSC1039_ARW=G1.0""

as for manipulation - you can as well manipulate with the data feeded to makeinputicc which is nothing more than argyll frontend, no ? try it... not sure that will benefit the end result though

[Hening Bettermann]

Thank you, AlterEgo. You are right about the matrix. But for now, I'm happy to have made this work, so right now, I'm not motivated to spend $90 for MakeInputICC...
Good light!

[AlterEgo]

Thank you, AlterEgo. You are right about the matrix. But for now, I'm happy to have made this work, so right now, I'm not motivated to spend $90 for MakeInputICC...
Good light!

tell us more about what Iridient author, Brian G, said about the pure black feeded into patch data/target description and how it can benefit pure matrix profile

[Hening Bettermann]

Well what he showed me was that the .ti3 file can be read and written with just any text editor. Adding a line with black values of zero, and increasing the 'number of  data sets' accordingly will enable the production of pure black. Without this, the blackest black the profile can produce is the blackest black of the target. In principle, absolute black should be possible, too. This is as I understood it.

[AlterEgo]

Well what he showed me was that the .ti3 file can be read and written with just any text editor. Adding a line with black values of zero, and increasing the 'number of  data sets' accordingly will enable the production of pure black. Without this, the blackest black the profile can produce is the blackest black of the target. In principle, absolute black should be possible, too. This is as I understood it.

what it does practically (I think) it just makes sure that black point tag in icc container is set to all zeroes - and you can simply modify the icc profile directly, the matrix itself is not going to be different with or without that modification in .ti3 - check it, so if you darkest actual patch in your shot used to generate the profile is {r, g, b} any matrix color transform will produce darker output with {r/2, g/2, b/2} data... not sure how iridient raw converter uses that blackpoint tag - does it somehow uses that to ensure that output after color transform is not going to be "darker" than that ? I am curious

[Hening Bettermann]

Hi AlterEgo,
it sounds like you know more about this than I do. I have simply followed Brians advice as I understood it. I have no idea about the inner workings of Iridient. I would not know how to check how the change affects the matrix, or how to manipulate the profile directly. -
Good light!

[AlterEgo]

I guess the origin of this virtual zero black patch is here = http://www.freelists.org/post/argyllcms/Camera-matrix-profile-adding-ti3-perfect-white-data-set

I did not see Iliah Borg or others (except the topic starter, Elle) from that thread to advocate this method though, including the author of Argyll itself... you are visiting that list, your shall know.

PS: for a matrix profile it did not change the matrix itself and RGB(0,0,0) * matrix = RGB(0,0,0) with any matrix - so what is the benefit ?