Luminous Landscape Forum
Raw & Post Processing, Printing => Colour Management => Topic started by: Hening Bettermann on May 04, 2015, 01:39:45 pm
-
I try to make an ICC camera profile for my a7r using Argyll. Scanin does not recognize the reference file, I keep getting the error
"scanin: Error - Scanin failed with code 0x10000004, read_elists: error opening match reference file 'ColorChecker.cht' "
I have tried with the Passport both in full size and reduced to 25 % (this helped in an earlier case, where Graeme told me that the noise from the image broke the pattern). I also tried to just use the part of the image that corresponds to the CC24 and rotate it (like in the error report quoted), but no avail. I have tried to contact both Graeme and the Argyll list, but no response this time. I'm stuck... :-(
-
do you see that file 'ColorChecker.cht' in the 'ref' folder of the Argyll installation? Can you open/read it?
Are you on Mac or Win?
-
I try to make an ICC camera profile for my a7r using Argyll. Scanin does not recognize the reference file, I keep getting the error
"scanin: Error - Scanin failed with code 0x10000004, read_elists: error opening match reference file 'ColorChecker.cht' "
I have tried with the Passport both in full size and reduced to 25 % (this helped in an earlier case, where Graeme told me that the noise from the image broke the pattern). I also tried to just use the part of the image that corresponds to the CC24 and rotate it (like in the error report quoted), but no avail. I have tried to contact both Graeme and the Argyll list, but no response this time. I'm stuck... :-(
if you can share the raw file that might be helpful (somebody can try on their system and may be provide a hint)... otherwise you can use rawdigger / makeinputicc argyll frontend workflow... that shall bypass your issue at all.
PS: http://comments.gmane.org/gmane.comp.graphics.argyllcms/11201
-
do you see that file 'ColorChecker.cht' in the 'ref' folder of the Argyll installation? Can you open/read it?
Are you on Mac or Win?
I'm on Mac 10.9.5, sorry for not specifying that. The .cht file is in the ref folder, and I can open and read it.
-
if you can share the raw file that might be helpful (somebody can try on their system and may be provide a hint)... otherwise you can use rawdigger / makeinputicc argyll frontend workflow... that shall bypass your issue at all.
PS: http://comments.gmane.org/gmane.comp.graphics.argyllcms/11201
Here is the raw file as well as the TIF for FTP download:
server: landshape.net; user name: [email protected]; password: hereinspaziert! (including the exclamation mark)
I will have to look at the Rawdigger alternative later - special thanks for the tip.
-
Here is the raw file as well as the TIF for FTP download:
server: landshape.net; user name: [email protected]; password: hereinspaziert! (including the exclamation mark)
I will have to look at the Rawdigger alternative later - special thanks for the tip.
here are 3 cgats files from rawdigger (with gamma = 1.0, 1.8, 2.2 - I am not sure which raw converter you want the profile for and scaled to 242 8-bit value) and .cie file to use with makeinputicc GUI frontend for argyll... so you just need to download makeinputicc, update argyll binaries there to v1.7 (released several days ago), and feed the proper (for your converter) cgats file + .cie to it... that's it.
https://app.box.com/s/eycaxf4t2y0ea4n9u8660z4k4jpdtj2a
PS: I do not have issues with scanin finding and using .cht file on my PC/Win8.1x64 - I put the full/absolute path to the file (may be you have issues with access rights in your case on Mac or something like this ?), however scanin has issues like "Error - Scanin failed with code 0x3, Pattern match wasn't good enough"... so that's why I did rawdigger way for you, thus avoiding the hassle with argyll recognition of the target in the file.
-
I have tried to contact both Graeme and the Argyll list, but no response this time. I'm stuck... :-(
And I replied on the ArgyllCMS mailing list here (https://www.freelists.org/post/argyllcms/ColorChecker-Passport-Target,1).
I'm not sure why you haven't received it.
-
And I replied on the ArgyllCMS mailing list here (https://www.freelists.org/post/argyllcms/ColorChecker-Passport-Target,1).
I'm not sure why you haven't received it.
Ooops! I must have overlooked it in my In box, or maybe I was too impatient w. r. to the time frame, before I put the list in vacancy mode. Anyway, thank you for chiming in here, and sorry for the extra bother.
I have now put dups of the .cht and .cie files in the same folder as the .tifs, and moved the whole folder from an external drive to the Desktop. I run Argyll by using ShellHere, which opens a Terminal window 'in' the folder where it is clicked. I tried both the full frame and the 1/4 version of the .tif, but still get the same error.
-
Hi AlterEgo,
thank you for your effort. - The file downloaded from your app.box... link is called 'Hening Bettermann.rar.' When I open it, it says '/Users/hening/Desktop/Hening Bettermann.rar is not RAR archive. No files to extract'
Also, I only have the Exposure edition of Raw Digger. Anyway, I should be able to run Argyll properly, I have done it before, even if I had the same problem before and obviously forgot the solution (as your comments.gmane... link shows). The strange thing is I made a profile for RawTherapee recently, and that worked.
The pattern match problem was the one Graeme solved by advising me to downsample the .tif file.
-
thank you for your effort. - The file downloaded from your app.box... link is called 'Hening Bettermann.rar.' When I open it, it says '/Users/hening/Desktop/Hening Bettermann.rar is not RAR archive. No files to extract'
I am using WinRAR 5.xx - it works... I will reupload plain files
try this one = https://app.box.com/s/u9ii8zmk00u27bo2w5i0r0d8vpy5muv9
-
AlterEgo,
thank you, this one works! -
I understand that if I want to go this route, I have to purchase RawDigger Profile version to make the cgats file myself. There is no upgrade path, so that is 90 $. Before I go for that, I hope I can make Argyll work via the command line. I was so proud believing that I had managed to handle that, basically... ;-)
Thanks for your effort!
Do I understand this correctly so that the point is that MakeInputICC makes .ti3 files and profiles from the raw file, independent of raw converter? Is that an advantage, since you will need to convert the raw in the end?
-
I understand that if I want to go this route
how often do you make profiles ? are you getting better in terms of time & effort results from that route vs just argyll ? may be wait for promotion, they do sometimes... or if once in a while there will be always somebody who can do that for you for free, just for fun - like I did, I never before assembled the data from several grids with RD - so it was just also an exercise for me.
There is no upgrade path, so that is 90 $.
RawDigger Profile Edition + FastRawViewer = $90 both
Do I understand this correctly so that the point is that MakeInputICC
makeinputicc is a free program that automates a lot of argyll work and makes it less painful... there are also some other gui frontends... for example RPP raw converter (donationware) is such front end too... or roughprofiler or CoCoa - if I am not mistaken about these 2 - never used them myself, but I guess they are GUI frontends automating manual tasks
Is that an advantage, since you will need to convert the raw in the end?
there are some advantages - for example rawdigger can do flat fielding , that alone might be worth a lot for a perfectionist who still has to go with targets (and not - see the other topic - with SSFs)
-
After a long Odyssee, I finally figured it out. - I could not understand why scanin did not recognise the tif after I put scanin in the same folder. I suspected scanin was corrupt, and this seems to have been the case, since installing the latest version of Argyll fixed it.
However, the resulting profile had a peak error of 78.996748 and an average error of 23.109304. After much wondering, the diag tif gave a hint: The tif was downscaled to 1/4 the size, and Argyll placed the frames (or what they're called) partly outside the actual color patches.
OTOH, using the full scale tif resulted in error "pattern match not good enough", as exspected, which is because the noise of the black frame breaks up the pattern. So downsizing the input tif to 1/2 was the proper compromise. Maybe it also helped to lower the resolution from 360 to 72 dpi.
The resulting profile has a peak error of 8.118006 and an average error of 2.438826. That's acceptable.
AlterEgo, in case it has your interest for your own profiles: MakeInputICC displayed a neutrality check for the profile it made from the raw. The graph shows a divergence between the R, G and B curves, regardless if I chose White Point unscaled or auto-scaled.
Opening the profile in ColorSync and looking at the R, G and B response 'curves', they all have slightly different gamma, and all slightly different from 1.
I'm afraid this would translate to a red cast in a real image.
Braggin' ;-) : The profile I finally managed to produce has gamma=1.0 for all three channels.
I used the following options:
scanin: -G1.0: approximate gamma encoding of image
colprof: -am: creating a matrix profile with gamma=1
-u: If input profile, auto scale WP to allow extrapolation; this means the profile can reproduce whites which are whiter than the whitest patch in the target, here the ColorChecker. This is unlike the standard behavior of profiling.
Furthermore, I manipulated the .ti3 file to allow for the production of pure black:
I added a line 'NEU00' with all values 00.0000, and accordingly increased the number of data sets from 50 to 51. (Tip from Brian Griffith, author of Iridient Developer)
MakeInputICC does not give me all these (and other) choices. There is a price for the convenience of a GUI...
Thanks to you who chimed in!
-
After a long Odyssee, I finally figured it out. - I could not understand why scanin did not recognise the tif after I put scanin in the same folder. I suspected scanin was corrupt, and this seems to have been the case, since installing the latest version of Argyll fixed it.
However, the resulting profile had a peak error of 78.996748 and an average error of 23.109304. After much wondering, the diag tif gave a hint: The tif was downscaled to 1/4 the size, and Argyll placed the frames (or what they're called) partly outside the actual color patches.
OTOH, using the full scale tif resulted in error "pattern match not good enough", as exspected, which is because the noise of the black frame breaks up the pattern. So downsizing the input tif to 1/2 was the proper compromise. Maybe it also helped to lower the resolution from 360 to 72 dpi.
The resulting profile has a peak error of 8.118006 and an average error of 2.438826. That's acceptable.
AlterEgo, in case it has your interest for your own profiles: MakeInputICC displayed a neutrality check for the profile it made from the raw. The graph shows a divergence between the R, G and B curves, regardless if I chose White Point unscaled or auto-scaled.
Opening the profile in ColorSync and looking at the R, G and B response 'curves', they all have slightly different gamma, and all slightly different from 1.
I'm afraid this would translate to a red cast in a real image.
Braggin' ;-) : The profile I finally managed to produce has gamma=1.0 for all three channels.
I used the following options:
scanin: -G1.0: approximate gamma encoding of image
colprof: -am: creating a matrix profile with gamma=1
-u: If input profile, auto scale WP to allow extrapolation; this means the profile can reproduce whites which are whiter than the whitest patch in the target, here the ColorChecker. This is unlike the standard behavior of profiling.
Furthermore, I manipulated the .ti3 file to allow for the production of pure black:
I added a line 'NEU00' with all values 00.0000, and accordingly increased the number of data sets from 50 to 51. (Tip from Brian Griffith, author of Iridient Developer)
MakeInputICC does not give me all these (and other) choices. There is a price for the convenience of a GUI...
Thanks to you who chimed in!
if you want g = 1 profile - you can select pure matrix option in makeinputicc - there you will see that trc tags in icc profile as exactly gamma 1.0 because with gamma = 1.0 you just have only matrix doing anything with your data ... see what makeinputicc is doing = ""D:/TEMP/MakeInputICC_Win/Argyll_bin_Win/colprof" -v -D "P1030200_RW2.icc" -qu -am -u -O "Z:/_DSC1039_ARW=G1.0.icc" "Z:/_DSC1039_ARW=G1.0""
as for manipulation - you can as well manipulate with the data feeded to makeinputicc which is nothing more than argyll frontend, no ? try it... not sure that will benefit the end result though
-
Thank you, AlterEgo. You are right about the matrix. But for now, I'm happy to have made this work, so right now, I'm not motivated to spend $90 for MakeInputICC...
Good light!
-
Thank you, AlterEgo. You are right about the matrix. But for now, I'm happy to have made this work, so right now, I'm not motivated to spend $90 for MakeInputICC...
Good light!
tell us more about what Iridient author, Brian G, said about the pure black feeded into patch data/target description and how it can benefit pure matrix profile
-
Well what he showed me was that the .ti3 file can be read and written with just any text editor. Adding a line with black values of zero, and increasing the 'number of data sets' accordingly will enable the production of pure black. Without this, the blackest black the profile can produce is the blackest black of the target. In principle, absolute black should be possible, too. This is as I understood it.
-
Well what he showed me was that the .ti3 file can be read and written with just any text editor. Adding a line with black values of zero, and increasing the 'number of data sets' accordingly will enable the production of pure black. Without this, the blackest black the profile can produce is the blackest black of the target. In principle, absolute black should be possible, too. This is as I understood it.
what it does practically (I think) it just makes sure that black point tag in icc container is set to all zeroes - and you can simply modify the icc profile directly, the matrix itself is not going to be different with or without that modification in .ti3 - check it, so if you darkest actual patch in your shot used to generate the profile is {r, g, b} any matrix color transform will produce darker output with {r/2, g/2, b/2} data... not sure how iridient raw converter uses that blackpoint tag - does it somehow uses that to ensure that output after color transform is not going to be "darker" than that ? I am curious
-
Hi AlterEgo,
it sounds like you know more about this than I do. I have simply followed Brians advice as I understood it. I have no idea about the inner workings of Iridient. I would not know how to check how the change affects the matrix, or how to manipulate the profile directly. -
Good light!
-
I guess the origin of this virtual zero black patch is here = http://www.freelists.org/post/argyllcms/Camera-matrix-profile-adding-ti3-perfect-white-data-set
I did not see Iliah Borg or others (except the topic starter, Elle) from that thread to advocate this method though, including the author of Argyll itself... you are visiting that list, your shall know.
PS: for a matrix profile it did not change the matrix itself and RGB(0,0,0) * matrix = RGB(0,0,0) with any matrix - so what is the benefit ?
-
The tif was downscaled to 1/4 the size, and Argyll placed the frames (or what they're called) partly outside the actual color patches.
If the image you included is that case, then I'd suspect that the 7 pieces of white tape have completely thrown the recognition off.
-
Well what he showed me was that the .ti3 file can be read and written with just any text editor. Adding a line with black values of zero, and increasing the 'number of data sets' accordingly will enable the production of pure black. Without this, the blackest black the profile can produce is the blackest black of the target. In principle, absolute black should be possible, too. This is as I understood it.
That assumes that the camera sensor is perfectly zero'd. This isn't always the case.
-
what it does practically (I think) it just makes sure that black point tag in icc container is set to all zeroes - and you can simply modify the icc profile directly...
Note that the black point tag is simply annotation - it has no effect on the actual profile itself.
-
That assumes that the camera sensor is perfectly zero'd. This isn't always the case.
but let us assume that we have such perfect camera with such sensor - how does that virtual black patch (RGB(0,0,0) -> RGB (0,0,0)) is going to alter a simple matrix being generated ?
-
but let us assume that we have such perfect camera with such sensor - how does that virtual black patch (RGB(0,0,0) -> RGB (0,0,0)) is going to alter a simple matrix being generated ?
It will have a weighting in the matrix fit of the sample points.
-
It will have a weighting in the matrix fit of the sample points.
I tried twice and and it did not change anything at all (except bkpt tag)... and how come (0,0,0) -> (0,0,0) shall change anything at all mathematically when you calculate a matrix ? again zero vector * any matrix = zero vector, it shall not by that fact affect the fitting... am I missing something ? if you say it shall have a weighting in how Argyll code does calculate matrix (colprof -am) then why the matrices are the same with and without black patch in .ti3
inserting black point in .ti3 like this
...
SAMPLE_ID SAMPLE_NAME RGB_R RGB_G RGB_B LAB_L LAB_A LAB_B
...
265 "GS00" 0.0 0.0 0.0 0.0 0.0 0.0
...
no changes vs pref. profile
-
> If the image you included is that case, then I'd suspect that the 7 pieces of white tape have completely thrown the recognition off.
Yes but only because scanin placed the frames on them, outside the actual target. With the target image downscaled to 1/2 instead of 1/4, the diag.tif looked different and worked. It looks to me like scanin wants a certain fixed minimum size for the target image.
> That assumes that the camera sensor is perfectly zero'd. This isn't always the case.
Sure, but that is beyond my control. But apart from that, my understanding is correct, or at least in accordance with yours? If so, I don't understand how the two following statements relate to each other:
> Note that the black point tag is simply annotation - it has no effect on the actual profile itself.
> It will have a weighting in the matrix fit of the sample points.
I imagine the latter translates to that the 0 black point lifts the shadows? (if it does anything at all).
-
I tried twice and and it did not change anything at all (except bkpt tag)... and how come (0,0,0) -> (0,0,0) shall change anything at all mathematically when you calculate a matrix ?
Yes you're right that for a pure matrix profile, 0,0,0 -> 0,0,0 won't change anything. So why add it ?
For any sort of profile that models an offset though, it will change things.
-
> If the image you included is that case, then I'd suspect that the 7 pieces of white tape have completely thrown the recognition off.
Yes but only because scanin placed the frames on them, outside the actual target.
It's setup to recognize rectangular patches, and bits of white tape look very much like high contrast rectangular patches. It doesn't have an ability to recognize general objects like frames (it's not A.I.), but will tend to ignore anything that doesn't look like rectangular patches.
With the target image downscaled to 1/2 instead of 1/4, the diag.tif looked different and worked. It looks to me like scanin wants a certain fixed minimum size for the target image.
It's size independent. Too low a resolution and it will fail (won't be able to detect edges, and/or not enough pixels in a patch), too high also risks not detecting edges and noise being detected as edges, but there is a very wide margin in between.
But apart from that, my understanding is correct, or at least in accordance with yours?
Adding such a patch fundamentally defeats the purpose of characterizing the device. If you already know/assume that zero maps to zero, why are you measuring it ? That's not to say it's always wrong, but be aware of what it implies.
If so, I don't understand how the two following statements relate to each other:
> Note that the black point tag is simply annotation - it has no effect on the actual profile itself.
> It will have a weighting in the matrix fit of the sample points.
If you edit the profile and change the black point tag, the way the profile says the device behaves doesn't change.
If you add a test point saying that perfect black measures as RGB=0 by the device, then this will have an effect on the cLUT/matrix/shaper model that colprof will create (except the special case of a matrix only profile, in which case there is no point in adding such a patch value).
I imagine the latter translates to that the 0 black point lifts the shadows? (if it does anything at all).
Its effect will depend on how the device actually measures or is extrapolated to measure perfect black. It may lift or reduce or make no change to the shadows.
-
Yes you're right that for a pure matrix profile, 0,0,0 -> 0,0,0 won't change anything. So why add it ?
that was not my idea - I just wondered as to why the topic starter, Hening B., did this for his pure matrix profile, he ref'd to something that Brian G, the author of Iridient raw converter told him and I was trying to find out why and what... I can imagine may be how this can be useful for LUT profiles for example, where naturally RGB(0,0,0) might be mapped to who knows where and you may be might add some virtual data to .ti3 to make sure that mapping is closer to what you want it to be in such areas, to bend that way the 2.x-3D LUT which otherwise might not be calculated properly - am I right ???... and I found apparently the source of that exercise = http://www.freelists.org/post/argyllcms/Camera-matrix-profile-adding-ti3-perfect-white-data-set
-
It may very well be that Brian gave his advice with regard to LUT profiles, and the ignorant transfer to a matrix-only profile goes on my account.