A Workflow with Beta RGB

[digitaldog]

AFAIK, all a SO is, is another copy of the full sized raw embedded into some Photoshop document. Whatever you can do with the raw as a SO, you can do without that raw being there. You can always update the instructions, moving from PV to PV, it's the metadata that counts. Seeing the raw along with other Photoshop components unique to PS is neat and useful I suspect (I admit not using them, all my raw work is done in LR). But short of that, it's a much bigger document and unless I'm missing something else, whatever flexibility you have to alter from raw, it's the same using either process.

[Jim Kasson]

OK, so this is effectively a new CMM which incorporates the gamut-mapping strategy, which is surely a better way to go if you want a good Perceptual mapping (algorithm and performance issues aside).  How do you see it fitting in with the current ICC model (the only way I can see it myself is that the new CMM would override the Perceptual mapping in existing profiles).

When I left the color management biz, ended my six-year vacation in research (IBM big-R Research, where, as we used to say, where the rubber meets the sky) to return to product development (I finished out my career as VP Engineering at Echelon), the ICC existed, and I was working with them although IBM never joined (some intellectual property issue, as I remember). When I left IBM in 1995, the ICC was only 2 years old. Therefore, my knowledge of everything that's happened since is as a user of the technology, not a developer or researcher. So what I'm about to say is suspect. But I'm going to say it anyway; just take it with a grain of salt. Andrew knows more than I do about the current inner workings of color-managed software, so maybe he'll chime in if I err.

A color rendering engine (ICC terminology: CMM), like the one in Adobe Ps, has as its inputs 1) the source image data, 2) the ICC profile of the source space, 3) the rendering intent, 4) the ICC profile of the output space, 5) other user inputs particular to the rendering engine, such as what to do about black and/or white point mismatches. There are ICC-suggested ways to use these inputs, but the engine is not constrained to follow them. In fact, the ICC says that people developing profiles and software are free to use whatever color mapping algorithms please them for the perceptual and saturation intents. Although the ICC profiles assume that rendering is a point process, and it would be a distortion of their intent for implementers to not adhere to that for the absolute colorimetric and relative colorimetric intents, it doesn't force all mappings to be point processes. For the perceptual and saturation intents, the CMM is allowed to use the input and output profiles in any way it chooses to come up with the mapping from input to output. In fact, the ICC, without explicit definition of the operations, calls CMMs that perform various optimizations outside the scope of the ICC specifications "smart" CMMs. The ICC also explicitly says that the same vendor could offer different perceptual rendering intents that provide different "looks", so having many different ways to do perceptual rendering isn't verboten.

So, all of that is a long-winded way of saying, yes, you're right.

Jim

[Jim Kasson]

The idea behind the 2-step mapping is that the first step is a Relative Colorimetric one from the large working space to the smaller one (of course there is a need to watch out for out-of-gamut issues when doing this); followed by the 2nd step which a Perceptual mapping to the destination.  What this should do is to reduce the desaturating effect of the Perceptual mapping from the original (large) workspace.

Sounds to me like you don't like some perceptual mapping implementation and you're fooling it by coming up with a two-step approach to get what you want. I think this would be hit-or-miss. It might do what you want for some images, and something you don't want for others.

Jim

[Jim Kasson]

I didn't realize you had actually implemented the algorithm Jim ... the final one in the patent I guess?  Very interesting ... I hope you do get to the time to re-code it!  If you do I'll be happy to do some testing .

I implemented many versions in a mixed programming environment with some calculations done in Smalltalk (easy to use, but slow), and the real image crunching done in an IBM-internal image processing language. I don't have any of the code (it belongs to IBM, not me), so I'd have to start over. There's a problem with a Matlab implementation: you need to buy a $5K license from them to distribute an executable. Without that, anybody using the code would have to buy a Matlab user license, including the Image processing toolkit. Not very practical.

I wonder if there's a way to do something similar as a Photoshop action? I'll think on it.

Jim

[Tim Lookingbill]

In fact, the ICC, without explicit definition of the operations, calls CMMs that perform various optimizations outside the scope of the ICC specifications "smart" CMMs. The ICC also explicitly says that the same vendor could offer different perceptual rendering intents that provide different "looks", so having many different ways to do perceptual rendering isn't verboten.

Jim

That explains a lot about the behavior of the rendering intents of the Fuji Frontier drylab profile created with ArgyllCMS by Fuji staff I downloaded off Fuji's European site just to see if the RI's made a big difference. 10 years ago when these discussions were still going on I was using DryCreekPhoto profiles for local RA-4 process Noritsu wetlabs lazer exposed silver halide printer that reproduced most DSLR colors adequately but would screw up other colors that were beyond its gamut I had to assume.

None of the rendering intents from those DryCreek profiles made a dent in fixing OOG. Fast forward 10 years later to the newer Fuji Frontier inkjet/drylab printer and the AgryllCMS RI's weren't needed because (I'm assuming) this printer's expanded gamut could handle a lot more colors my DSLR/Raw processing could throw at it.

Another issue I noticed about these print experiments is I had to make far bigger changes to the edits to see a difference over what the RI's would provide. I printed several copies of the same file converted using different rendering intents and couldn't tell any of them apart.

[TylerB]

my apologies for jumping in too quickly, based on faulty memory... some clarifications

Referencing what I had to do with the orange flower edit demo posted earlier in the thread, what do you mean by "fit" the gamut of drum scanned color into BetaRGB that wouldn't fit in AdobeRGB and what did that involve? How did it manifest to tell you that was the problem? Do you mean you had to edit color detail so it didn't turn into blobs of posterized color as I had to do in the orange flower? Or was all this "not fitting" manifested in the print?

badly stated.. I did not "fit" the gamut, I "fit" colors in the image that, due to film and scanner gamut would not fit in the working space, would be clipped after conversion. I never made these decisions based on eventual output gamut as that changes with device. This was for "master" files. So it's just the usual tools as I'm sure you understand, hue/sat, selective color, replace color, etc.. while still in the input space. OOG warning, using the working space in preview, was rarely as informative as necessary, so it tended to be trial and error. Editing in input spaces is not ideal, but if limited to the few image colors outside the eventual conversion, it works, and what other way is there to avoid clipped color? My understanding is that evaluating input gamut is tricky at best, but the normal tools like colorthink show a Howtek profile from a Hutch target as much larger gamut than most of the usual working spaces, including BetaRGB, even ProPhoto in very minor places. Whether or not this is a problem is image dependant. I'm sure all this translates over into captures as well, although you will find other threads here about the difficulties of the concept of gamut as it applies to capture.

Not D Cam4 or D Cam5. This profile (DCam 5) makes ProPhotoRGB looks small and it is the only profile I have seen that ecompasses all visible colors (not even LAB)

My info was incomplete, based on one 3Dgamut map provided by Holmes, I had forgotten it was based on DCam3, which could not contain the gamut of even a 9800. I can not test the others without purchase if I recall. 2D comparisons are inadequate, 3D is required. A 2D mapping shows BetaRGB as containing the gamut of my 9900/HarmonGlossBaryta profile, while a 3D mapping reveals cyans that BetaRGB can not contain. I'm sure your info about the larger spaces is correct, but 3D mappings would be interesting to see. Additionally, spaces that large put us back to the problem of less than ideal perceptual conversions to output, and I still think the ideal print centric working space remains illusive. PhotoGamut was an interesting idea, but too small right out of the gate. Image referenced smart CMMs, as Jim referenced, was an interesting idea, I don't think anything really came of it.
There are proponents of only colormetric conversions to output space, massaging only those colors that would be out of printer gamut into place pre-printing, there's something to be said for that. Hope I haven't gone to far OT.

[Robert Ardill]

AFAIK, all a SO is, is another copy of the full sized raw embedded into some Photoshop document. Whatever you can do with the raw as a SO, you can do without that raw being there. You can always update the instructions, moving from PV to PV, it's the metadata that counts. Seeing the raw along with other Photoshop components unique to PS is neat and useful I suspect (I admit not using them, all my raw work is done in LR). But short of that, it's a much bigger document and unless I'm missing something else, whatever flexibility you have to alter from raw, it's the same using either process.

I'm not sure if I quite understand you Rodney.

My concern is simply this: will I be able to go back to Photoshop CS6 to process an image that contains a raw smart object that has been created in some future version of Photoshop CC / ACR?  

(The reason for my concern, obviously, is that I don't want to have no way out of the CC pay-as-you-go model.  With Lightroom it's not a problem right now because I do have a perpetual license (as I do with Photoshop CS6) and as long as Adobe allows upgrades I'm happy.  The day that Lightroom becomes pay-as-you-go only is the day that I will be thinking seriously about getting out.)

I think backward compatibility with Photoshop CS6 is OK in most areas, but smart objects/smart filters in general are potential trouble.  For example, if the smart filter is a new filter, clearly it won't function in CS6.  Of course I could always rasterize the smart objects, but that would mean going through all of my tiffs and psds before unsubscribing to CC.  Similarly, I think that if the smart object uses a process version > PV2012 that it won't function correctly in CS6.

So if this is true, then I would make sure that I do not have smart objects in my files that use post CS6 filters or raw smart objects that use process version > PV2012.

I'm just checking with you guys to see if I've got this right, or if there are other gotchas that I haven't thought of. Or is there a workaround I haven't thought of (for example, some clever way to edit the raw file in the smart object using Lightroom)? Or some information about how Adobe are going to handle this sort of backward-compatibility issue that I'm not aware of (for example by releasing versions of ACR that will run with CS6 to handle post PV2012 process versions)?

Robert

[Tim Lookingbill]

OOG warning, using the working space in preview, was rarely as informative as necessary, so it tended to be trial and error.

Tyler,

What did you go by that forced this trial and error editing session? Was it color detail distortion in the preview? or OOG warning from some analytic scanner GUI?

It is about the preview. It has always been about the preview. Did the preview generated in the scanner interface during capture stay the same opening the final scan in Photoshop and assigning the source scanner profile if it wasn't already tagged by the scanner software?

Without it manifesting in the preview on the display there is NO PROOF that OOG is going on or whether it is degrading the image preview. You're in theory land and I'm trying to get you tell us whether you were relying on theory from OOG analytics or the preview.

[TylerB]

Tyler,

What did you go by that forced this trial and error editing session? Was it color detail distortion in the preview? or OOG warning from some analytic scanner GUI?

It is about the preview. It has always been about the preview. Did the preview generated in the scanner interface during capture stay the same opening the final scan in Photoshop and assigning the source scanner profile if it wasn't already tagged by the scanner software?

Without it manifesting in the preview on the display there is NO PROOF that OOG is going on or whether it is degrading the image preview. You're in theory land and I'm trying to get you tell us whether you were relying on theory from OOG analytics or the preview.

Nope no theory. Sorry I was not clear, simple conversion from input space to working space.. are any color channels clipped? If so.. undo and work those colors. I never let the scanner software convert, for this reason, it only assigned the scanner profile, conversion later in PS, per above

[Robert Ardill]

Sounds to me like you don't like some perceptual mapping implementation and you're fooling it by coming up with a two-step approach to get what you want. I think this would be hit-or-miss. It might do what you want for some images, and something you don't want for others.

Jim

Hi Jim,

In general I use relative rather than perceptual, unless the clipping is too severe (I prefer the print to be as close to the image as possible).  But sometimes that's not possible because the image colors are too far out of gamut.  Trying to punch these pesky pixels into port can do more harm than good (and I completely agree with Andrew on this), but with some images the perceptual mapping can result in the more saturated colors getting desaturated (which is presumably one of the reasons for your invention).

If that happens I (currently) have four options:
1. Do a perceptual conversion to the destination profile and carefully and selectively fix the problem from there.
2. Do a 2-step relative/perceptual mapping. The intermediate working space needs to be big enough so that the clipping is not a problem and yet small enough so that the perceptual mapping works well.
3. Make an image-specific profile using Argyll with the image gamut as the source gamut.  This may result (should result) in an improvement over the straight perceptual mapping.
4. Use a different profile to see if that works better, or try to optimize the profile to improve the saturation (for example by tweaking the perceptual intent custom sliders in i1 Profiler).

In all cases I'm trying to fit a triangle into a smaller blob, of course, so any solution is going to be a compromise.  So it's not a question of fooling anything, and I don't have a particular bias towards any one of these techniques: it's just a question of trying to understand the options available and working as best as I can within the constraints of the system.  

Robert

[Tim Lookingbill]

Nope no theory. Sorry I was not clear, simple conversion from input space to working space.. are any color channels clipped? If so.. undo and work those colors. I never let the scanner software convert, for this reason, it only assigned the scanner profile, conversion later in PS, per above

How was the preview affected by the clipped data?

My orange flower demo is totally clipped in sRGB and wasn't when I edited the Raw file to look as it should in ProPhotoRGB. All the detail in the preview was retained in sRGB and printed just fine to a printer whose 3D gamut model says it can't reproduce sRGB's 255 red channel.

Yes, it's just theory until you see it affect the preview.

[digitaldog]

My concern is simply this: will I be able to go back to Photoshop CS6 to process an image that contains a raw smart object that has been created in some future version of Photoshop CC / ACR?

The raw Smart Object is just a copy of the raw so yes you can get access to that raw (you have another floating around too). How any or all of this interacts with Photohsop elements you've created that are newer and may not be understood by earlier versions of Photoshop is a different possibility. But the raw in the SO is just that, a raw which is read only.

[TylerB]

How was the preview affected by the clipped data?

My orange flower demo is totally clipped in sRGB and wasn't when I edited the Raw file to look as it should in ProPhotoRGB. All the detail in the preview was retained in sRGB and printed just fine to a printer whose 3D gamut model says it can't reproduce sRGB's 255 red channel.

Yes, it's just theory until you see it affect the preview.

I guess we value and utilize previewing differently. In the particular case I am describing, very early in the workflow, it's all about the histogram.

[Robert Ardill]

No my images It's far too small. Even output to CMYK (SWOP V2), sRGB clips colors. Did you see my video on gamut? It's OK for web because of how it's based. It's not ideal for print, there are no sRGB printers. If you are concerned with gamut clipping, sRGB isn't the space you'd ever consider using for anything but posting to the internet. And someday, should wide gamut displays be the norm, we'll point towards Adobe RGB (1998) instead of sRGB.

Hi Andrew,

I've had a look at your excellent video on gamut.  You have a real talent for explaining complex things in a way that is very understandable and interesting!

I've had another look at sRGB v print profiles (with good gamuts and black point) and it is clear that there is a risk of clipping of the saturated dark colors (in the case of Canson PhotoHiGloss in the greens/blues), even in an image that appears quite unsaturated.  The same is true but to a lesser extent with Adobe RGB.  Beta RGB is fine as is ProPhoto, of course.

So I regretfully concede that sRGB is not the best working space for print (not that I ever suggested that it was, at least not globally, for all images irrespective of gamut and saturation).  Of course it's fine for black and white prints and for images that really do have very low saturation throughout.

Robert

[Tim Lookingbill]

I guess we value and utilize previewing differently. In the particular case I am describing, very early in the workflow, it's all about the histogram.

Then I take it the preview is not affected by clipping in the histogram. Thanks for confirming.

[Robert Ardill]

The raw Smart Object is just a copy of the raw so yes you can get access to that raw (you have another floating around too). How any or all of this interacts with Photohsop elements you've created that are newer and may not be understood by earlier versions of Photoshop is a different possibility. But the raw in the SO is just that, a raw which is read only.

Actually, Andrew, I don't think you're right. Just try this if you can: create an image with a raw smart object that has been developed to PV2012.  Save the image. Open it in Photoshop CS4.  Open and close the raw smart object.  What do you see?  Nothing should have happened, right?  Wrong. The raw smart object has the development settings from PV2012 and when you open it in CS4 this gets changed to PV2003 and the development is kaput.  The whole purpose of having the raw smart object is obliterated (well, almost).  Sure, the raw image is still there - but the development settings it had are gone.

This is more of an issue than you think, IMO.

Robert

[digitaldog]

The SO/ raw wasn't changed, some metadata about it might be. The equivalent of the sidecare, embedded I suspect in the PS document. If you had a raw and some metadata that said you wanted PV 2012, then you took both back to ACR or LR that was older than that PV, it wouldn't show it of course, it would "refert" to PV2010 or PV2003. I don't think that's different in Photoshop expect one would suspect the version of ACR understands the most modern features. But going back, it's the instructions that are not recognized. I don't see how the SO is different but again, I'm an novice in their use.

[Jim Kasson]

I guess we value and utilize previewing differently. In the particular case I am describing, very early in the workflow, it's all about the histogram.

Do you know the joke about the drunk looking for his keys? If you don't I'll be happy to tell it. If you do, when it comes to controlling image gamut when editing in an RGB color space, the histogram is the lamppost.

The histogram allows you to make sure that you're not exceeding the gamut of the editing color space, but is useless as a measure of how you're doing wrt gamut of any output space that happens to be different. If you're editing in PPRGB, the output space is guaranteed to be different. The only reason it seems to work fairly well is that, as Andrew has already pointed out in this thread, in good editing color spaces R=G=B=x defines the neutral axis. Couple that with the property of RGB color spaces that the chromaticity gamut is small near the white point, and smell near the black point, and you can see that the chromaticity the top and the bottom of fully-populated histograms doesn't have much room to move around.

Or does it? There's nothing to say that the points in the image that give you the top part of the red histogram channel are the same points that give you the top part of the blue histogram channel. We're just lucky that it happens to work out that way for most images. Flowers, not so much.

I think we need other tools floating over the image like the histogram window, tools that take samples of the image like the histogram does, but displays those samples in some device-independent color space in cylindrical coordinates with a luminance axis, a chroma radius, and a hue angle. The user could then superimpose her choice of output device gamuts in the same display. As the user edited the image the display would keep up in real time, like the histogram (sort of) does.

Or maybe there's another way to present the information. I'm open to suggestions. As has been pointed out in this thread, gamut alarms are limited in that they don't tell you how far out you are. Maybe there's a way to modify them --false color? -- so they do.

Jim

[TylerB]

Then I take it the preview is not affected by clipping in the histogram. Thanks for confirming.

Sorry Tim, I honestly don't know what you are getting at. I'd not assume it is or isn't, I'm not "confirming" anything.. I'm not paying much attention to it yet. My primary task in the very beginning is to have ALL the info. What is done with it from there on out can be subjective and previewing becomes increasingly important. But initially, I have to have a file with integrity, no clipped channels, among other things. With regard to scanning, a color managed workflow, which ONLY applies to transparencies, depends on sound practice, good profiles, and adequate working spaces and/or conversion process to it. Now I have, in PS, the most accurate digital representation of the film I can muster, have thrown no info away, and am in a suitable perceptually uniform editing and storing space. From there on out it's artistic or corrective decisions and previewing comes more into play. Obviously little of this is relevant these days, input color management is an evolving animal with regard to capture.

I think we've strayed OT, my initial post was regarding BetaRGB, it's relevance in light of increasing output device performance, and the general point of useful working spaces, why aRGB fell sometimes fell short in that workflow. Is there something you are getting at I'm not understanding?

[Tim Lookingbill]

Sorry Tim, I honestly don't know what you are getting at. I'd not assume it is or isn't, I'm not "confirming" anything.. I'm not paying much attention to it yet. My primary task in the very beginning is to have ALL the info. What is done with it from there on out can be subjective and previewing becomes increasingly important. But initially, I have to have a file with integrity, no clipped channels, among other things. With regard to scanning, a color managed workflow, which ONLY applies to transparencies, depends on sound practice, good profiles, and adequate working spaces and/or conversion process to it. Now I have, in PS, the most accurate digital representation of the film I can muster, have thrown no info away, and am in a suitable perceptually uniform editing and storing space. From there on out it's artistic or corrective decisions and previewing comes more into play. Obviously little of this is relevant these days, input color management is an evolving animal with regard to capture.

I think we've strayed OT, my initial post was regarding BetaRGB, it's relevance in light of increasing output device performance, and the general point of useful working spaces, why aRGB fell sometimes fell short in that workflow. Is there something you are getting at I'm not understanding?

The points made by everyone in this thread are all relevant to the subject of working spaces as gamut containers especially whether there is or isn't a connection to how it relates to quality previews. Folks don't pay for histograms, they pay for a quality preview of the image. We're trying to connect functionality to the working space concept.

I understand your requirements to capture all data according to a histogram. I've read in some Adobe forum discussions in the past some stock agencies not accepting images with clipped data even in an 8 bit sRGB jpeg. You're workflow describes best practices established by a histogram at the scan stage. Understandable. But I was trying to pin you down on you're first statement that suggested you were manipulating the source scan data to deliver a better histogram with a disregard to what it did to the preview. That's why I was asking about the quality of the preview. Did the preview distort or change going strictly by histogram analytics?

I've noticed the same concerns editing an audio waveform. Nobody wants clipping because they think it creates distortion in the sound, but there are folks doing this and reporting no change in sound quality, just loudness. In fact the aiff file for Propellerheads' "On Her Majesty's Secret Service" viewed in Audacity is a brick wall of clipped data but I swear I've never heard such fuller, louder dynamics in a piece of pop music than this selection that didn't destroy my speakers with tons of audible artifacts. It's bass heavy down to about 30hz, loud and filled with noise from the sampled original '60's movie theme. Listening on headphones is pretty scratchy sounding but in my car it's kicks ass loud and clean with no audible artifacts especially in the bass which is HEAVILY clipped.