Raw histograms, how hard can it be?

[torger]

Some ranting coming up!

I'm just testing through my new H4D-50 back, and just as every camera I've tried out there the histogram is b*ll. How hard can it be for camera manufacturers to make a raw histogram which is actually useful to those of us which work with manual exposure?

That is one that doesn't change with white balance (because it shows raw data), either show all RGB channels or only the brighest one (not a luminance weighting which can be misleading ie not clip when one channel is clipped), and one that shows the top three-four stops rather than some random gamma-curve mapping so you can actually see how much an under-exposed frame needs to be adjusted.

But nooooo, it just seems too hard for manufacturers to grasp. The solution seems to be to wait until sensor technology has increased dynamic range so much that optimal exposure won't matter any longer.

The Hasselblad designers are really funny because they also have a different color space in the histogram than in the highlight blinkies. It seems to be some camera color space scaled for white balance in the histogram (not too bad, if it wasn't for the luminance weighting), and sRGB for the highlight blinkies (duh!). That is you may be without clipping in the histogram, but highlight blinkies active, and then you may or may not have raw clipping. No wonder that so few photographers understand how digital exposure work when the manufacturers does everything they can to make it confusing.

Fortunately it's quite easy to adapt to the quirks of histogram and blinkies on the Hasselblad. By looking at the preview render quality of the highlights under the blinkies one can figure out if there is raw clipping or not. If there's a smooth gradient - no raw clipping, if it looks blown out to white with a sharp edge - raw clipping. If there are no blinkies showing at all, the image is most likely not optimally exposed. For colored highlights the blinkies can start to show as much as two stops from raw clipping.

As said all cameras and backs I've tested have these type of issues (although the interesting mismatch of histogram and blinkies may be unique for Hasselblad). I haven't tried that many systems in person though, out of curiousity it would be interesting to know if there are some brand/model out there that actually show true raw exposure.

I think the real reason why manufacturers are so sloppy with the histograms is that they want to make a "user-friendly film-like workflow", ie the idea is that you should shoot like slide film, what looks like a good finished image on the LCD preview is the right exposure. With ETTR low contrast scenes look over-exposed with a standard film-curve, and with proper ETTR you don't leave any "highlight recovery" range. Fine. But it would still be nice to have an option to get raw histograms and blinkies for us that have no issues with processing ETTR files.

[araucaria]

I feel you.

The only raw histogram I'm aware of is on Canon cameras with the magic lantern hack.

[Paul Gessler]

You can call it what you want, but I think it comes down to laziness (in a somewhat good way) and not enough people asking for it to overcome that laziness.

What I mean when I say laziness in a somewhat good way: The camera's LCD is going to display a JPEG image regardless of the shooting mode of the camera. If RAW, it will display the embedded JPEG. If JPEG, it can simply display the JPEG as written to the card. Now imagine the software group responsible for the display code. For the histogram routine, would you rather use the common format (JPEG) to produce the histogram, or would you develop, maintain, and embed in-camera two separate methods to compute the histogram depending on the stored format of the file being displayed?

If you do choose the raw histogram method, you now need to send around double the amount of data over your interface in order to display any single image.

These hurdles are definitely surmountable, to be sure (as Magic Lantern, et al have shown), but not enough people (in the companies' minds) have asked for this feature. The companies' best interest is to focus on what the masses are asking for. While we LuLa members may be asking for it en masse, we're not really representative of the big companies' customer bases* at this point. For most customers, the JPEG histogram is adequate. (And many consumer DSLR owners don't even utilize that!)

*While the thread started with discussion of Hasselblad, which has a much more specialized customer base than CaNikon/etc., I think it will take a big player's addition of this feature to spur on adoption by others.

[torger]

Hasselblad seems to render the file live from the raw, because the preview image embedded in the file has no highlights in them, ie show white areas where the blinkies are. And indeed when you look at an image from the card for the first second or so you seen no highlights (ie just the embedded preview) and then they appear. The histogram is clearly not made from the preview (like in the CaNikon case) as it does not match. Although I cannot really be sure I would not be surprised that for Hasselblad they could provide the function through a firmware upgrade if they'd like. It's probably more difficult to fix the blinkies as they may actually be bound to the preview image (but that could possibly be rendered in software too...).

As said I think the problem is that the standard camera workflow is not designed for making optimal raw exposures. There's the fixed "film curve" and the cameras need to underexpose to get "film-like" highlight behavior in the raw converter. That's why we hear some talk about puzzling things like "highlight recovery" features of a back when it's actually only about varying degrees of under-exposure. Optimal raw exposures can sometimes look bad with the standard film-like processing... I think that "political issue" is far more difficult to overcome than the technical.

It would be nice though to have as an advanced setting. Although the back has ok DR it's not a Sony Exmor and when shooting landscape I often have an interest to cram the most out of the sensor... I will be able to do it anyway by knowing the quirks, but well, I just needed to complain a bit. It feels just soooo unnecessary to make it harder than it needed to be.

[deejjjaaaa]

and when shooting landscape I often have an interest to cram the most out of the sensor.

more often than not landscape shooting with MF means tripod and sufficiently slow process that allows you to bracket exposure, to spot meter - is it not ?

[torger]

more often than not landscape shooting with MF means tripod and sufficiently slow process that allows you to bracket exposure, to spot meter - is it not ?

I don't think many use spot meters these days, as you can fit quite many images to a CF card . It's has for me become a bit of a sport to guess exposure and try to get it right at first try. I do succeed from time to time. I often bracket one brighter shot too if I don't feel I'm sure I'm having an optimal exposure, which would not be needed it the histogram tools where there.

So yes, static photography is beneficial when it comes to working around quirky tools. I would still enjoy a better design.

In Hasselblad's case, even if they would not like to have a raw histogram because it would be just too good it's puzzling that they don't make the histogram and highlight blinkies match.

[tjv]

The histogram is one thing, but Anders, how is back performing? Looking forward to hearing your considered impressions of image quality.

[torger]

The histogram is one thing, but Anders, how is back performing? Looking forward to hearing your considered impressions of image quality.

I won't get a replacement Silvestri battery until the hollidays are over so I haven't shoot outdoor except for one test shot. I've started with some "lab" tests though and I'm writing a review, mostly to gather my own thoughts and see that I look at all aspects, I haven't decided yet if I will make the review public as if I do that I need to provide pictures too and then the amount of work goes up a lot. Now it's just a huge blob of text.

A few short things though - haven't tested fully yet but the "long" exposure (128 seconds officially on this, firmware limited at 150 seconds) looks quite good, the 128 looks much cleaner than the 30 seconds on the Aptus 75. Not that I was much limited by 30 seconds before, but on occasions I had to make underexposed 30 second shots so getting a quite clean 128 seconds will help from time to time. And it's really nice to not have a blackframe, ie as soon as you've shot your 100 second image or whatever you can shoot the next instead of having to wait 100 seconds for a black frame.

I love that I can change CF cards while running, seems robust concerning CF in all, which cannot be said about the Aptus.

Going from 33 to 50 megapixels is not that of a resolution increase, 50% area, but only 25% linear. However if you like me feel that 33 megapixels is on the limit of having to compromise on print density for your desired print sizes, adding those 25% on each side does make a real difference.

Focus check capability on the screen could be better, but I think it's usable. It's good enough to make me confident enough that I nailed the image.

Dynamic range is supposed to be the weak spot of this back due to the older Kodak sensor, it's not worse than my old Aptus though and that's okay by me.

Phocus conversion is recommended over Lightroom. It's okay in Lightroom, but many little things are better in Phocus, less aliasing artifacts, better hot pixel supression, quite well-balanced default noise reduction (although I would prefer to have it user-controlled), like the color better. Most will consider Phocus too limited to be the only post-processing tool though, it can do a high quality basic raw conversion, but that's it. LCC correction in Phocus is fine, but you don't get dust spot removal. As I'm a RawTherapee contributor I will use that, will probably make a few improvements there to raise the quality level. By some reason RT gets more aliasing issues with this back than with the Aptus, but it looks great in Phocus so it's an RT problem.

I think there's little reason to shoot at larger aperture than f/16 if we consider sharpness. With proper sharpening settings f/11 is only very marginally sharper, and as I'm an aliasing allergic I don't mind some diffraction onset. Concerning aliasing I hoped for a little bit less on this smaller pixel size, but there's almost no difference from the 33 megapixel back. With only 25% pixel pitch difference it's not too surprising though.

Hasselblad sometimes gets criticism for their build quality, and indeed the body has some plastic parts in some places which feels kind of cheap for a camera at this level, but the back feels solid with all metal including the bay door. There's no fan, the back does get a bit warm but I haven't seen any obvious negative impact on image noise from that.

So far I haven't seen anything that would make me truly disappointed with the back, so it looks good. The largest question now is, is really this relatively small resolution increase worth it to me, or should I stay with what I'm used to? Either way I have a back (or camera) to sell so it's equal work.

Now I have reverse engineered the file format to be able to use 3FR directly in RawTherapee, but it doesn't stop there as said it seems like I need to make some demosaicer adjustment too (if I don't get someone else in the team to do it :-) ), so I guess for me personally that's been the largest negative, that it didn't hook in right away in the workflow I'm used to. Although I love programming I already have more than I need. Likewise I can imagine that it can be a big change for someone leaving say Lighthroom or Capture One for Phocus.

If I had come from no back at all it had been a simpler choice. The main things I get compared to my Aptus is more resolution and a bit longer exposure, the rest is about the same. The most likely is that I will keep the back, but I have realized that my resolution hunger won't really be satisified even with 50 megapixels, or even 80... so future looks good for medium format

Oh, 49x37 is larger than 48x36. A tiny bit, but I kind of notice it on the wides. It didn't hurt that they get a tiny bit wider.

I need to do some field tests with the external battery to get the feel of that too. I think it will be alright, but I want to test before committing.

Edit: adding a few things. Of course it works very well with the Schneider wides with movements. This is the highest end for SK28 and SK35, and the compatiblity has advantages in longer lenses too but not as large, this when we compare to Dalsa-based 6um backs. There is no tiling, no microlens ripple, no crosstalk issues on any of the available tech lenses. The reason I have ended up with a H4D-50 is not becuase I think it's great with an external battery on a cable, but because 1) there's great deals to have on these cameras now and then (and I got one), 2) it has better screen for focus check than the CFV-50, but now when I've tried the H4D-50 I guess the CFV-50 screen is usable enough too, 3) tech wide+movements compatibility, 4) more resolution, more large-format like.

Upgrading in Aptus range was a dead end, only Aptus-II 10 seemed like a reasoanble upgrade but I don't like it's panoramic format and for my taste the wide angle compatibility was not as good as I wanted. My far fetched hope now is that I can stay with this 50 megapixel back until next CMOS generation comes and that is wide-angle friendly, and that Hasselblad again makes a good value offer like the current CFV-50c.

For anyone that can live with the current CMOS limitations, CFV-50c seems like a no-brainer, but again second hand H4D's and H3D's sometimes show up with low prices. To anyone already used to working without live view a CCD back will not be a problem. And if you agree with my taste that f/16 is an ideal shooting aperture concerning file quality, you can't really miss focus even if working on ground glass. I think f/11 is safe enough too on the ground glass, but f/16 is really fool proof

[tjv]

Great info, thanks!
I've long wanted a CFV-50 or H4D-50, and now thinking the 50c is a good option, so this kind of info is invaluable. I'm one of the crazy people who actually LIKE composing and focusing with a ground glass, so that doesn't dissuade me. The thing that does, however, is not having a usable 400ISO for when I'm in a pinch...

[torger]

The Hasselblad CCD backs are truly "ISO-less" by the way, the only thing that differs a higher ISO file from a lower is metadata tag that say it's a higher ISO and the preview image which is brighter according to the ISO, but the data is always recorded at ISO50. That is ISO400 is equivalent to ISO50 3 stops under-exposed. I haven't done any ISO testing but the results won't be comparable to a recent CMOS back or even Phase One sensor+.

The Phocus noise reduction is pretty effective so one can be tricked into beleiving that the sensor is better than it is in the dark shadows, but of course noise reduction can only clean up noise it can't fix color (but rather the opposite, that's the main reason I'm really careful to apply it).

If one learns how the misleading raw histogram and blinkies works one can expose the sensor effectively and then the DR is perfectly fine. In backlit scenes with a sky you will need a grad filter, just as with my old Aptus.

Another thing worth mentioning is the well-balanced sensor size. 49x37mm was once the high end MF sensor size, and the focal lengths 28 and 35 and 90mm image circles is clearly designed for that, it corresponds to the same FoV as 32 and 40mm on 645 fullframe (ie the Rodenstock 32 and 40). I'm not that into wides myself, the widest I have is the SK35 and I use it in one out of 20 shots or so, ie about as often as I use my SK180. I find beauty in a balanced system with all options open and that works in the full range though, I also find beauty and elegance in the small light distortion-free traditional symmetric "large format" lens designs so there's value for me to have a back that is designed to support that. It's the engineer side of me that likes that kind of thing, so to me it's not only about image quality.

The CFV-50c on the other hand is forcing a sensor to work outside it's intended purpose, but the brutal DR and fine tonal response makes it work well within limits anyway as we've seen. Getting a Rodenstock 32 to match a CFV-50c will give you great images, reasoable wide angle and shift range related to the sensor, but I think it's kind of a sad waste of glass. The SK Digitar wides was designed for the Kodak sensors, but you can still within limits use them with Dalsa 6um and some do. The Rodie retrofocus range was primarily designed for the full-frame Dalsa 6um sensors (P65+ and later), but you can still within limits use them with the Sony CMOS, and some do.

[yaya]

The SK Digitar wides was designed for the Kodak sensors

Not true. Most lenses in the Digitar line (except for 28XL, 43XL and 60XL) came out when the Philips-Dalsa 12µ and 9µ chips where as or more popular than the Kodak chips. As far as 6µ sensors go, in the field that are many more back with Dalsa than with Kodak.

[torger]

Not true. Most lenses in the Digitar line (except for 28XL, 43XL and 60XL) came out when the Philips-Dalsa 12µ and 9µ chips where as or more popular than the Kodak chips. As far as 6µ sensors go, in the field that are many more back with Dalsa than with Kodak.

Ok I simplified, let me rephrase; the Digitar line was designed assuming sensor properties that today are only available in the Kodak 6um, and older Dalsa. The 28, 43 and even the 60 push Dalsa 6um over its designed limit if you use the full image circle. That the field of views and image circle size just make so much sense with the 49x37mm I suppose was a stricke of luck perhaps, but I assume the 32/40 in Rodie range is made as a full-frame counterpart to 28/35.

I'm fully aware that there's today in the field more Dalsa than Kodak, I would guess that there's like 50 units of Dalsa 6um for each Kodak 6um on tech cams. The excellent tech cam compatibility of Kodak 6um has been a well kept secret due to Hasselblad's almost total lack of interest in tech cams. Before the CFV-50c there was CFV-50 which just as the CFV-50c had excellent price/performance. Hasselblad has had ~0 representation in the tech cam camp, it looks a little better now though. It's a bit too late for the Kodak though, the Rodenstock range serves the Dalsa 6um very well and also is a tad bit sharper so anyone striving for the highest end use Rodies and Dalsa, and preferably Phase One IQ line, and in MF the highest end gets the most attention. Hasselblad's tech cam big name Sean Conboy uses 6um Dalsa + Rodies via the H4D-60 by the way.

That there probably are more Dalsa 6um sensors running on the SK28 and SK35 than Kodak is not because the Dalsa is better suited for them, it's because of the weak Hasselblad representation in the tech cam marketplace. The SK28 is an extreme example, it's so poorly suited for Dalsa 6um that some dealers refuse to sell them, and with the Kodak 6um you shift it to the edge with no issues except for some vignetting. Either the Schneider lens designers did not have any clue at all about how sensors work when designing that lens, or they looked at the Kodak designs.

Anyway, a tiny step down from the highest end you find this excellent combination of 50 megapixel 49x37mm sensor with symmetrical wides. I'm not so sure that symmetrical wides are just legacy lenses waiting to die either, just like view cameras now gets a revival with CMOS live view, symmetrical wides can get a revival with back-illuminated and/or light-shielded CMOS sensors. That's what I'm hoping for in the longer term, and perhaps that was what the Schneider lens designers were hoping too. Or they were just clueless, hard to know which...

[tjv]

Most people I've met who use either the CFV-50 or HXD-50 shoot the back primarily a 100ISO. I'm not sure why, but I'm guessing it's to preserve highlights? Either way, I've seem some great samples used on a tech camera, all at 100ISO, that look stunning. I haven't heard anyone else mention the annoying histogram, but perhaps that's just because it's a relatively standard annoyance for cameras across the board?
Anyway, I'd love to see some samples when you've had time to shoot in the field.
Thanks again for the info!

[torger]

Most people I've met who use either the CFV-50 or HXD-50 shoot the back primarily a 100ISO. I'm not sure why, but I'm guessing it's to preserve highlights? Either way, I've seem some great samples used on a tech camera, all at 100ISO, that look stunning. I haven't heard anyone else mention the annoying histogram, but perhaps that's just because it's a relatively standard annoyance for cameras across the board?
Anyway, I'd love to see some samples when you've had time to shoot in the field.
Thanks again for the info!

Yes histograms is a standard annoyance across the board as far as I understand, and I think few have the capability to analyze how the histogram actually work like I can through my raw converter expertise and many tools to look into the actual file data and see what happens. So it's not a big deal, they're not much worse than anyone else in this respect.

I don't know why it's common to shoot at ISO100, I would suspect to get down the shutter speed because it won't make any difference in highlights, the raw data range in the file is always full range at ISO50 regardless of which ISO you shoot at.

I probably won't have the battery for a week or two so field shooting is quite far away . I'm currently doing some lab tests regarding noise and dynamic range, which is this sensor's/back's weak spot (considering the recent advancements through Sony CMOS, but it's also noticably worse than Dalsa 6um) and is a key thing to investigate for anyone interested in this, because here's the point where it's behind the competition.

Running in RT so I see the "truth" (ie no noise reduction applied) I can see that at 100% pixel peep it's a tiny bit noisier than my Aptus 75, but a tiny bit less noisy if averaging over all pixels. Noise is like on most CCDs well-behaved and random though, and if you run Phocus there's some fine-tuned noise reduction which make it competitive in terms of practical results. You can hand-tune noise reduction in other software too of course if you're into that thing. As said the danger with noise reduction is that you worsen tonality, so if you focus at how clean a shadow look you may miss that tonality has become worse, therefore I like to make comparisons without noise reduction enabled.

Anyway I consider the H4D-50 to be roughly equivalent to the Aptus 75 at pixel peep in terms of noise, and that means that it should be about 1.5-2 stop behind in terms of practical shadow push capability if we compare to Sony 50 megapixel sensors (eg the CFV-50c). This is for some the difference between using grads and not using grads. I'm a bit skeptical about pushing shadows hard on the Sony's though, as the freedom of noise is thanks to the electronics, there's still very low exposure (few photons) which means that there's not so much to get in terms of tonality. If you will suffer from this will depend on the subjects you shoot and your post-processing style. Lots of back-lit scenes and lots of shadow push in post and you will miss those 2 stops. I don't do that too often myself, and I don't mind grads that much so I can live with it.

The back has no fan and gets quite warm after being on for a while and reaching "working temperature", and I have noted that this affects noise. For short exposures it's negligible, but for a 120 second long exposure you lose about 1 stop. That is ISO50 at 120 seconds will look like ISO100 at a short shutter speed. If the back has been turned off for a while so it's not warm, and you start it and make a 120 second long exposure first thing it will have as little noise as a short exposure, the difference is hot pixels (which is normal for CCDs) but those are effectively cleaned up by the raw converter so we can disregard from them. And by the way, the back doesn't shoot black frames which is a great convenience. That is after a 120 second frame you can immediately shoot another rather than waiting for the back doing a 120 second black frame. I don't often shoot very long exposures, but 8 - 16 seconds happens quite often and not having to wait for a black frame is also in these cases quite nice.

In all long(ish) exposure performance is quite impressive in MF-land, it's not a P45+ of course, but no black frame, and relatively little difference in noise between short and long exposures is a big thing to me.

The Aptus 75 shoots black frames and still gets about 1.5 stop noise increase at its max 30 seconds. If I shoot the Hasselblad at 30 seconds when it has reached "working temperature" it's no more than say 0.2 stop noise increase or so, so there's definitely a win for me in the "normal" working range too. The quality loss at 30 seconds was a bigger issue for me than the 30 second limit itself with the Aptus.

For comparison it would be interesting to know how the Dalsa-based IQs and Credos do at their max limits, I don't know, perhaps they don't degrade almost anything at all to their official limit as the Hasselblad, or maybe they're more like my Aptus...

[tjv]

The files I've seen from the H4d-50, shot by a very well known architectural photographer in varied lighting conditions were very, very good. Certainly as good as, if not better than the files I shot under similar conditions when demoing a Leica S2 on several occasions. It might not be king of the pack when it comes to dynamic range, but the files had a look to them that was very pleasing for a person coming from shooting film and scanning with an Imacon 949.

I'd be interested to see who you get on with programming RT to make the most of the 'Blad 50mpx files. It is probably an exercise in futility, but have you thought about contacting Hasselblad directly and communicating to them – in their own language, no less – how you think the histogram should be displayed? If anyone can make a compelling technical case for it, it'd be you.

[torger]

I have contacted Hasselblad in Sweden for the histogram explaining how and why it should work in the way I want, also asked for the 3FR spec (separately) so I wouldn't need to reverse engineer. I do these kind of things all the time :-), provided feedback to Linhof on details on the Techno camera that I think should be improved etc. From Linhof I actually got good replies, but Hasselblad has been silent. I've heard though that one disadvantage of Hassy is that their customer relations is not that good so I didn't expect much. On the other hand, detailed opinions and suggestions about technical things is however hard to get through with in any company, my experience is that only in really small companies or if you have a contact further into the company you can reach through.

Any medium format back used in its sweet spot will produce great results. The purpose of my lab tests is to figure out in what conditions the camera works well and what's tougher for it to do, and then think if it's strong points suits my shooting style well and its weak points hurts it or not. For example 30 sec exposures with my Aptus 75 is stretching it outside its sweet spot, while the Hassy H4D-50 don't break sweat, so if my intention was to bury the Aptus I would do longer exposures. If I had a CFV-50c and the intention was to bury the H4D-50 I'd shoot crazy backlit scenes without grads and push the shadow several stops, or if I'd like the H4D-50 to win I'd put on a SK28 and shift 15mm.

Concerning dynamic range MF backs had good range already in ~2004, then there's been only very small improvements in later sensors, the Kodak KAF-50100 which is from 2008 if I remember correctly is not much better than the older. The next noticable improvement came with Dalsa 6um sensors, first in P65+, and then the latest further step came with the Sony CMOS. Image quality is not only about dynamic range though. As said it is good, it's just that there's tech nowadays that's better, and just as one might desire more megapixels because there's more out there one might desire more DR too. MF systems is often evaluated based on skin tones in studio conditions, that's however not my expertise or need so I won't look at that aspect, but I'll see if I can figure out some other color test.

When I'm ready with the tests I'll lift it out in a separate thread, it's a bit strange to have all H4D-50 testing in this thread which started as a histogram rant  :-)

[Rory]

Upcoming Nikon firmware update for D800, D600, D610:

New RAW Histogram. This feature displays impressive full-screen histograms for all 3 color channels (red, green and blue) simultaneously, based on data directly from the image sensor. The new feature allows advanced photographers to fine-tune perfect exposure for each shot.

Read more on NikonRumors.com: http://nikonrumors.com/#ixzz3OHg4XJqw

Cheers
Rory

[tjv]

Wow, that's a great update! It'd made critical exposure sooooo much easier!

[torger]

I believe it when I see it . Sounds like a fantastic update, if Nikon does it it will be easier to convince others to follow. In a way it's logical that it would come to the D800 series of cameras first, I bet there's a lot of landscape photographers using that and it's primarily that genre that needs this feature.

Hassy's user interface seems very much software based, so adding a raw histogram to it should be a piece-of-cake(tm) for them, but I guess they haven't had many requests for it, so far at least.

[BernardLanguillier]

I believe it when I see it . Sounds like a fantastic update, if Nikon does it it will be easier to convince others to follow. In a way it's logical that it would come to the D800 series of cameras first, I bet there's a lot of landscape photographers using that and it's primarily that genre that needs this feature.

It would mean that Nikon is starting to really understand the way people want to use their cameras and to believe in the potential of those cameras as extremely refined tools adressing advanced users needs.

Many of the improvements made to the D810 were already hinting at the fact that they had finally understood the needs of stitchers, but this would be another level.

That alone would be a revolution.

Cheers,
Bernard