Pages: [1]   Go Down

Author Topic: The difference between 3FR and FFF format  (Read 10075 times)

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
The difference between 3FR and FFF format
« on: December 19, 2014, 06:05:33 am »

I wondered what the difference between the Hasselblad 3FR format and FFF format is. Most of my test files has been the FFF format, and I know this is what you get from Phocus when shooting tethered or importing a 3FR file, while the 3FR format is what the back writes natively to the CF card.

Both files are formatted the same, so what's the difference? I've tried to find an answer before but not been successful. After getting bad results with my own H4D-50 3FR in dcraw-based raw converters I looked deeper into the format, and the answer is quite simple - the 3FR format contains the raw data with the analog calibration data on the side (just like Phase One's IIQ format), while the FFF is the same but with calibration applied and removed. Dcraw don't apply any calibration data for 3FR files, so you get the uncalibrated result which may lead to poor results. If you convert to DNG the calibration data is applied of course (the DNG format does not support calibration data).

I've done some basic comparisons between a few raw converters and I must say I'm impressed with Phocus. It's not so feature rich in terms of post-processing, but the conversion is great. As most commercial raw converters they cook the file a bit without telling the user, which I generally don't like, but I must say that the automatic noise reduction made in the dark shadows is the best I've seen, they've really maximized what my Kodak CCD can do and the DR looks like it's more than it actually is. The result is better than in Lightroom, I think the demosaicing and texture is better too, so if I had to choose between the two I'd use Phocus. I haven't analyzed color yet as I haven't had suitable shooting conditions for that yet.

However, as an RT contributor I'm eager to reverse-engineer the calibration data and make it render great in RawTherapee :-)
« Last Edit: December 19, 2014, 06:09:35 am by torger »
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: The difference between 3FR and FFF format
« Reply #1 on: December 19, 2014, 08:40:34 am »

Hi Torger,

Glad to hear that you are happy with that new MFD of yours. Those engineering efforts of yours are much appreciated.

Best regards
Erik
Logged
Erik Kaffehr
 

Theodoros

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2454
Re: The difference between 3FR and FFF format
« Reply #2 on: December 19, 2014, 09:39:26 am »

I wondered what the difference between the Hasselblad 3FR format and FFF format is. Most of my test files has been the FFF format, and I know this is what you get from Phocus when shooting tethered or importing a 3FR file, while the 3FR format is what the back writes natively to the CF card.

Both files are formatted the same, so what's the difference? I've tried to find an answer before but not been successful. After getting bad results with my own H4D-50 3FR in dcraw-based raw converters I looked deeper into the format, and the answer is quite simple - the 3FR format contains the raw data with the analog calibration data on the side (just like Phase One's IIQ format), while the FFF is the same but with calibration applied and removed. Dcraw don't apply any calibration data for 3FR files, so you get the uncalibrated result which may lead to poor results. If you convert to DNG the calibration data is applied of course (the DNG format does not support calibration data).

I've done some basic comparisons between a few raw converters and I must say I'm impressed with Phocus. It's not so feature rich in terms of post-processing, but the conversion is great. As most commercial raw converters they cook the file a bit without telling the user, which I generally don't like, but I must say that the automatic noise reduction made in the dark shadows is the best I've seen, they've really maximized what my Kodak CCD can do and the DR looks like it's more than it actually is. The result is better than in Lightroom, I think the demosaicing and texture is better too, so if I had to choose between the two I'd use Phocus. I haven't analyzed color yet as I haven't had suitable shooting conditions for that yet.

However, as an RT contributor I'm eager to reverse-engineer the calibration data and make it render great in RawTherapee :-)

I don't think there's a difference... IMO it's just re-naming. As far as the quality of Phocus (and Flexcolor) as RAW converters, not only I agree with you, but I created a thread on it some weeks ago... http://www.luminous-landscape.com/forum/index.php?topic=95481.0 . The DR improvement is not "more than there is" as you say... simply other converters (like LR or CR) are unable to present the captured magnitude of it. The same also happens with Capture one.
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: The difference between 3FR and FFF format
« Reply #3 on: December 19, 2014, 10:10:20 am »

Hi Keith,

I feel that ACR/LR has some homework to do on MFD-raws. My guess is that the pipeline is optimised for OLP-filtered images, now that Nikon and Sony seems to drop the OLP-filter, I would believe that Adobe will rework the demosaic routines so they work better with non OLP-filtered images.

Best regards
Erik


After six years of using Hasselblad I'm still unsure what transformation and differences there are between the 3FR file on the card and the FFF files once in Phocus.

I do know that my files produced using Phocus are superior to those produced by ACR. Perhaps unsurprising given that Phocus is dedicated software.

Enjoy your camera.
Logged
Erik Kaffehr
 

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: The difference between 3FR and FFF format
« Reply #4 on: December 19, 2014, 02:50:08 pm »

Keith,

Only LR/ACR has this behaviour of the ones I tried, that is LR, Capture One, Raw Therapee, AccuRaw, Iridient Raw Developer.

I hope that fix it, as more and more cameras drop the OLP filter.

The images blow show this quite well, they are not sharpened an upsized to 200%.

http://echophoto.dnsalias.net/ekr/Articles/RawConverters/Mosaic/Aliasing_NS_200_percent.jpg

Best regards
Erik

Hi Erik

It must be a lot less complicated to hone a dedicated raw convertor to a given line of cameras rather than to a hotchpotch of cameras from differing manufacturers.
Logged
Erik Kaffehr
 

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: The difference between 3FR and FFF format
« Reply #5 on: December 19, 2014, 02:56:31 pm »

As I wrote in the initial post there is a difference between the two formats, 3FR = uncalibrated raw data with calibration data in tags on the side, and FFF = calibration applied and calibration tags removed. A converter that parses the format gets to the same result of course so from a user perspective there is no difference. However it's relevant to us that write raw conversion software. Also worth noting is that many smaller third party apps have its raw conversion based on dcraw, which does not apply calibration data as the format of it is not reverse-engineered. A while ago I reverse-engineered that for the IIQ format (now available in latest dcraw), and obviouslyt I have more work to do now with 3FR. I don't like to store my images in proprietary formats, so the least thing I want is to have the format reverse-engineered and covered in open-source software.

MFD CCDs are harder to convert than newer CMOS, as the former is much more "analog", there's slight non-linearity in the AD-conversion (hence calibration curves), and there's more read noise which can disturb demosaicers and create false texture.

Even the calibrated FFF file for my H4D-50 has slight difference between green channels, which put some demosaicers off target and they start producing mazing artifacts. This can be compensated for with a threshold though ("green equiliberation" in RawTherapee where this is a user setting), so its quite easy but still an extra thing to think about.

I do not agree with the view that it's about to "extract" the DR they capture, you can't really do magic there in signal terms, the raw data is what it is, but you can do "psychovisual magic" ie, make a noise reduction which is pleasing to the eye, and that's what Phocus does very successfully. Some demosaicers worsen the noise and make it look more blotchy, so the best technique is probably to have a demosaicer focused on detail in the bright areas and one focused less on detail and more on smoothness in the noiser shadows, I suspect Phocus has such a technique rather than using the same demosiacer over the whole image.

Concerning demosaicers specially made for AA-filter-less cameras I don't know if there needs to be that much of a difference, I think Lightroom makes a quite good work in the bright areas, and all have aliasing problems. Maybe Phocus has a little bit less, have looked more at texture so far. Possibly texture is a bit overdone in LR, and that might be that a demosaicer designed for AA-filtered cameras is a bit more sensitive to catch texture, but that's just a speculation.

I haven't had time to make comparisons with my Aptus yet so I don't know what I will think of the absolute image quality when comparing side by side. I hope to be able to do that quite soon, but this calibration data problem will delay me, seems to be more coding than shooting for me the coming time...
« Last Edit: December 19, 2014, 03:09:21 pm by torger »
Logged

EricWHiss

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2639
    • Rolleiflex USA
Re: The difference between 3FR and FFF format
« Reply #6 on: December 20, 2014, 09:00:41 am »

Toger,
Thanks for the information.
Eric
Logged
Rolleiflex USA

BobShaw

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2218
    • Aspiration Images
Re: The difference between 3FR and FFF format
« Reply #7 on: December 21, 2014, 05:10:06 am »

3FR is just the compressed format that Hassy uses to save some space on the card. When you shoot tethered or import them to Phocus they expand to FFF. You don't need the 3FR once you have converted them to FFF.
Logged
Website - http://AspirationImages.com
Studio and Commercial Photography

orc73

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 318
Re: The difference between 3FR and FFF format
« Reply #8 on: December 21, 2014, 12:55:33 pm »

...just to complete the raw format discussion:
so DNG export from Phocus would be identical to FFF ?
The formats would be read in LR just the same? (including 3fr)

I love love love the colors and rendering of Phocus, the use is kind of a pain though and I export the tiffs unchanged to Lightroom.

If somebody has a profile to get a similar rendering in Lightroom, I would pay for it. Might start a thread on that.
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: The difference between 3FR and FFF format
« Reply #9 on: December 21, 2014, 02:25:07 pm »

Hi,

I did some work on the P45+ I have a year ago: http://echophoto.dnsalias.net/ekr/index.php/photoarticles/79-p45-colour-rendition

I have generated a dual illuminant profile for my P45+, and I am quite happy with it. I cannot say about Hasselblad and Phocus, as Phocus doesn't support my P45+ and Capture One doesn't support Hasselblad.

Unfortunately, I don't shoot portraits so I cannot comment on skin tone.

Here are two samples from a recent shoot. Left one is mixed light, strobe and natural evening light. The one the right is with a golden reflector in natural sunset light. Both processed in Lightroom with my dual illuminant profile. Raw files are included.

http://echophoto.dnsalias.net/ekr/Articles/Shoots/PortraitSamples/

You would need a ColorChecker card and either Adobe DNG Profile Editor or ColorChecker Passport software to generate DCP (DNG Colour Profiles). Both programs are free. Adobe DNG Profile Editor can tweak the profile.

Best regards
Erik

...just to complete the raw format discussion:
so DNG export from Phocus would be identical to FFF ?
The formats would be read in LR just the same? (including 3fr)

I love love love the colors and rendering of Phocus, the use is kind of a pain though and I export the tiffs unchanged to Lightroom.

If somebody has a profile to get a similar rendering in Lightroom, I would pay for it. Might start a thread on that.

« Last Edit: December 21, 2014, 02:35:24 pm by ErikKaffehr »
Logged
Erik Kaffehr
 

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: The difference between 3FR and FFF format
« Reply #10 on: December 21, 2014, 03:24:16 pm »

...just to complete the raw format discussion:
so DNG export from Phocus would be identical to FFF ?
The formats would be read in LR just the same? (including 3fr)

I haven't verified to be 100% positive, but yes it should be identical is see no reason why it would not. The raw sample data in the DNG should be identical to that in the FFF.

However, Lightroom's color model is not the same as Phocus so color rendering won't be exactly the same. I have not formed any opinion on my own yet, but I'm quite sure most will prefer Phocus. Lightroom's reputation concerning color is not good in the MFD camp. The cooking of the raw data is not the same either so the result will differ a bit between Phocus and Lightroom. I think though that if we stay in Lightroom a FFF 3FR and DNG will look exactly the same, even if the DNG came from Phocus rather than Adobe's own DNG converter.
« Last Edit: December 21, 2014, 03:27:03 pm by torger »
Logged

BobShaw

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2218
    • Aspiration Images
Re: The difference between 3FR and FFF format
« Reply #11 on: December 21, 2014, 05:15:43 pm »

so DNG export from Phocus would be identical to FFF ?

Why would they be? DNG from any raw file is missing proprietary information. Read why Hasselblad dropped using it on their website.

I love love love the colors and rendering of Phocus, the use is kind of a pain though and I export the tiffs unchanged to Lightroom.
That's why you use it. Unless you have unticked all the boxes like lens correction etc on the right hand side then it won't be unchanged, it will be raw corrected.

If somebody has a profile to get a similar rendering in Lightroom, I would pay for it. Might start a thread on that.
You are already paying for Lightroom, but not Phocus! Personally I will use continue to use Aperture unless forced out as it uses the same raw processing as Phocus, being the Mac OS and is much more user friendly.
Logged
Website - http://AspirationImages.com
Studio and Commercial Photography

torger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3267
Re: The difference between 3FR and FFF format
« Reply #12 on: December 29, 2014, 03:14:35 pm »

I've now spent some serious time on the 3FR calibration data format and reverse-engineered the parts I need to get my H4D-50 files read properly in RawTherapee (will provide upstream to dcraw-Dave at some point, but for now it's only in RT). It seems to work well for the other 31/40/39/50 Kodak-based Hasselblad backs, but I haven't been able to test them all as most of my test files are FFF rather than 3FR. The older 22 and 16 megapixel backs have a slightly different format, and I imagine the H4D-60/H5D-60 with it's Dalsa sensor has a much different format as it needs a somewhat different type of calibration, and those I have not reverse engineered.

Before someone thinks that I find reverse-engineering fun, I'd say that it's the worst type of programming I do, it's a game of frustration, I really hate it and would love it if all manufacturers used open formats. However I hate it even more having a clunky workflow and I really want to deal with 3FR directly and archive and know that it can be read far into the future when Hasselblad and Phocus might be history. So I went for the challenge, and even though frustrating as you're stuck 90% of the time staring at numbers and trying to see a pattern, I can't deny that there's satisfaction when you finally figure it out.

Anyway, what does the calibration data contain? It contains more information than what's actually applied. The two major aspects which has a clear visible effect is flatfield correction and a special sort of level correction. The flatfield correction simply fixes subtle non-uniformities in color response over the surface (the IIQ format also have this). The levels correction was hard to figure out as it's a type of correction I haven't seen before. It transfers a small fraction of pixel data from neighboring pixels above and below, maybe compensating for some sort of signal leakage during readout. This may also explain why my H4D-50 back has "bayer green split", that is that green1 does not have the same response as green2. The difference is quite large before calibration (and the major reason why the file is not as usable without calibration applied) and much smaller but still present afterwards. I've noted that Adobe's DNG converter adds the "BayerGreenSplit" tag to indicate this issue (it's not a unique problem, it's present in a number of cameras) while Phocus DNG conversion does not include the tag. The remaing separation seems to be less than 1%, so it's not much to worry about.

If someone knows more about CCDs than I do and has an explanation of why the level correction is the way it is I'd love to hear it.

If someone's interested in the source code, it's here https://code.google.com/p/rawtherapee/source/detail?r=ed57ad3e96ea4f8cba643f3a117c530e3c4fc972
« Last Edit: December 29, 2014, 03:22:44 pm by torger »
Logged
Pages: [1]   Go Up