Adobe Discards Valuable Metadata When Processing RAW Files

[LesPalenik]

141 fields of potentially useful metadata destroyed

Don Komarechka writes:
"Imagine if Lightroom could tell if your camera was in Continuous shooting mode, detect at what frame-per-second value you were shooting at, and then look at the timestamp of your images to auto-stack your bursts together. What if I was able to sort my photos based on macro magnification, or create a smart collection that contains all of my bracketed shots or multiple exposure images? What if I shoot different subjects with different AF settings, and can then organize my images based on this? There are a lot of scenarios where this data can become useful for organizing large catalogs, but Adobe doesn’t decode it and considers it garbage data."

Discarding Valuable Metadata

[Mark D Segal]

Interesting, thanks for raising this; are there other raw converters that discard less of this metadata? Would not a keep/discard decision depend on whether the application itself will have tools that could use the data?

[howardm]

I would personally think it to be 'poor form' for something that considers itself a cataloging tool to make a keep/toss decision based on its own 'need' for it.  Somewhat like gamut reduction (maybe a poor metaphor), once you lose the data, you can't get it back.  The tool should do the right thing and keep the data for the future, even if it's not immediately useful (or used).

Of course the data is still in the original file and that isn't mod'd by LR but does a user really go down that road or do they export an image that has been processed by LR?  99% of the time, the latter so the data is zombied.

[Mark D Segal]

I'm part of your estimated 1% that keeps the raw files, and in fact I seldom convert to any other format - from camera to print all within LR most of the time. I think there could be many others doing this too; nonetheless, your point about the data being retained in case of future usefulness makes sense.

[john beardsworth]

Click bait....

Is it really unreasonable that software companies should ignore (not "discard") maker note metadata which they don't need, and which camera makers have failed to standardise or use consistently between models or over time? Look around a Lr catalogue's SQL and you'll find that Adobe don't "discard" the maker note metadata at all - it's stored in unparsed form.

[LesPalenik]

Click bait....

Is it really unreasonable that software companies should ignore (not "discard") maker note metadata which they don't need, and which camera makers have failed to standardise or use consistently between models or over time? Look around a Lr catalogue's SQL and you'll find that Adobe don't "discard" the maker note metadata at all - it's stored in unparsed form.

Are you saying that all of the original metadata is preserved? And if so, only in the original RAW file or also in the processed TIF file?

[AlterEgo]

Are you saying that all of the original metadata is preserved? And is so, only in the original RAW file or also in the processed TIF file?

if you keep original raw files then yes, it is naturally preserved there... only if you convert to DNG and discard original raw files then it is a well known fact that Adobe discards some data, intentionally or by error, and when found/caught they will often silently fix that in the next version, etc, etc... the best example of the client taken for a ride (with preservation) was the Library of Congress 

[Mark D Segal]

Click bait....

............

I don't know what you are trying to achieve with this kind of slur. I've gone through both the Petapixl link and the ExifTool page and see nothing that qualifies as "click bait", unless I'm terribly naive about hidden agendas, but at my age it's not very likely.

[john beardsworth]

Are you saying that all of the original metadata is preserved? And is so, only in the original RAW file or also in the processed TIF file?

I said in the Lr catalogue's SQL. Last time I checked, with Nikon raw files, it was "all", and it was then written out into the TIF file (for whatever good that is).

The article's author fails to understand that an inflammatory word like "discard" doesn't mean the same as "not use" or "ignore", and that the difference isn't merely semantic. 

[john beardsworth]

I don't know what you are trying to achieve with this kind of slur. I've gone through both the Petapixl link and the ExifTool page and see nothing that qualifies as "click bait", unless I'm terribly naive about hidden agendas, but at my age it's not very likely.

It's a ridiculously inflammatory title, Mark, for what's just a superficial rant. Adobe, nasty Adobe, discard valuable metadata? Not "Camera Makers fail to record potentially-valuable metadata consistently and make using it worthwhile"?

Adobe and others don't use non-standard metadata for a variety of reasons. But do they really need it? "Imagine if Lightroom could tell if your camera was in Continuous shooting mode, detect at what frame-per-second value you were shooting at, and then look at the timestamp of your images to auto-stack your bursts together." Well, imagine if Adobe had actually introduced that auto stacking feature in Lightroom 1.0 and hadn't needed to figure out Nikon's way of recording Continuous in camera bodies x y z, in bodies a, b c, then Canon's way, Fuji's way etc.

[Mark D Segal]

It's a ridiculously inflammatory title, Mark, for what's just a superficial rant. Adobe, nasty Adobe, discard valuable metadata? Not "Camera Makers fail to record potentially-valuable metadata consistently and make using it worthwhile"?

Adobe and others don't use non-standard metadata for a variety of reasons. But do they really need it? "Imagine if Lightroom could tell if your camera was in Continuous shooting mode, detect at what frame-per-second value you were shooting at, and then look at the timestamp of your images to auto-stack your bursts together." Well, imagine if Adobe had actually introduced that auto stacking feature in Lightroom 1.0 and hadn't needed to figure out Nikon's way of recording Continuous in camera bodies x y z, in bodies a, b c, then Canon's way, Fuji's way etc.

OK John, "inflammatory" and "click bait" also mean different things. From what I read, I agree that "discard" and "destroyed" are clearly incorrect ways of describing the status of the information. If "Discards" were changed to "Doesn't Use...." the title would be more accurate; but to my mind, the fundamental point remains that while the data remains there, whether it gets used say in LR obviously depend on whether Adobe builds tools that need to use it. If they aren't, it's a moot issue - because as long as the data remains available in the raw file it will be used if tools that need it be developed in the future.

[digitaldog]

The first question you all should be asking is, does this EXIF data amount to a hill of useful beans?
AFAIK, any of this data, useful or not, could be kept, or embedded somewhere in a DNG.

[rdonson]

One thing that Adobe has always omitted from Canikon cameras is focus points.  It is available in the metadata.  Products like BreezeBrowser and the Canon and Nikon software have shown those focus points for over a decade.  That is often useful although perhaps not for everyone.

I don't know if other companies include focus point data in their metadata. 

[Mark D Segal]

Hi Ron, absolutely agree. Having this information on hand would be very useful for analyzing (after the fact), when it occurs, why a photo doesn't look sharp enough in the right places!

[LesPalenik]

I looked again at the original article on Petapixel, and noticed a lively forum discussion also there. Quite interesting and educational.
One poster there mentions another "missing" piece of data (lens magnification factor). I would imagine that this attribute could be also very useful for filtering the images in LR.

And contrary to John Beardsworth accusation about Click and Bait, my intentions were only to share potentially important information, and nothing nefarious. I definitely learned something new by reading the posts on both forums, Lula and PP.

[Simon Garrett]

There is proprietary data in custom metadata fields in camera-written jpegs and raw files (and raw files are in any case usually proprietary formats).  The meaning of those is usually not described publicly, and sometimes the precise meaning is not clear anyway. 

Adobe could copy the proprietary fields when writing new versions of the files (often in different formats), although they might argue that they've no business writing data in someone else's proprietary format.  In some cases that maker's data, originating from the raw file, would simply not apply (or not be relevant) when written to a jpeg or tif. 

I quite agree that it's a pity that proprietary data is lost, but at the same time it's not entirely clear what to do with private-format data, especially when you don't know exactly what it means. 

[john beardsworth]

And contrary to John Beardsworth accusation about Click and Bait, my intentions were only to share potentially important information, and nothing nefarious. I definitely learned something new by reading the posts on both forums, Lula and PP.

For clarity, I was referring to the article's inflammatory title as click bait, not your repetition of it.

[rdonson]

There is proprietary data in custom metadata fields in camera-written jpegs and raw files (and raw files are in any case usually proprietary formats).  The meaning of those is usually not described publicly, and sometimes the precise meaning is not clear anyway. 

Adobe could copy the proprietary fields when writing new versions of the files (often in different formats), although they might argue that they've no business writing data in someone else's proprietary format.  In some cases that maker's data, originating from the raw file, would simply not apply (or not be relevant) when written to a jpeg or tif. 

I quite agree that it's a pity that proprietary data is lost, but at the same time it's not entirely clear what to do with private-format data, especially when you don't know exactly what it means.

Its been a long time since I looked at the Canon SDK for developers.  When I last looked the focus points were in the SDK.  It may be that Adobe prefers NOT to sign the agreement with Canon for the SDK but at least some of the metadata is documented and available.

[john beardsworth]

Its been a long time since I looked at the Canon SDK for developers.  When I last looked the focus points were in the SDK.  It may be that Adobe prefers NOT to sign the agreement with Canon for the SDK but at least some of the metadata is documented and available.

So they sign the Canon SDK, hope (perhaps in vain) that Canon do stick to their own documentation, and keep updating software when the SDK changes. They do the same with Nikon and a few other major camera makers - until users with less-common but supported raw files demand equal treatment.

Even if we limit the exercise to focus points, how much cost can be justified? People have already said  focus points are of limited value. Anyway, wouldn't we be better off with a focus mask or similar? Surely there has to be a limit to how much software vendors should be force to dance to the crazy tunes played by camera makers?

[rdonson]

John, I really don't care how much it would cost Adobe.  They'll either do it or they won't.  Obviously they haven't done it since we started using ACR and then Lr.  I'm also not going to lose sleep over this. 

Before ACR I used BreezeBrowser and it was a nice feature to see the focus points.  Using Canon DPP back then was akin to a sharp stick in the eye.  Seeing the focus points helped us figure out that our DSLRs were back/front focusing among other things.  With new zone focusing systems it would be nice to see what the camera actually focused on without guessing.  This isn't an issue for most landscape photographers because landscapes rarely move when you're photographing them.  If you shoot action or sports though it would be a nice tool to have.  Perhaps at some point AF systems will be so good we'll just assume that anything that's not tack sharp is operator error.

Photoshop's Focus Mask is a very slick tool but not a substitute for seeing what the camera actually focused on.