Luminous Landscape Forum

Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: gerryrobinson on April 19, 2013, 05:17:29 pm

Title: Should Camera Companies adopt non-proprietary raw?
Post by: gerryrobinson on April 19, 2013, 05:17:29 pm
I'm starting this to ask:
If we, as a photographic community,
believe it is our best interests to support the adoption of a non-proprietary raw format, by the camera manufactures or not. If so why and if not why.

While reading a recent thread it was pointed out how large the LULA community is and since the camera companies just might (hopefully) be listening let's tell them what we think.
Personally I think that a standard format is necessary for archival reasons, for the long term preservation of our images, as well as being able to re-develop the image with newer processes, which, for me, is one of the main advantages that digital photography offers. Since the camera companies create the formats for their products, the ball seems to be clearly in their court.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 19, 2013, 07:58:47 pm
I'm starting this to ask:
If we, as a photographic community,
believe it is our best interests to support the adoption of a non-proprietary raw format..

Gerry, the answer is a resounding yes!

Currently DNG would fit the bill.
Far from being a proprietary format it has been developed, yes by Adobe, with the express purpose and hope of being adopted for exactly that role.
Adobe has ensured that the DNG format is open and has offered the format lock, stock, and barrel to the ISO so that the format could become an industry standard.
Some people are seeing a conspiracy theory in this but I invite anyone to research the development of the TIFF format (who did that) and its eventual adoption as a standard format by the ISO.

However, does this mean that DNG would have to be the format eventually adopted? Simply, no.

Many mainline camera manufacturers are currently ignoring the issue however there are no technical reasons mitigating against a DNG being generated in-camera.
Even if they offered the option of DNG (or another mutually agreed on open RAW format) alongside their own proprietary format this would be a great step forward.
In the last few days I have read some wild arguments about firmware bloat as a consequence but again some knowledge about the DNG format and its historical timeline demonstrate an intriguing fact: before the release of DNG as an open format most camera manufacturers had appalling RAW formats that were a real mess. Subsequent to the release of DNG the next generation of proprietary RAW formats across a lot of brands suddenly improved out of sight. Coincidence?

A commercial argument has also been put forward to defend the camera-makers current stance. However, Jeff Schewe has been making the point, ad nauseum, and he is right, that the real 'secret sauce' for the camera-maker lies in the sensor/processor, and what that combination does, not in the subsequent recording of 0's and 1's in the RAW file.
Even if the makers want to embed secret information in the RAW file for their own RAW processing software the DNG standard already allows for that.
As already pointed out it is likely that DNG genetics are to be found in many proprietary RAW formats right now. So, for those who know how this works there would not be a financial cost for the technical adoption of DNG, or perhaps another RAW format.

Rather, the prime stumbling block appears to be a stubborn selfish pride by most camera-makers to even acknowledge that there may be a problem.
Actually, my last statement is pure speculation since the SILENCE from these camera-makers on this issue has been ABSOLUTELY DEAFENING.
So, we have no real way of knowing what their motivations are for ignoring the issue just that it would seem to have nothing to with a technical barrier nor, in my opinion, is it a financial/marketing issue.

What is going to be needed is sustained, reasoned (and reasonable) pressure from us - their customers - to address the issue.
Once the bulk of camera-makers acknowledge the desire for a standardized open RAW format, my guess is that the ISO, for them anyway, will move fairly quickly on this. Everyone knows what needs to be addressed for ISO standardization to be achieved.

Only time will tell whether DNG or some other format will be adopted.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Eric Myrvaagnes on April 19, 2013, 08:10:46 pm
Tony is absolutely right: "The answer is a resounding yes!"
In that other thread, Jeff Schewe and Andrew Rodney explained the issues (over and over again) very clearly.

As a Canon owner all my raws come as either CR or CR2 files, and since LightRoom and PhotoShop handle those so well, I've not bothered to save them as DNGs. But I am about to start saving all of my raws as DNGs on a separate hard drive. Once we have a standard, non-proprietary raw file format, I'll switch to that. But I fear it may not happen in my lifetime.

Eric M.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 19, 2013, 10:18:55 pm
Many mainline camera manufacturers are currently ignoring the issue however there are no technical reasons mitigating against a DNG being generated in-camera.

Actually, with DNG v1.4 it could be said that DNG has actually been advanced to the point where DNG is now a different story. DNG 1.4 allows two additional flavors of DNG, Lossy DNG–which takes the linear raw image data, maintains the linear scene referred image data and allows JPEG compression which means that Lossy DNG maintains a scene referred image (as apposed to a gamma encoded and thus baked color space) which allows far more flexibility in editing that JPEG offers. The other option is downsampled Lossy DNG.

The potential file size savings could mean that camera makers could offer both standard DNG and Lossy DNG instead of native raw and JPEG.

While Lossy DNG is not truly raw–it requires demosiacing while maintaining a linear gamma, the same as the previous Linear DNG offered, the ability to add JPEG (currently about JPEG level 12 I think, but that could change) mean you can keep an image in scene referred color while maintaining raw type editing.

So, here are the numbers for a Canon 1Ds MII CR2 raw file:
Native raw: 27.3MB
Raw converted to normal DNG: 23.4
Raw converted to Lossey DNG (no downsampling): 8.2MB
8-bit JPEG level 10-12: 4.5MB

So, Lossy DNG is not quite as small as JPEG, but retains the flexibility to retain raw style editing from a linear file vs. trying to edit a baked color space/fixed gamma JPEG file.

If you wanted to save even more and only need a lowered amount of resolution, you could use both lossy DNG plus downsampling–with far better potential than Canon's SRAW. So, if you took a 24MP raw capture and wanted a lossy DNG that was 10MP in size, the resulting file size would be down to 4.3MP.

So, what you THINK you might know about DNG, has recently changed. There's an except from my Digital Negative book detailing some of the recent changes in DNG in a DPP article: DNG File Format & DNG Converter (http://www.digitalphotopro.com/technique/software-technique/dng-file-format-and-dng-converter.html).

BTW, this article also contains info about the National Digital Information Infrastructure and Preservation Program (NDIIPP; www.digitalpreservation.gov (http://www.digitalpreservation.gov/)) about some of the aspects and risks inherent in the long term conservation and preservation of all types of original digital files (it's not just a digital photography issue). Another organization that is working on these issues is the National Archives (http://www.nationalarchives.gov.uk/) in England.

To my way of thinking, this is all a no brainer...photographers who understand the issues (and it would behoove photographers to know this stuff) should lead the industry to the position that the current proliferation and spread of undocumented, proprietary raw file formats is putting our digital original raw images at risk for long term access for posterity.

There are different views about what to do about this situation...no, I don't suggest boycotts of camera companies who don't document their raw files. Photographers are too independent (and hooked on new cameras) for that to work. But, I do think that if the photographic industry at large all shared the view that undocumented, proprietary raw files are less than optimal and that the camera companies should be pressured by public opinion to reassess their positions and work toward improving the current poor situation and also come forward and offer to work with various agencies that are working towards long term conservation and preservation practices and standards would be a good result.

And no, I'm also not suggesting that photographers rush to convert all their proprietary raw files to DNG and discard their original raws...I've never said that nor has Adobe. I use DNG for specific reason in certain cases...

The main thing I see as being the best short term result is that photographers of all the various types of photography all agree that the current situation must change and to quit arguing on the behalf of the camera companies' continued poor policies regarding undocumented, proprietary raw files.

At this point, digital photography has been like the wild west with unprecedented advances and changes in the industry. As we've seen recently, the revolution is changing to an evolution. Cameras and sensors are making mild advances not revolutionary changes of late. This is the perfect time to start the serious discussion about the issues because it's now that various aspects of a standardized raw file format might actually become a competitive advantage for some camera companies. The current best example is Leica whose recent cameras all offer DNG raw file output which enjoys de-facto support in any application that full support DNG...Leica no longer needs to create and maintain raw file formats, all they need to do is follow the DNG specs (which is really not that hard–just look at the DNG SDK if you are geeky.

So...I've said my piece...as you might expect, Michael has similar thoughts. So, if you have technical questions, post them here. I'm pretty sure we have an excellent LuLa member here that can speak directly to any technical DNG issues, Eric Chan (MadManChan). Just know that it would be more useful to have a civil discussion without name calling and other bullshyte taking place. On the previous thread about this, I tried very hard at restraint and encourage the Mods to be flexible and not close the thread...
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: ErikKaffehr on April 19, 2013, 11:58:15 pm
+1

I use DNG, serves me fine.

Erik Kaffehr
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jwstl on April 20, 2013, 12:27:42 am
I'll admit up front that I don't know a lot about this subject but it seems to me the camera makers would be more likely to include a DNG option once it's accepted by the ISO. Until then it's someone else's format so the camera makers feel they might as well stick with their own.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bryan Conner on April 20, 2013, 01:14:23 am
I fully support having dng as an option in camera.  I convert all of my Canon raw files to dng upon import.  I do so mainly because of the smaller file size.  I can see absolutely no difference in the file content on my screen when pixel peeping and certainly not in a resulting print.  I think that dng is at least just as future proof concerning the ability for people in the future to be able to read the file format when compared to ANY raw format.  The fact that the Diginerds/geniuses at Adobe can take the manufacturer's proprietary raw files that contain "secret" information and reverse engineer it to create an application that does a better job in both output as well as user interface than the people that create the source file tells me all I need to know. 

I have really enjoyed reading Jeff's and Andrew's comments in other threads.  I just wish the two of them would open up and really tell us what they think on the subject.  :D

I have often thought about sending my Canon software cd that contains DPP back to Canon with a note attached stating that there software does not support an open documented raw file format, therefore I can not use it.  Please give it to someone that can use it.  I do not use it even for CR2 and CRW files that my camera generates, by the way.  Adobe's PV2012 and ACR/Lightroom produces the best overall image quality with my files in my opinion.  YMMV.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bart_van_der_Wolf on April 20, 2013, 04:36:14 am
In the last few days I have read some wild arguments about firmware bloat as a consequence but again some knowledge about the DNG format and its historical timeline demonstrate an intriguing fact: before the release of DNG as an open format most camera manufacturers had appalling RAW formats that were a real mess. Subsequent to the release of DNG the next generation of proprietary RAW formats across a lot of brands suddenly improved out of sight. Coincidence?

Hi Tony,

I suppose you have some objective evidence for that theory?

Quote
A commercial argument has also been put forward to defend the camera-makers current stance. However, Jeff Schewe has been making the point, ad nauseum, and he is right, that the real 'secret sauce' for the camera-maker lies in the sensor/processor, and what that combination does, not in the subsequent recording of 0's and 1's in the RAW file.

Is that so?

Raw files do more than record the exposed and converted data as processed by the camera. Outside the exposed image area there is a number of proprietary masked and other sensel data (to do with black level, vignetting, banding, amongst others). Is all that copied into the DNG file?

Quote
Even if the makers want to embed secret information in the RAW file for their own RAW processing software the DNG standard already allows for that.

If so (and it certainly wasn't in early DNG versions, so who guarantees it is now?) what does a(n) (error free?) conversion add for high quality future conversions? DNG is just a conversion to another format. My Raw files can be read just fine, bit by bit, and can be converted into an image by the manufacturer's software (with generally reported better color than from other converters), and by a number of independent software solutions, and even with something as universal as DCRaw, although they won't be able to utilize the proprietary data in the Raw file (that is, assuming it is still there).

Quote
As already pointed out it is likely that DNG genetics are to be found in many proprietary RAW formats right now.

Is that so? Do you have any credible and independent source for that theory?

Quote
Rather, the prime stumbling block appears to be a stubborn selfish pride by most camera-makers to even acknowledge that there may be a problem.

It's just another theory. Do you really think that pride is the reason, or could it be that they see no benefit in putting Adobe in a spotlight. What commercial benefit would having to spend resources in keeping up with a changing file-format bring them? Besides, it would be more of a benefit for Adobe to only having to deal with a single Raw file format than the manufacturers of cameras. Is Adobe willing to share some of those savings with the camera manfacturers?

The camera manufacturers now have the liberty to produce hardware, which produces a Raw file, with exactly the data in it that the camera generates, and can utilize for a potentially high quality output. Then they move to the next technological step and that will produce a somewhat different Raw file, without the need to think about incorporating deprecated legacy parameters or file structures. They can also reserve space for features that didn't make it in time for the initial release, but which will be needed for a later firmware upgrade (and they won't have to tip off Adobe about it in advance).

I think people have a too romantic view of commercial manufacturing companies. They are not in this world to secure the future of our Kodak moments, they are here to make money. I'm not saying that's a good thing, but it is what it is. They will only adopt something universal, if it brings a lot of clear (competitive) benefits, and no downsides.

Cheers,
Bart
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 20, 2013, 05:09:36 am
I'll admit up front that I don't know a lot about this subject but it seems to me the camera makers would be more likely to include a DNG option once it's accepted by the ISO. Until then it's someone else's format so the camera makers feel they might as well stick with their own.

Unfortunately the problem is likely to be a bit circular with not a lot of action likely to be taken on standardization of DNG, or some other format, in this context unless the big camera boys make the right noises.
So to suggest that camera makes should wait for the standardization of a format before adopting it won't work.

A lot of effort is going to be needed to bring a very unwilling bride and groom together for their nuptials.
However since what is wanted is a good and constructive thing a shot gun would not be appropriate for the wedding.
It is only going to work if the camera makers see, appreciate, and accept the benefits of what is being proposed.

With regard to DNG itself is a rapidly maturing format with considerable flexibility and utility and could be an excellent choice for a standardized RAW format.
In reference to Jeff's comment on how DNG is changing - so is every other useful format.
JPEG and TIFF are regularly being updated - it is just that the ISO controls the process.
One of the key issues here is ensuring continued compatibility between newer versions and older versions.
So, in contradistinction to several posts decrying DNG because it is "changing", we want useful formats to continue to be updated and upgraded because doing this maks them even more useful with time.

The bigger Digital Asset Management problem is growing every second with formats being orphaned all the time including sound, video, graphics, and of course RAW photographic formats. Trillions of documents and files of various nature are already, to all practical purposes, orphaned and not accessible. Perhaps, if you have the budget and time of a large government agency and acces to both the software and hardware that made that particular format and you are able to get it working it is ,perhaps, possible to port information from one format to another.
For most of us though software that ran on Windows95, remember that?, so recently and is not currently supported on newer hardware and operating systems is gone for us.
I don't myself have access to a system that could run Windows95.
Offhand, I don't know anyone who does.

With regard to cameras dozens of first-generation RAW formats are already orphaned not least because their manufacturers are out of business and no-one else is prepared to support their formats now.
On another note, the third-party RAW converters, generally try to maintain backward compatibility of as many formats as possible but currently I doubt many will add dead formats now because very few individuals will be using those cameras.
The problem is that millions (?trillions) of images may be residing on hard drives completely inaccessible because no software can currently open them.

The point of the current exercise is to prevent this problem proliferating to truly catastrophic levels.
Having a open format RAW would be the first step in safeguarding our precious images (nothing necessarily to do with financial worth - but for many it would be) but bullet-proof back-ups and archives are also required if we are not lose our images.
Basically we are in the unenviable position of negatives of my grandmother shot in 1910 surviving for another hundred years and my super-dooper digital RAW images dying a death on an expired hard drive or having nothing to read them with in a few years.
Even companies like Canon, Adobe, Nikon, and Phase One are not immune from the extraordinarily brutal Darwinian process governing business in the digital age. Remember Kodak and IBM - they WERE truly iconic multinational companies that at one time almost defined business.
Kodak is dead and buried, and IBM: so invisible currently that I am not sure whether they exist anymore!

So remember that nothing is forever and if we, both as individuals, and as a group, do not protect our interests no one else will.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 20, 2013, 06:41:49 am
Hi Tony,

I suppose you have some objective evidence for that theory?

Is that so?

Raw files do more than record the exposed and converted data as processed by the camera. Outside the exposed image area there is a number of proprietary masked and other sensel data (to do with black level, vignetting, banding, amongst others). Is all that copied into the DNG file?

If so (and it certainly wasn't in early DNG versions, so who guarantees it is now?) what does a(n) (error free?) conversion add for high quality future conversions? DNG is just a conversion to another format. My Raw files can be read just fine, bit by bit, and can be converted into an image by the manufacturer's software (with generally reported better color than from other converters), and by a number of independent software solutions, and even with something as universal as DCRaw, although they won't be able to utilize the proprietary data in the Raw file (that is, assuming it is still there).

Is that so? Do you have any credible and independent source for that theory?

It's just another theory. Do you really think that pride is the reason, or could it be that they see no benefit in putting Adobe in a spotlight. What commercial benefit would having to spend resources in keeping up with a changing file-format bring them? Besides, it would be more of a benefit for Adobe to only having to deal with a single Raw file format than the manufacturers of cameras. Is Adobe willing to share some of those savings with the camera manfacturers?

The camera manufacturers now have the liberty to produce hardware, which produces a Raw file, with exactly the data in it that the camera generates, and can utilize for a potentially high quality output. Then they move to the next technological step and that will produce a somewhat different Raw file, without the need to think about incorporating deprecated legacy parameters or file structures. They can also reserve space for features that didn't make it in time for the initial release, but which will be needed for a later firmware upgrade (and they won't have to tip off Adobe about it in advance).

I think people have a too romantic view of commercial manufacturing companies. They are not in this world to secure the future of our Kodak moments, they are here to make money. I'm not saying that's a good thing, but it is what it is. They will only adopt something universal, if it brings a lot of clear (competitive) benefits, and no downsides.

Cheers,
Bart

Bart, if you are asking whether I have evidence on a level that a Grand Jury might find useful, then no!
However if you are prepared to do some digging around you will find references to the interesting coincidence regarding the state of many proprietary RAW formats pre and post the release of DNG as an open format.
I don't have a specific list of references to give you: my statement (theory) was a summary composite of sources read over the years.
Do your own research and draw your own conclusions.

As for the structure of DNG I seem to recall even Wikipedia mentions the options for companies to place proprietary information into DNG files. Perhaps you could look at more technical information such as the specification - I have it downloaded somewhere but it is not the most user-friendly document in existance today.
BTW I am well aware of metadata additions to the file that govern the issues that you mention. DNG is able to interface with that information as well as any other RAW format. Certain manufacturers are using DNG as their RAW format in current models that are in your local camera store. Also bear in mind that none of this is lost if one converts proprietary RAW to DNG in Lightroom or with Adobe's standalone DNG converter.
Also, if that information is really hidden unable to be accessed by DNG then no third-party RAW converter will have access to that information either.

Considering that even blind Freddy can see that there is a growing, and alarming, problem with the proliferation of RAW formats do you not think it perplexing the absolute silence that, particularly but not exclusively, Nikon and Canon, maintain on this issue. Your quote of my text was somewhat one-sided since I go on to stress that none of us know what they are thinking - PRECISELY BECAUSE THEY WON'T TALK ABOUT IT: NOT ONE WORD, NIKS, NADA, NOTHING!

There is nothing romantic in anyone's view who believes that a universal standardized RAW format is the way forward.
That is not the end of the problem should it be solved since we also have to have safe, archival-level storage of our images that is cost-effective.
If you are the CIA you probably have a solution - for the rest of use it is still costly, but improving.

Basically, nothing will change without sustained consumer effort.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 20, 2013, 09:43:26 am


Cheers,
Bart

Hi, Bart...thanks for the reasonable, thought out logic.

Unfortunately, this discussion is much like any discussion with someone from the far right side of religion, abortion, gun rights, etc.  

I respect their position.  However, they do not respect mine.

I, at least for now, have decided that any response or attempt at logical discussion with them only gives them a soapbox to preach from.  I am hoping that lack of response will allow any thread they open to have a quick dead.

Frankly, it really does not matter.  Any thought that these discussions by a few will influence the actions of Canon or Nikon are, in my opinion, delusional.

So...my thought...let them talk to themselves...   :D
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 20, 2013, 12:23:20 pm
Raw files do more than record the exposed and converted data as processed by the camera. Outside the exposed image area there is a number of proprietary masked and other sensel data (to do with black level, vignetting, banding, amongst others). Is all that copied into the DNG file?

Yep...outside of some early raw file formats (Canon's CRW is one I know of) and some oddball raw file formats like Sigma cameras, all the raw image data and all metadata–even if DNG doesn't understand or use them–is stored in the DNG. Both Thomas and Eric have stated this and there are provisions for this metadata migration in the DNG SDK.

I also know (cause I've seen it) that certain individuals can actually take a DNG without an embedded raw and reconstitute a valid proprietary raw file format from it. Currently, there is no way of doing that which has been released...so it's sort of a home brew operation :~)

DNG is simply a well formed and documented raw file format that is consistent with TIFF-EP which is the basis of most the current crop of proprietary raw file formats out there (with a couple of exceptions such as Sigma).
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on April 20, 2013, 03:18:25 pm
Yep...outside of some early raw file formats (Canon's CRW is one I know of) and some oddball raw file formats like Sigma cameras, all the raw image data and all metadata–even if DNG doesn't understand or use them–is stored in the DNG. Both Thomas and Eric have stated this and there are provisions for this metadata migration in the DNG SDK.

I also know (cause I've seen it) that certain individuals can actually take a DNG without an embedded raw and reconstitute a valid proprietary raw file format from it. Currently, there is no way of doing that which has been released...so it's sort of a home brew operation :~)

DNG is simply a well formed and documented raw file format that is consistent with TIFF-EP which is the basis of most the current crop of proprietary raw file formats out there (with a couple of exceptions such as Sigma).

Yeah, but as an open format, it has a weired downside as well. In the motion version, the fact that it is open actually blocks the developpement of support into proprietary NLEs. That also happens with open EXR too. Nobody is willing to develop the support and put engineers at work as it will also be usable by the competition. So we got this surreal situation that DNG cinema can be supported by a camera that features it,
in the case of Black Magic, in their software Resolve; but it's not a standart yet. Only if and when they'll have no other choice that commercialy support it (because the demand will be such) they will do it.
For example, Avid has an environement where proprietary formats can write their codes for native support, say Red for example. It's a convention between the brands, But who they will ask to for DNG support? (exactly like what the US president asked to the europeans: "if I want to call Europe, who am I talking to?").
 
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: PeterAit on April 20, 2013, 03:39:57 pm
To be honest, this strikes me as a solution in search of a problem. If you are worried about future lack of support for proprietary RAW files in the future, then convert to DNG on import. Other than possibly making your imports a bit slower, this would seem to solve all the potential problems that people are fretting about. No?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 20, 2013, 05:07:06 pm
If you are worried about future lack of support for proprietary RAW files in the future, then convert to DNG on import.

Yes, but that puts the burden on the user and again gives the camera companies a pass to continue the proliferation of undocumented, proprietary raw files. So, this doesn't address the underlying problem, just shifts the burden to millions of users instead of a relatively few camera makers.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: PeterAit on April 20, 2013, 05:26:26 pm
Perhaps LightRoom's default could be to convert imports to DNG? No burden on the user, no need to whine to the camera manufacturers, and we can get back to making beautiful photos.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 20, 2013, 05:43:45 pm
Perhaps LightRoom's default could be to convert imports to DNG? No burden on the user, no need to whine to the camera manufacturers, and we can get back to making beautiful photos.

Again, this is less about DNG (it's a proposed standardized raw file format submitted to the ISO for inclusion in the next TIFF-EP spec) and more about dealing with the root problem of the proliferation of undocumented, proprietary raw file formats. If you don't want to deal with the issue personally, that's fine. However, there are people who see the issue as a problem so it would be useful if those who don't care at least do nothing to interfere with those who do care.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on April 20, 2013, 05:49:32 pm
Yes, but that puts the burden on the user and again gives the camera companies a pass to continue the proliferation of undocumented, proprietary raw files. So, this doesn't address the underlying problem, just shifts the burden to millions of users instead of a relatively few camera makers.
Exactly. Those are the key words describing the situation, perfectly resumed in this sentence.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 20, 2013, 06:04:02 pm
Yeah, but as an open format, it has a weired downside as well. In the motion version, the fact that it is open actually blocks the developpement of support into proprietary NLEs. That also happens with open EXR too. Nobody is willing to develop the support and put engineers at work as it will also be usable by the competition.
 

Well, I'm not at all familiar with the issues of digital video file formats...I'm aware of CinemaDNG (http://en.wikipedia.org/wiki/CinemaDNG) but I really have knowledge of the issues and the various venders involved. So, I don't know if CinemaDNG has any technical benefits or weakness. I gather it's gotten some support but I don't know where it stands relative to the DNG raw file format.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on April 20, 2013, 06:15:42 pm
Well, I'm not at all familiar with the issues of digital video file formats...I'm aware of CinemaDNG (http://en.wikipedia.org/wiki/CinemaDNG) but I really have knowledge of the issues and the various venders involved. So, I don't know if CinemaDNG has any technical benefits or weakness. I gather it's gotten some support but I don't know where it stands relative to the DNG raw file format.
Apparently, Adobe removed it to Premiere support because officialy they were not happy with the delivered performances within the app. It's true that it is not "suitable" as an editing format, but then, so is "AVCHD". The prob is that nobody wants to write for free.
The main issue would be on the debayer aspect. But I'm afraid that now that Raw video is becomming more democratized, we will see the same mess very soon as still and an endless list of proprietary Raw formats wich support is going to put everybody crazy. Will the motion industry fully embrassed cinema DNG as a standart? I think there is a pressure among users but...
It happened to be similar with open EXR when Industry Light and Magic intended to built a standart. Here all FX artists are using it but as it's not supported everywhere we have roundtripping obligations and in the end the users are on bondage.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on April 20, 2013, 06:43:05 pm
Being a little more precise. Taking Red example, as Red is a commercial company that wants to make profits, sell their cameras and codecs; they developped supports
that now with time has become very matured. The metadatas files is clearly separated from the raw file  where the info remains untouched and those metadatas can
work in the camera itself, in the raw dev itself (provided by Red for free) or in the editorial facilities as well as finishing apps.

If you take for ex Adobe Premiere or Avid, the Red source settings is very similar and it works the same way, because the provider is Red. But who will take the initiative
to do that with cinema DNG ? There is no commercial brand involved behind and if Adobe or Avid are not willing to do the job for integrate support in their products as Red
did, we can't do anything and it's frustrating.

So it would be like independants decide to group their efforts for free, or free donations, a little bit like what happened with the Hacked GH2 or 5D2, but as they don't access
the brands core programmation, the only hoppe is that cameras that adopted cinema DNG have such great success that commercial interests have no other choices to
integrate it as a standart, forced to do it.

Because Arri talks to Apple and says: great, we will provide Prores option in our Alexas. Then, they talk to Avid and say: great, we will provide DNxHD in our Alexas too, so Avid guys will be happy.
But then, will they say: we will provide a cinema DNG version? ...no, because they have their Arriraw.
  
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 20, 2013, 07:58:10 pm
Perhaps LightRoom's default could be to convert imports to DNG? No burden on the user, no need to whine to the camera manufacturers, and we can get back to making beautiful photos.
Peter, just to enlarge on Jeff's point.
This is NOT a DNG crusade.
This is all about the adoption of a standardized open RAW format.
DNG is the only candidate in play currently but it may not be the format eventually adopted.
What if many people converted all their RAW files to DNG and DNG was not adopted as the standardized RAW format.
If Adobe were to subsequently go out of business and DNG lost its support then all those DNG files would effectively be orphaned.
BACK TO SQUARE ONE!

So, the only way forward is the adoption of a standardized RAW format.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 20, 2013, 08:31:54 pm
If Adobe were to subsequently go out of business and DNG lost its support then all those DNG files would effectively be orphaned.

Well, actually not...because DNG is publicly documented :~)
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 20, 2013, 08:44:46 pm
Well, actually not...because DNG is publicly documented :~)

True, but someone still has to actually continue to make the software that can see the file.
The fact that it is an open format makes this possible but does not guarantee that it actually happens.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 20, 2013, 09:32:20 pm
Perhaps LightRoom's default could be to convert imports to DNG? No burden on the user, no need to whine to the camera manufacturers, and we can get back to making beautiful photos.

BTW Peter it would be nice if all we had to think about was how to make beautiful photos rather than how to guarantee the survival of our digital negatives.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 20, 2013, 10:12:09 pm
BTW Peter it would be nice if all we had to think about was how to make beautiful photos rather than how to guarantee the survival of our digital negatives.

Tony Jay

Good grief....some of us try to do that....but some people keep dragging us away from taking pictures with FUD about the end of the world for future images...something which, after we have thought about it carefully, still do not believe it is an acute problem...at least one which needs constant harping on about.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 20, 2013, 10:31:24 pm
Good grief....some of us try to do that....but some people keep dragging us away from taking pictures with FUD about the end of the world for future images...something which, after we have thought about it carefully, still do not believe it is an acute problem...at least one which needs constant harping on about.

You can stick your head in the sand if you want to ... but don't complain if you get your arse shot off eventually.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 20, 2013, 10:41:21 pm
Good grief....some of us try to do that....but some people keep dragging us away from taking pictures...

You are under no obligation to pay attention to this subject nor this thread. And personally, this is the last message of yours I will ever respond to directly here on LuLa. You just ceased to exist to me bud.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 20, 2013, 10:42:51 pm
You can stick your head in the sand if you want to ... but don't complain if you get your arse shot off eventually.

Tony Jay

Thank you for you CONTINUAL useless words of warning.  I, like mny others, take full responsibility for our actions.  

I have been "warned"...but, of course, there is no current "solution"....only whinging for some future, questionable needed fix....with some unknown availability date, and without any proof that, even if it individually works, is the entire solution....or if there continue to be other items that need to be solved or looked out for.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 20, 2013, 10:45:03 pm
You are under no obligation to pay attention to this subject nor this thread. And personally, this is the last message of yours I will ever respond to directly here on LuLa. You just ceased to exist to me bud.

Stop acting like a child.  You cannot get someone to agree with your position, so you act like that...?

I have disagreed with your position, but agreed with your right to have it.

My comment was at TonyJay's comment, not your position...which I agreed was a situation needing attention...but not agreeing to your path of solving it....as others have also said.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 20, 2013, 11:01:48 pm
Good grief....some of us try to do that....but some people keep dragging us away from taking pictures with FUD about the end of the world for future images...something which, after we have thought about it carefully, still do not believe it is an acute problem...at least one which needs constant harping on about.
You are still not paying attention - every RAW image is at risk currently, PAST, PRESENT, AND FUTURE.
The fact that YOU regard this as FUD says much more about you than the issue at hand.

Jeff Schewe and Andrew Rodney are world-class authorities on matters photographic yet you have been telling them over and over for the last few days that they are talking CRAP (FUD).
Nothing that you have said gives anyone with even a smatter of knowledge on this subject of Digital Asset Management any confidence that you have the slightest idea what you are talking about yet you are dismissing individuals who do actually know their stuff.

In an earlier post on this thread you complained that you respect their point of view (not from my reading of your posts anyway) but no-one respects your point of view (could it be that you are the one talking crap?).
In fact Jeff Schewe in particular has been remarkably restrained in handling the faeces that you have been throwing at him. Not known as someone who tolerates fools lightly he has actually been very polite to you.

In the quote above you say that you have thought carefully about the issue and cannot see it as an acute problem.
I would suggest that you have not mainly because nothing in your contributions so far gives me any reason to believe that you have any grasp of the problem at hand.

As I said, it would be really nice not to concern oneself with issues of Digital Asset Management but we are NOT currently residing in that sort of universe.
Without a universal standardized open RAW format ALL digitalized RAW images are currently at risk.

Frankly, if you want to stick your head in the sand then, for you, the solution is easy - don't torture yourself by reading and replying to threads such as this.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 12:28:03 am
You are still not paying attention - every RAW image is at risk currently, PAST, PRESENT, AND FUTURE.
The fact that YOU regard this as FUD says much more about you than the issue at hand.

Jeff Schewe and Andrew Rodney are world-class authorities on matters photographic yet you have been telling them over and over for the last few days that they are talking CRAP (FUD).
Nothing that you have said gives anyone with even a smatter of knowledge on this subject of Digital Asset Management any confidence that you have the slightest idea what you are talking about yet you are dismissing individuals who do actually know their stuff.

In an earlier post on this thread you complained that you respect their point of view (not from my reading of your posts anyway) but no-one respects your point of view (could it be that you are the one talking crap?).
In fact Jeff Schewe in particular has been remarkably restrained in handling the faeces that you have been throwing at him. Not known as someone who tolerates fools lightly he has actually been very polite to you.

In the quote above you say that you have thought carefully about the issue and cannot see it as an acute problem.
I would suggest that you have not mainly because nothing in your contributions so far gives me any reason to believe that you have any grasp of the problem at hand.

As I said, it would be really nice not to concern oneself with issues of Digital Asset Management but we are NOT currently residing in that sort of universe.
Without a universal standardized open RAW format ALL digitalized RAW images are currently at risk.

Frankly, if you want to stick your head in the sand then, for you, the solution is easy - don't torture yourself by reading and replying to threads such as this.

Tony Jay

Tony, it is not an acute problem, that is one which will stop us from successfully processing our images.  Nor is there imminent danger of not being able to process our images in the near term.  There is a potential future problem, which may be alleviated with an open RAW standard....but it still does not provide a guaranty of no future loss...those who care still need to be vigilent

I have stated that an open RAW standard would be nice and something that I would be glad to use.  (though I doubt it is the total solution and others have posted that it might even be a problem as other may not accept it or see the economic advantage of using it).

I have used the term FUD (not CRAP, which is your term).  FUD has been and still is an accepted term as one method for attempting to convince consumers/buyers to not buy a competitor's product/idea/concept using uncertainty and/or mistrust vs. using factual features and benefits. In this case, it is being used continually to rally the mass of photographers to be unhappy with the "selfishness", "stupidity", "pride", etc. of the camera manufacturers.

OK....let's assume you had 100% of the photographers agreeing that an open RAW standard was really wanted.  However, there is no suggestion that these photographers take any action as far as buying cameras or using the RAW formats currently being provided.  Also, at that point (this is still an assumption) there would still need to be work done to translate/convert native RAW to Open RAW.

So, the status is...camera manufacturers still producing native RAW.  Photographers still buying those cameras.  Open RAW available for use, but work needs to be to to prepare for use and then users need to convert if they want to use...and their only net gain is some potential archival potential....a lot of work for little gain.

What is need?  Get the camera manufacturers to produce Open RAW.  Great idea..!!  What is the value proposition for them to do it?  The 100% of photographers that would like Open RAW, but are still buying there products that do not have it?  As I said to Jeff and Andrew in the other thread, this needs to be accomplished in the back rooms and board rooms of the camera industry...both hardware and software.

Brad said earlier in this thread, what I have been saying all along, "...I think people have a too romantic view of commercial manufacturing companies. They are not in this world to secure the future of our Kodak moments, they are here to make money. I'm not saying that's a good thing, but it is what it is. They will only adopt something universal, if it brings a lot of clear (competitive) benefits, and no downsides..."

Yes, Data Asset Management is a concern.  I have never claimed to be a expert in it.  I am concerned about it and attempt to be vigilent in my actions, directions, and buying patterns relative to it.  However,  I do not go about daily worrying about items I can not have any effect on any more than I can personally have any effect on world peace.  If this forum were continually pushing me and being filled with assorted post about believing in world peace (with out any action statement), I would react quite similar as I have to the believe in Open RAW (with no action statement).

BTW....this forum is entitled "Digital Cameras & Shooting Techniques".  It is not a place for world peace.    It is a place to discuss "making beautiful pictures".  I question if it is a place for statements like, "...it would be nice if all we had to think about was how to make beautiful photos rather than how to guarantee the survival of our digital negatives."  But you have the right to make that....I have the right to ignore it or comment that your statement is "...FUD about the end of the world..."
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 01:12:39 am
If you believe that the issue is not acute then you blithely disregarding obvious facts.
Andrew Rodney himself told you how he lost RAW images from first-generation digital cameras.

You believe that your RAW images are safe - for how long: five years, ten, fifty years?
You may not care - that is your perogative.

For the industry and the photographic community as a whole with an eye to the future the lack of a standardized open RAW format makes it highly unlikely that any RAW images captured now will be accessible in fifty years, or even twenty years given the casualty rate amongst businesses involved in computing/software/digital/photographic genres.

You do have your head in the sand - simply put.

I was wondering when Jeff would lose patience with you - I guess he feels calling his contributions FUD did it.
It must feel great getting one of the world's most influential authorities on matters photography so thoroughly pissed off with you.

BTW as a relative newbie to this forum I am not sure you have any place suggesting what should or shouldn't be discussed on any particular subforum. Personally, if I had started this thread I would have put it in a different subforum but I feel sure that the moderators will shift it, if, and when, they feel so inclined. So, none of your business.

You are demonstrating your lack of understanding in almost every mini-paragraph of your latest post.
There is curently no need for camera makers to produce an open RAW format: one actually exists - DNG.
DNG has already been adopted by certain camera-makers.
You can buy cameras today in your local camera store that use DNG.

The DNG format has been available freely for any camera maker or software company to incorporate into their products.
Contrary to your continual mantra there are no massive technical barriers to overcome.
What is needed is consensus.
Will that process require certain changes to made? Almost certainly!
Since those changes will likely only involve issues of metadata and not the bulk of recorded information since DNG files as well as most proprietary RAW formats base their format on a particular iteration of the TIFF standard once consensus is reached engineering the final first standardized DNG format would be in the order of days.
Does that mean that DNG would then be cast in stone never to change again - no.
Any useful format is continually updated with due regard to backward compatibility (read the histroy of the TIFF and JPEG formats, gee haven't I mentioned that before?)

Consumer pressure in the past has been the only way many manufacturers of diverse products have made important changes to their products.
Leaving the directors of companies involved in photographic products to their own devices will not work - remember Kodak and the spectacular strategic bungles that their board made that led to the demise of one of the biggest, and formally most profitable, enterprises in our history.
Kodak developed the digital sensor and yet, inexplicably, failed to see the future, preferring instead to pursue their established analog products.
Come to think about it their approach and yours are eerily similar.

I am aware that none of what I am saying will benefit you, apparently, but it may help others who need a little bit of perspective.
Ultimately, DNG may never be adopted as the universal RAW format but it is likely that it will be a derivative of the DNG.

Your quip about saving the world shows both your contempt and stupidity.
Perhaps Jeff is right.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 01:28:02 am
If you believe that the issue is not acute then you blithely disregarding obvious facts.
Andrew Rodney himself told you how he lost RAW images from first-generation digital cameras.



This was not from digital cameras, but Photo CDs  He did not totally lose them, as there are means to recover them...he has to make decision if the effort is worth it.  With 20/20 hind sight it might have been avoided.

Please get your facts correct.

In any of the threads I have read, I do not believe that proprietary RAW format has yet be documented as a reason for image loss.  That does not diminish the risk...from a number of factors, only one of which is format.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 01:34:24 am

You believe that your RAW images are safe - for how long: five years, ten, fifty years?
You may not care - that is your perogative.


Frankly, fifty years out, without someone around to be concerned about the other aspects of DAM, I think there is more to worry about than RAW format.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 01:41:02 am
For the industry and the photographic community as a whole with an eye to the future the lack of a standardized open RAW format makes it highly unlikely that any RAW images captured now will be accessible in fifty years, or even twenty years given the casualty rate amongst businesses involved in computing/software/digital/photographic genres.



That is what makes the issue much bigger than just the RAW format. 

If we are going to get "glory-eyed" about saving the world of images, let us not be myopic about the proprietary nature of the RAW format.  How about the RAW converter software?  If we are not worried about that, then the only thing we should worry about preserving is the finished product...TIFF or JPEG.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 01:47:16 am

You are demonstrating your lack of understanding in almost every mini-paragraph of your latest post.
There is curently no need for camera makers to produce an open RAW format: one actually exists - DNG.
DNG has already been adopted by certain camera-makers.
You can buy cameras today in your local camera store that use DNG.

So are you saying that the solution is to only buy these cameras...I don't think you are.

DNG is NOT an industry standard.  While it is offered as one, it is currently an Adobe product.....and while I am sure their ideal is to benefit the market, they are the main benficiaries...at least currently...of it's expanded use.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 01:55:02 am

The DNG format has been available freely for any camera maker or software company to incorporate into their products.
Contrary to your continual mantra there are no massive technical barriers to overcome.
What is needed is consensus.


OK....so DNG has been made freely available....but not accepted by the majors.  You surmise no MASSIVE technical barriers...not sure what you mean or what experience you have in this area....but what is most important, are you aware of a detailed technical analysis and cost analysis by any major camera mfgr?  (and don't blame them if they have not shared it with you.it is the way business works).

You say that all is need is consensus (I am assuming here the community of photographers).  Have you ever done a business/financial plan for a major product from conception through launch?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 02:07:32 am

Consumer pressure in the past has been the only way many manufacturers of diverse products have made important changes to their products.
Leaving the directors of companies involved in photographic products to their own devices will not work - remember Kodak and the spectacular strategic bungles that their board made that led to the demise of one of the biggest, and formally most profitable, enterprises in our history.
Kodak developed the digital sensor and yet, inexplicably, failed to see the future, preferring instead to pursue their established analog products.


Consumer pressure is not "consensus" pressure, but buying pressure.  Kodak's death (probably a Harvard study by now) was most likely the result of poor insight into the market changes and consumer buying power....and the ability to "eat your children before someone else eats them".  That is, if your main source of income is film, you cannot protect that market by not developing digital products if others are going to develop them and have the market shift from film to digital.

If an open RAW standard has the ability to create such a market shift, then some majors may fall the way of Kodak as a result.  The way to get them to accept open RAW is to present them with the value proposition of these facts.  Consumer "consensus" without consumer buying change does not close the equation.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 02:12:20 am


Your quip about saving the world shows both your contempt and stupidity.


It is not contempt. My view is a pragmatic one.   The statement was an attempt to display your statements as impracticable idealism.

Calling someone stupid is for the school playground.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 02:18:48 am

BTW as a relative newbie to this forum

You registered in Sept 2011

I registered in Feb 2005
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Rhossydd on April 21, 2013, 05:22:05 am
Just to get back to the OP.

Yes, I'd like to see a ratified and standardised raw format that every camera maker supported and offered. It really is the 'missing link' in digital photography at the moment.

Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 05:34:10 am
Just to get back to the OP.

Yes, I'd like to see a ratified and standardised raw format that every camera maker supported and offered. It really is the 'missing link' in digital photography at the moment.

Thank you.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: stamper on April 21, 2013, 05:54:56 am
I'm starting this to ask:
If we, as a photographic community,
believe it is our best interests to support the adoption of a non-proprietary raw format, by the camera manufactures or not. If so why and if not why.

While reading a recent thread it was pointed out how large the LULA community is and since the camera companies just might (hopefully) be listening let's tell them what we think.
Personally I think that a standard format is necessary for archival reasons, for the long term preservation of our images, as well as being able to re-develop the image with newer processes, which, for me, is one of the main advantages that digital photography offers. Since the camera companies create the formats for their products, the ball seems to be clearly in their court.




I wonder why this thread was started considering the heat that the previous thread generated? This post doesn't have anything to say that wasn't said in the previous one.  What were the motives?  ::)
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 06:01:20 am
I wonder why this thread was started considering the heat that the previous thread generated? This post doesn't have anything to say that wasn't said in the previous one.  What were the motives?  ::)
I doubt there was anything sinister here Stamper, the OP is fresh out of the box.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bart_van_der_Wolf on April 21, 2013, 07:59:09 am
If you believe that the issue is not acute then you blithely disregarding obvious facts.
Andrew Rodney himself told you how he lost RAW images from first-generation digital cameras.

Hi Tony,

I'm not sure if/where Andrew, in another thread maybe, said that he lost Raw images.
Assuming he can read/copy the Raw files from disk, converting them to another format doesn't solve an issue.

Access to the code to interpret all the file data may be an issue when the original manufacturer ceases to exist, but there are plenty openly available solutions (DCRaw, RawTherapee, etc.), and a number of commercial ones, to decode the image data to a degree that an excellent image can be produced (sometimes with even better quality than the original solution). Without access to all of the original proprietary data (by means of an SDK, or with the source code), converting to another file format doesn't solve an issue.

To give you an idea, DCRaw apparently can convert (amongst others) the following Kodak Camera Raws (don't know which one Andrew Rodney is referring to):
•Kodak DCS200
•Kodak DCS315C
•Kodak DCS330C
•Kodak DCS420
•Kodak DCS460
•Kodak DCS460A
•Kodak DCS520C
•Kodak DCS560C
•Kodak DCS620C
•Kodak DCS620X
•Kodak DCS660C
•Kodak DCS660M
•Kodak DCS720X
•Kodak DCS760C
•Kodak DCS760M
•Kodak EOSDCS1
•Kodak EOSDCS3B
•Kodak NC2000F
•Kodak ProBack
•Kodak PB645C
•Kodak PB645H
•Kodak PB645M
•Kodak DCS Pro 14n
•Kodak DCS Pro 14nx
•Kodak DCS Pro SLR/c
•Kodak DCS Pro SLR/n

I have no idea what the proprietary data in those Raw files could add to the conversion quality, but I do know that the demosaicing quality has improved a lot since those cameras were current, so maybe todays conversion quality trumps the old, even without the missing info. I'm glad that we didn't convert the original Raws into something that is an interpretation/abstraction of the original data, and afterwards came to realize that we didn't convert it as good as we can today. Still, it would be nice to also understand the unknown bits.

Quote
You believe that your RAW images are safe - for how long: five years, ten, fifty years?

I don't think it's an issue about being safe. Safe from what? It can't be floods or Global warming, because converting to another format would not make any difference in that respect.

The only potential issue from a user perspective seems to be the interpretation of some of the proprietary bits, the non-imaging sensel data, and the undisclosed maker notes in the EXIF metadata (as far as they are related to image-data decoding). Again, without access to an SDK or original source code that utilizes that data, converting to another file format doesn't change anything for the better.

We also should realize that much (most?) of the turmoil is caused by companies like Adobe who, for commercial reasons, force people to update the image editing software, because newer camera models are deliberately locked out. It is therefore no surprise that Adobe also offers a 'DNG solution' (for an issue they partially create themselves), which would benefit them by not having to interpret multiple Raw formats.

Frankly, I'm increasingly amazed by the high conversion quality that programs like RawTherapee can produce, even without intimate knowledge of the unknown bits ..., and some of the issues I see with the ACR/LR Raw conversion quality. It would seem wiser if they spent more resources on that conversion quality before others (also lacking the proprietary info) take too much of a lead.

Cheers,
Bart
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 09:11:59 am
Bart:

I cannot speak for the details of what Andrew was referring to in another thread dealing with the same subject matter as this one.
Maybe the information that you have shared may benefit Andrew - who knows.

Safe in the context that I referred to the concept includes everything that would be important in digital asset management, partly but certainly not exclusively, it includes being able to use that RAW image that one may have so assiduously otherwise preserved.
If the format of that RAW file is orphaned (no software that recognizes it) in the future then what is the point of preserving it?

Actually writing that paragraph has stimulated my memory about another issue that Andrew raised with regards to his RAW files that became orphaned. It is also an issue that I have raised independently.
Just because an image format is technically accessible does not necessarily mean that for most of us it is practically accessible.
To use an extreme example: If the FBI requires an image from a file format that is practically orphaned, if it is technically possible, then the considerable resources of the FBI can be put to the task. If national security is at risk the FBI will find a solution.
If accessing RAW images requires us to find and run long out of date software on obselete computers running superceded operating systems for practical purposes this may not be possible even if it is technically possible.
There is no doubt that is a current reality and not just theoretical posturing.

Currently our hope is based on the fact that no more proprietary RAW formats are lost and third-party RAW converters continue to support formats for cameras that are no longer made by companies that no longer exist and do not go out of business themselves, and, if they do, that new players will reverse engineer those formats all supposing that they can access the software development kits in that situation.
I deliberately made that last sentence an absolute mouthful to demonstrate what I think is a dodgy proposition.
Again this is not a theoretical situation - many formats are already gone, dead and buried, along with the companies that engineered them.

How much easier just to have a universal open RAW format so even if camera-makers insist on continuing to spawn new RAW formats for every camera model that they produce at least we can convert those image timeously to the open format secure in the knowledge that at least the open RAW format WILL BE SUPPORTED by RAW converters into perpetuity.
(Of course there are lots of other ways that images need to be protected into the future but this is a fundamental step.)

The ideal situation is that each and every camera-maker actually adopts that open RAW format as the RAW format for every camera model that they produce.
Amongst other benefits this will remove that annoying situation of buying newly released camera models and having to wait several months for the third-party RAW converters to reverse engineer the RAW format and update their software.
It will also remove any need for us to convert our files from format to format.

The bottom line is that no RAW file format is safe.
The brutal Darwinian process that is especially prevalent in the electronics industry means that not even companies that currently hold iconic status are necessarily safe for the long-term.
Perhaps our own demise will precede that of Canon or Nikon or Phase One or whatever in which case the issue may not concern us personally but on a society-wide basis gazillions of digital images may be lost to the future.
Maybe one day we will have digital archeologists cum anthropologists whose entire job description is to try and figure out details of our existance from digitalized images and documents in formats where the initial challenge is just to decode the format itself.
Maybe the previous statement is a poor attempt at levity but the implications are real enough.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: PeterAit on April 21, 2013, 09:53:36 am
Peter, just to enlarge on Jeff's point.
This is NOT a DNG crusade.
This is all about the adoption of a standardized open RAW format.
DNG is the only candidate in play currently but it may not be the format eventually adopted.
What if many people converted all their RAW files to DNG and DNG was not adopted as the standardized RAW format.
If Adobe were to subsequently go out of business and DNG lost its support then all those DNG files would effectively be orphaned.
BACK TO SQUARE ONE!

So, the only way forward is the adoption of a standardized RAW format.

Tony Jay

Tony,

I understand, and I never thought it was a "DNG crusade!" Still, I just don't see any practical advantage to have a single accepted RAW file standard, therefore I question the wisdom of putting effort into that goal. Given that DNG is documented, even if Adobe withdraws support, the files will always be readable. Any competent programmer could write an application to translate DNG into whatever new "file format of the month" is being used.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 10:16:17 am
I understand, and I never thought it was a "DNG crusade!" Still, I just don't see any practical advantage to have a single accepted RAW file standard, therefore I question the wisdom of putting effort into that goal. Given that DNG is documented, even if Adobe withdraws support, the files will always be readable. Any competent programmer could write an application to translate DNG into whatever new "file format of the month" is being used.

I agree that it is possible for anyone with the knowledge to write software that recognizes DNG but that in no way guarantees that it will actually happen.

Also, It would be much preferable if none of us had to deal with this issue of porting our images from format to format to keep them current with software that can read them.
In the bigger picture considering the billions of RAW images that are being shot daily it doesn't take Einstein to figure out that not a high percentage of them are going survive the impost of being ported from format to format purely because the magnitude of the task is just overwhelming and it takes a lot of effort and motivation from a lot of people to keep it up.
You personally may be motivated enough to do it and perhaps large organizations with vested interests in the images that they own will do it but the bottom line is that the losses will be staggering.

If we had a universal RAW format produced out of the camera then none of these issues come up.
Any RAW converter/developer will recognize the image.
No reverse engineering formats required, no delays waiting for formats to be supported.
Images can be archived in the knowledge that in decades to come they will probably be recognized by software currently in production.
None of this cancels out the rest of the links of the chain required for robust digital asset management into the future but it is an essential first step.

The issue is much much bigger than one's personal image collection.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: PeterAit on April 21, 2013, 10:28:36 am
Well, Tony, I think that you and I just have different outlooks on the future, with you tending to fear the worst. Why fret about a possible future situation that almost surely will never come to pass, and if it does will be easily rectified? I have more important issues to fret about - like whether taking the lens off is a clockwise or counter-clockwise twist (don't laugh, my Nikon and Panasonic are opposite and I am forever getting it wrong!).
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 10:40:26 am

You personally may be motivated enough to do it and perhaps large organizations with vested interests in the images that they own will do it but the bottom line is that the losses will be staggering.



Yes they will....and it has nothing to do with RAW format.

Of all the analog negatives ever made....what % of them survive.  How many of them wound up in landfills when "Aunt Sally" died?  Assuming she had not already thrown them out to make room in her closet.

Even in the film industry, where great effort was taken to preserve the original works, much was lost...for many reasons....none of which was format related.

In digital...how many SD cards have been lost...which were never copied to a computer.  When the family photographer, "Uncle Harry" passes,  who is going take on the role to find all his digital negatives and preserve the bits someplace....either for memories of future family members, historians wanting a picture of a past era....or, maybe, a museum, when Uncle Harry is posthumously discovered as a great artist.

It is a much bigger issue...too big to be narrowly focus on an open RAW format.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 10:43:05 am
Peter, sadly the issue is not just a theoretical future one.
Many RAW formats are already dead and buried and who know how many millions (?billions) of RAW images already shot in those formats are just taking up useless space on hard drives, servers and other storage locations.
Those images are already dead men walking because there is no practical way of opening them right now.

The biggest fallacy appears to be that this issue is an airy fairy theoretical issue that may only happen in the future.
It is already part of our reality now, recognized, or not.

You may be confident that your .NEF files will be supported ad infinitum - I am not - and recent history is on my side.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 10:52:29 am
Peter, sadly the issue is not just a theoretical future one.
Many RAW formats are already dead and buried and who know how many millions (?billions) of RAW images already shot in those formats are just taking up useless space on hard drives, servers and other storage locations.
Those images are already dead men walking because there is no practical way of opening them right now.

The biggest fallacy appears to be that this issue is an airy fairy theoretical issue that may only happen in the future.
It is already part of our reality now, recognized, or not.

You may be confident that your .NEF files will be supported ad infinitum - I am not - and recent history is on my side.

Tony Jay

Please name one widely used (millions of RAW images) RAW format that cannot be opened.

The only ones that was put forward in Jeff's arguments were Kodak DCS and Photo CD.  Subsequent, reliable, information showed that all these images were accessible and usable.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Rhossydd on April 21, 2013, 10:54:59 am
I just don't see any practical advantage to have a single accepted RAW file standard
Really ?

A bigger question is why do camera manufacturers keep on changing formats and making it harder for software vendors to support their products ?
Does each new iteration of CR2/NEF/etc on every new model really help anyone ?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 11:10:17 am
jrsforums: as per usual your selectivity is astounding.

No-one, and certainly not I, has suggested that the question of an open RAW format, by itself, will solve every issue related to the safe and perpetual archiving of digital images.

Your are just trumpeting your ignorance from the rooftops.
There was a reason why it I stressed the term 'Digital Asset Management' in many of my posts. It was to emphasize that the issue is much broader than just adopting an open RAW format.
I invited you in one of my posts, if memory serves, to go and read up about the topic.
Again, if memory serves, you derisively dismissed me.
I also seem to recall earlier that you admitted that digital asset management was not an area of strength for you.
Do you really suppose that your latest comments are groundbreaking news to me, or anyone else for that matter?

The whole point of the exercise is to limit the attritional loss of RAW files into the future that could otherwise be saved and accessible into the future.
As I have just mentioned to Peter Ait the issue is much bigger than one's personal image collection.

You have shown conclusively that facts will not sway you.
You have complained, bitterly, that this topic, is depriving you of the opportunity to do other things that you would rather be doing.
Yet you keep coming back to layer inanity on inanity.
My medical training suggests that you have an interesting five-axis diagnosis.

This is the last post of yours to which I will respond.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 11:22:42 am
Really ?

A bigger question is why do camera manufacturers keep on changing formats and making it harder for software vendors to support their products ?
Does each new iteration of CR2/NEF/etc on every new model really help anyone ?

That is a good question....why do they do it?  In fact, why do the ALL (?) do it?

Do you think they do it to disrupt, get even, whatever with the software vendors?

Maybe they want to disrupt and delay the early adoption (purchasing) of their new camera until the software makers support the new RAW?

Some have suggest the manufacturers are just arrogant, stupid, prideful, sloppy, etc. when creating their new RAW format.

I have no fact...only suspicions....if someone really knows, we would all be made wiser by it.

However, I do know how many development operations work.  They are going to do things that give them the best product (features, operation, speed, etc.) and do it in the most efficient manner.  Expense is a key determinant to the actions taken.  If a standard format reduced, rather than increased expense, it would be used (unless it effected other parameters, such as operational speed or components required).

Forgetting, for the moment, the internals to the camera.  Canon provides RAW conversion software, which they make no money on.  However, even though they know the internals of RAW changes way before the rest of the world, they need to incorporate those changes into their code...which drives an expense, which will effect the overall cost, therefore the price, of the camera.  If this cost could be saved without causing increased cost or other issues you can be sure the Canon financial people would on this like flies on stink.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 11:41:00 am
jrsforums: as per usual your selectivity is astounding.

No-one, and certainly not I, has suggested that the question of an open RAW format, by itself, will solve every issue related to the safe and perpetual archiving of digital images.

Your are just trumpeting your ignorance from the rooftops.
There was a reason why it I stressed the term 'Digital Asset Management' in many of my posts. It was to emphasize that the issue is much broader than just adopting an open RAW format.
I invited you in one of my posts, if memory serves, to go and read up about the topic.
Again, if memory serves, you derisively dismissed me.
I also seem to recall earlier that you admitted that digital asset management was not an area of strength for you.
Do you really suppose that your latest comments are groundbreaking news to me, or anyone else for that matter?

The whole point of the exercise is to limit the attritional loss of RAW files into the future that could otherwise be saved and accessible into the future.
As I have just mentioned to Peter Ait the issue is much bigger than one's personal image collection.

You have shown conclusively that facts will not sway you.
You have complained, bitterly, that this topic, is depriving you of the opportunity to do other things that you would rather be doing.
Yet you keep coming back to layer inanity on inanity.
My medical training suggests that you have an interesting five-axis diagnosis.

This is the last post of yours to which I will respond.

Tony Jay

I have only responded to statements you have made.  If you have expressed them poorly, I cannot help it.

I have done some reading, mostly without your suggestion.
From: http://dpbestflow.org/node/634
Long-term accessibility
In the relatively short history of raw photography, there have been some formats that have become obsolete, and therefore largely inaccessible. The early Kodak cameras used a patented method to record color, and the software to read it has not been updated to work with modern computers. This has put the images at risk of inaccessibility.

For modern DSLR cameras (generally those produced after the year 2000), the risk of inaccessibility is significantly less. Even if commercial software packages eventually drop support for very old cameras, open source software will probably continue to provide access. The program dcraw, written by Dave Coffin, includes support for a very large number of cameras. Because it is open source, it is unlikely to simply go away. The codebase can be picked up and modified in the future by any number of independent developers.


You don't agree with my position, you call me stupid.  Then that I "complain, bitterly"...when one I responded "Good grief" to one of your arm waving posts.  Then that my responses are "inanity", mainly because you don't agree with them, not because they are not well thought out.

Doctor, if you are one, rather than playing remote, biased, diagnostician in an attempt to wn your arguement or to diminish your opponent....HEAL THYSELF

BTW....I do not see any FACTS from you.  I am waiting for the name of the one lost RAW format, with millions of images, which cannot be recovered.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bryan Conner on April 21, 2013, 12:40:15 pm
My momma used to tell me that I would argue with a fence post.  I wish she were here to read this thread now.  ;D
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: michael on April 21, 2013, 01:31:49 pm
OK guys. End of personal attacks.

On topic please, or it's closed.

BTW, one major university with a substantial collection of contemporary photography told me about a year ago that they were not accepting raw files from photographers work that they were collecting unless they were in DNG. Reason? They feared the inability for future generations to be able read the data.

They have a program in place to migrate over time to new storage formats (that's a given) but they believed (rightly or wrongly – and I think rightly) that they didn't believe and trust in proprietary formats for archival and historical storage purposes. They also told me that a national consortium of museum conservators were planning on making DNG their defacto standard across institutions.

So, my personal observation is... where's the beaf?

Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bart_van_der_Wolf on April 21, 2013, 01:56:02 pm
BTW, one major university with a substantial collection of contemporary photography told me about a year ago that they were not accepting raw files from photographers work that they were collecting unless they were in DNG. Reason? They feared the inability for future generations to be able read the data.

Hi Michael,

That's interesting, and very strange at the same time. A bit similar to storing an Ansel Adams negative (or a scan of it) to recreate one of his prints ... And in these times, a digitally enhanced output file may even be impossible to create from a single Raw file unless one also has the different adjustment layers and or Raw conversion versions that led to the final image. Very strange, unless they teach journalism.

Quote
They have a program in place to migrate over time to new storage formats (that's a given) but they believed (rightly or wrongly – and I think rightly) that they didn't believe and trust in proprietary formats for archival and historical storage purposes. They also told me that a national consortium of museum conservators were planning on making DNG their defacto standard across institutions.

One can only hope they don't throw the original Raws away, and embed the full Raw in the DNG. They'll have to spend some extra money on storage capacity as well then.

Cheers,
Bart
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 01:59:33 pm
OK guys. End of personal attacks.

On topic please, or it's closed.

BTW, one major university with a substantial collection of contemporary photography told me about a year ago that they were not accepting raw files from photographers work that they were collecting unless they were in DNG. Reason? They feared the inability for future generations to be able read the data.

They have a program in place to migrate over time to new storage formats (that's a given) but they believed (rightly or wrongly – and I think rightly) that they didn't believe and trust in proprietary formats for archival and historical storage purposes. They also told me that a national consortium of museum conservators were planning on making DNG their defacto standard across institutions.

So, my personal observation is... where's the beaf?



No beef.  Good factual information.  I understand their reasons for it.  Good thing, that for their uses, that DNG is available.

However, that is not the crux of the disagreement.  The major disagreement, in my mind, is that we are all expected to form a consensus that DNG needs to be integrated into all cameras.  This is without DNG, yet, being an industry standard and without any understanding, or willingness to discuss, the possible ramifications to the camera manufacturers.

If one does not agree or is not silent, they are summarily attacked, without any regard for reasoned arguments by the person not agreeing.  The "by-words" are that by not agreeing we are giving the manufacturers a "pass" and that "if not a part of the solution, you are a part of the problem".

I am willing to agree to disagree.  However, would request that, if the issue is promoted again, that we can retain the right to again present our views.

Your forum...your call.

John.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: PeterAit on April 21, 2013, 02:03:06 pm
Beef.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 21, 2013, 02:14:58 pm
We also should realize that much (most?) of the turmoil is caused by companies like Adobe who, for commercial reasons, force people to update the image editing software, because newer camera models are deliberately locked out. It is therefore no surprise that Adobe also offers a 'DNG solution' (for an issue they partially create themselves), which would benefit them by not having to interpret multiple Raw formats.

Your statement "for commercial reasons, force people to update the image editing software" is not correct...while there may indeed be a component of "commercial reasons" in Adobe's policy, there are substantial technical reasons as well. Aside from the Adobe policy of not updating software they no longer sell, the way software code is developed forces that sort of line in the sand. Both Camera Raw and Lightroom are software platforms that have substantial system level dependancies...

Camera Raw is a plug-in that lives inside the Photoshop plug-in SDK. Each version of Photoshop has it's own SDK that is dependent on the OS level APIs and services. Trying to maintain backwards compatibility of current and future ACR versions inside of past versions of Photoshop would require branching the code into current OS system support and previous OS system support. For Windows, that might not be an onerous burden, but for Mac it would be almost impossible because of fundamental and massive changes Apple has forced on developers over the years. Camera Raw first showed up in Photoshop 7. It would be impossible to port the current ACR to be able to work in Photoshop 7. In Photoshop CS3, Adobe had to migrate from Codewarrior to Xcode to support Universal Binaries that provided PPC/Intel dual support. In System 10.7, support for Rosetta was removed so nothing on OS 10.7+ can run any PPC code.

Just how would you suggest Adobe update previous versions of Camera Raw for current and future as yet, unreleased cameras? Reverse engineer them and go back to that original code and rewrite them to include these new cameras? How many versions of Photoshop and ACR should Adobe offer backwards compatibility for? One version? Three? How many code branches of Camera Raw should Adobe spawn off?

Looking at Lightroom, if Adobe wanted to offer backwards compatibility for previous versions of LR to open new and future cameras, they would have to maintain code branches for each and every version of LR that needed backwards compatibilities...

So, Thomas developed DNG as a method of providing backwards compatibility of new and future raw files for past versions of Camera Raw and Photoshop. With the proper DNG version preferences, users of previous version of ACR and LR can access and convert new raw files for backwards compatibilities. A user that has a camera only supported by ACR 7.4 can use DNG Converter 7.4 to make a compatible DNG that will open using ACR 2.4 in Photoshop CS or LR 1. Oh, and DNG Converter is free...

So, there are indeed real technical issues and yes, Adobe adopted the policy of updating only currently shipping application versions. Yes, there were financial reasons for that policy, but also valid and substantial technical reasons. If Adobe were to adopt a policy of backwards compatibility updates to previous versions of ACR/LR, it would take developmental resources away from current and new development. So, in effect, Adobe would deprive current/future customers for the benefit of previous customers. As a current customer, I would have a problem with that.

Interestingly, when you access a native raw in ACR or LR, the initial thing the ACR pipeline does is convert the native raw data to DNG data on the fly. I don't think this is a secret as I think Eric or Thomas has mentioned this before (if it wasn't, I'm sorry Eric and Thomas :~). The conversion of native data to DNG data is fundamental to ACR/LR's ability to parse the raw image data and associated metadata such as white point, ISO, camera data and lens data. So, it is somewhat ironic that some people who don't want to use DNG actually are using DNG every time they process their natives raws in ACR/LR. It's kinda yet another feather in the cap of Thomas Knoll.

So, no, I don't think it's proper to lay the blame for the current situation regarding undocumented, proprietary raw file formats the feet of Adobe (and other 3rd parties). The blame falls at the feet of the camera makers who eschew the use of any standardized raw file format and continue to propagate new raw file formats each time they release a new camera. And the situation will not improve until the camera companies quit doing that.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bart_van_der_Wolf on April 21, 2013, 03:27:14 pm
Your statement "for commercial reasons, force people to update the image editing software" is not correct...while there may indeed be a component of "commercial reasons" in Adobe's policy, there are substantial technical reasons as well. Aside from the Adobe policy of not updating software they no longer sell, the way software code is developed forces that sort of line in the sand. Both Camera Raw and Lightroom are software platforms that have substantial system level dependancies...

Hi Jeff,

Of course they do, and they hopefully exploit them, and the Camera makers also do not want to maintain deprecated legacy data in new Raw formats, just for the benefit of others.

Quote
If Adobe were to adopt a policy of backwards compatibility updates to previous versions of ACR/LR, it would take developmental resources away from current and new development. So, in effect, Adobe would deprive current/future customers for the benefit of previous customers.

Quote
Interestingly, when you access a native raw in ACR or LR, the initial thing the ACR pipeline does is convert the native raw data to DNG data on the fly.
Quote
I don't think this is a secret as I think Eric or Thomas has mentioned this before (if it wasn't, I'm sorry Eric and Thomas :~). The conversion of native data to DNG data is fundamental to ACR/LR's ability to parse the raw image data and associated metadata such as white point, ISO, camera data and lens data. So, it is somewhat ironic that some people who don't want to use DNG actually are using DNG every time they process their natives raws in ACR/LR.

Which begs the question why a separate DNG conversion does support certain camera models, and a direct attempt to open them (via DNG) doesn't.

Quote
So, no, I don't think it's proper to lay the blame for the current situation regarding undocumented, proprietary raw file formats the feet of Adobe (and other 3rd parties).

Who's blaming them for that? They have no control over what the camera manufacturers need.

Quote
The blame falls at the feet of the camera makers who eschew the use of any standardized raw file format and continue to propagate new raw file formats each time they release a new camera.

You make it sound as if it's a goal, to change the Raw data that is beneficial to recreate the original signal. Could it perhaps be because technology advances and they need to adust the postprocessing possibilities to keep up?

Quote
And the situation will not improve until the camera companies quit doing that.

I hope they keep improving their own technology, in competition with others, despite the wish from some that they reach some sort of an agreement with their competitors about what to store and how. Patents make that a bit difficult.

Cheers,
Bart
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Fine_Art on April 21, 2013, 03:42:40 pm
I agree with Jeff, the primary responsibility for the problems is with camera manufacturers. They want to lock people to their systems. Their primary goal is to make money, not to give you what you want/need.

DNG is useful. I have no issue with it's versions considering it is free. "Never look a gift horse in the mouth."

It would be nice if the manufacturers gave the user the option of output formats. Raw, DNG, TIFF, FITS. I would go out of my way for a FITS capable camera.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 21, 2013, 03:43:26 pm
Which begs the question why a separate DNG conversion does support certain camera models, and a direct attempt to open them (via DNG) doesn't.

Not sure what you mean...the ACR pipeline (which includes ACR, LR & DNG Converter) all share the same native raw support given the same processing pipeline version. Where have you ever seen ACR/LR fail to open a raw file that DNG could, or visa versa at a given version?

Quote
Who's blaming them for that? They have no control over what the camera manufacturers need.

You were when you said: "We also should realize that much (most?) of the turmoil is caused by companies like Adobe who, for commercial reasons, force people to update the image editing software, because newer camera models are deliberately locked out."

I sure read that as you saying the turmoil was the fault of Adobe (and other 3rd parties). And "newer camera models are deliberately locked out" is also incorrect...nobody is being directly locked out of anything except users of new cameras not being able to open new cameras until software has been updated...they've been locked out by the camera makers not using standardized, compatible raw file formats. Unless you are saying that it's the camera makers deliberately locking out their users–which I actually agree with :~)
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Fine_Art on April 21, 2013, 03:52:16 pm
Logical fallacy. Post hoc ergo propter hoc

I used to have software that opened Sony A350 files. When I got the A55 the files were scrambled. Blame the software? no.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Rhossydd on April 21, 2013, 04:59:55 pm
I have no fact...only suspicions....if someone really knows, we would all be made wiser by it.
....
Forgetting, for the moment, the internals to the camera.  Canon provides RAW conversion software, which they make no money on.  However, even though they know the internals of RAW changes way before the rest of the world, they need to incorporate those changes into their code...which drives an expense, which will effect the overall cost, therefore the price, of the camera.  If this cost could be saved without causing increased cost or other issues you can be sure the Canon financial people would on this like flies on stink.
Well it makes no sense to me (in the west).

Jeff Schewe's posts in the previous LR thread have given us some insight into how the Japanese camera manufacturers approach this issue and how Adobe have dealt with the situation.

Ultimately a single, good RAW file format standard would benefit everyone. I've not yet read any credible argument that disagrees with that position.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bart_van_der_Wolf on April 21, 2013, 05:02:37 pm
Not sure what you mean...the ACR pipeline (which includes ACR, LR & DNG Converter) all share the same native raw support given the same processing pipeline version. Where have you ever seen ACR/LR fail to open a raw file that DNG could, or visa versa at a given version?

Then what's the problem, Jeff? Is there a problem opening the (partly proprietary) file data, or isn't there? If there isn't, then why the need for conversion to DNG, and the call for Camera manufacturers to join in a unified file format?

Quote
You were when you said: "We also should realize that much (most?) of the turmoil is caused by companies like Adobe who, for commercial reasons, force people to update the image editing software, because newer camera models are deliberately locked out."

Yes, there is a group of people who blame (=turmoil) the Camera manufacturers, because they themselves would like to have an easier job, which is understandable. The camera manufacturers get no benefit in that game, it would only drive up their cost, slow down development, and create issues with patents. Let's stay realistic, it won't happen that way.

Cheers,
Bart
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bart_van_der_Wolf on April 21, 2013, 05:07:37 pm
Ultimately a single, good RAW file format standard would benefit everyone.

Hi,

Does 'everyone' include the camera makers? Otherwise it will not happen, even if it were possible to avoid patent issues?

Cheers,
Bart
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Rhossydd on April 21, 2013, 05:20:01 pm
Does 'everyone' include the camera makers?
Why wouldn't it ?
Did Canon* get any advantage from making different file formats from the 5d, 5DmkII and 5DmkIII ?

The point here is why change the file format for every camera model ? Does it benefit anyone ?

*For example or insert the camera maker of choice using their own formats for each camera iteration.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bart_van_der_Wolf on April 21, 2013, 06:29:54 pm
Why wouldn't it ?
Did Canon* get any advantage from making different file formats from the 5d, 5DmkII and 5DmkIII ?

The point here is why change the file format for every camera model ? Does it benefit anyone ?

Hi,

First of all, it's not an entirely new file format. It's still basically a Bayer CFA sensel arrangement, with a CR2 filename extension.

There were technological advances from 12.8 Megapixels with 12-bit precision, at 3 frames per second, to 21.1 MP and 22.3MP  with 14-bit precision, at a 3.9 and 6 frame rate per second readout speed which required more/multiple (parallel) readout channels which also require calibration data to be recorded. Video capture was added, which probably required more thermal calibration info to be recorded as well. It only seems logical that some changes were required in the Raw data that was recorded.

It seems to me that many users did benefit from those changes (directly in image quality, and later in competitive offerings), and it also urged the Raw converter producers to optimize their software (by utilizing the capacity of the graphic-card's GPUs).

Also, the addition of more lens and dust related metadata in the EXIF maker notes datasection, potentially allows for more targeted distortion and vignetting correction (possibly in the masked Raw data fields) and dust removal, e.g. in JPEGs and video directly from the camera.

Why people resist technological progress in favor of uniformity, beats my sense of logic.

Cheers,
Bart
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 06:43:10 pm
Does 'everyone' include the camera makers? Otherwise it will not happen, even if it were possible to avoid patent issues?

Hi Bart

What patent issues?
There are already use-cases where camera-makers have adopted DNG as their RAW format.
Also, what patent issues are there with every camera been able to generate a JPEG?
The idea is to have an ISO-standardized open RAW format.
So, whether DNG is adopted as a universal RAW format or something else patent issues are the least of the problems.

And yes it does include the camera-makers.

With regard to your comment about technological advancements somehow being stymied by a standardized RAW format that doesn't wash.
A RAW format is a recording medium not a generating medium.
With regard to the use-cases above using DNG in several different proprietary brands of cameras and models clearly they did not feel fettered.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on April 21, 2013, 07:02:02 pm

The point here is why change the file format for every camera model ? Does it benefit anyone ?


It's completly meaninless, but is there a parameter that escapes to our understanding?

If that non-sense is happening for now too long and put users on bondage, there must be a reason.
Who it benefits? I don't know, but it should benefit to some interests.

Now, the fashion here is to archive footage in JPEG2000. I thought that in digital preservation it was "safe" and it apparently
isn't really that much. There are concerns on JPEG2000 viability in the future. New debates are open, on air currently.

When this Raw video is going to be democratized soon and every camera maker will provide combo cams with video+still capabilities in Raw
I'm just about sure that very few will adopt cinema DNG and will see the same non-sense that what's happening in still for decades.
The surreal part, will be how will they deal with their raw dev like RedCineX? and how those developpers will be integrated into pipelines?
It's gona be a huge mess. Nothing will read nothing and DITs will have to invent oscur roundtrippings to make the machinery work.

Who's going to benefit this mess?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bart_van_der_Wolf on April 21, 2013, 07:22:29 pm
Hi Bart

What patent issues?

Hi Tony,

Suppose that Canon would want to actually exercise some of their patents for tri-chromatic capture per photosite. Do you think that they would like to tip off their competitors ahead of the event? Or, if Dr. Eric Fossum (http://en.wikipedia.org/wiki/Eric_Fossum) (the inventor of the CMOS image sensor) produces a new (sub-micron sensel) sensor generation (which he is working on), he would want to allow competitors to jeopardize his investments by speeding up their efforts or counter-measures.

Quote
There are already use-cases where camera-makers have adopted DNG as their RAW format.

Yes, and they serve niche markets.

Quote
Also, what patent issues are there with every camera been able to generate a JPEG?

I'm not sure what you are referring to.

Quote
The idea is to have an ISO-standardized open RAW format.

Yes, I wouldn't mind. But what issue does it solve?

Quote
So, whether DNG is adopted as a universal RAW format or something else patent issues are the least of the problems.

I'll leave it to IP and patent lawyers to pass the ball between them, but I do recall issues around GIF file patents threatening everybody using the internet, and claims around IPIX panorama file formats and panorama photography in general that at the time drove Prof. Helmut Dersch out of providing freely accessible panorama solutions to everybody. That certainly stifled progress in that area of industry.

Quote
With regard to your comment about technological advancements somehow being stymied by a standardized RAW format that doesn't wash.

Why not?

Quote
A RAW format is a recording medium not a generating medium.

Doesn't the data in the Raw file determine the potential output quality?

Quote
With regard to the use-cases above using DNG in several different proprietary brands of cameras and models clearly they did not feel fettered.

To give just one example, I've corresponded with some Pentax 645D users who report significant (quality) differences/limitations between the DNG and PEF files and the various available Raw conversion alternatives/qualities.

Cheers,
Bart
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 08:14:12 pm
Hi Bart

Lets put the issue in perspective.
All the patent issues that you are referring to involve analog to digital signal conversion.
Once that, and all the subsequent number crunching is done, then the finished product is recorded as a RAW file of some description.
There is no need for Canon, or anyone else for that matter, to show their cards.
The recording medium per se does NOT have to be unique for Canon files to still show unique image characteristics, such as they are, or will be.

If these apparently unique RAW file formats can now be converted to DNG right now in Lightroom with no loss of quality what would be the problem with this happening in camera?

I am aware of the interesting differences in how Canon RAW converter handles .CR2 files compared to Lightroom or ACR, and while the differences may be partially due to secret metadata instructions in the RAW files that Adobe is not privy to, subjectively I question the benefits from an aesthetic perspective.

As for you querying the benefits of a a standardized open RAW format, if by now you don't see it perhaps you never will.
Why do we bother having JPEGS and TIFFS?
According to your reasoning we should get along quite well with everyone producing their own unique output formats.

Actually, Bart, I do detect an understandable scepticism about whether an open RAW format will ever come to pass. I understand that because I share that concern. However, that does not detract from the importance of tackling the issue.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 21, 2013, 08:45:42 pm
The camera manufacturers get no benefit in that game, it would only drive up their cost, slow down development, and create issues with patents. Let's stay realistic, it won't happen that way.


Let's just be clear here...there is no know issue with patents as it relates to files written to media. Exactly what patents are you referring to or are you just imagining there are patent problems?

As for camera makers getting no benefit, actually, that's not really true...each and every new camera requires additional work for the camera makers to update their own software. That could be mitigated of they adopted a standard because new cameras would enjoy auto support. Yes, they still need to do the work specifying the metadata and the sensor responses–which they have to do anyway...you are overstating the amount of work that would be required...why are you doing that? Simply to be devil's advocate? Or do you know something I don't know?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 21, 2013, 08:51:26 pm
There were technological advances from 12.8 Megapixels with 12-bit precision, at 3 frames per second, to 21.1 MP and 22.3MP  with 14-bit precision, at a 3.9 and 6 frame rate per second readout speed which required more/multiple (parallel) readout channels which also require calibration data to be recorded. Video capture was added, which probably required more thermal calibration info to be recorded as well. It only seems logical that some changes were required in the Raw data that was recorded.

Uh, ok...so some changes are recorded as metadata in the raw file...so? What's to say that data can't be recored in a meaningful instead of undocumented, proprietary method? Do you think there is intellectual property in proprietary raw files? I don't...(and neither do the guys I know who decode them). The big secret is there are no secrets...

now, if you are talking about analog>digital converters and the onboard DSP chips, yes, that is big time, top secret tell nobody secret. But once that data is written to media, no so much.

Come on Bart, I'm expecting better of you...you should know this stuff :~)
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Chris Kern on April 21, 2013, 09:02:33 pm
I've been following this thread with a sense, as Yogi Berra famously put it, of “déjà vu all over again.”

Long ago, when the Internet (then called by other names) was in its infancy, there were a bunch of competing electronic mail storage formats and retrieval protocols.

Each manufacturer claimed its approach was superior to the others.  Each had customers who were attached, or at least accustomed to, a particular manufacturer’s product.

You could exchange messages between one system and another through the manufacturers’ proprietary electronic mail gateways, assuming the manufacturer provided the appropriate conversion software and the current version of it worked properly, but if you changed messaging service products, it was essentially impossible to maintain access to your message archives unless you had squirreled away a copy of the old software and still had the appropriate platform to run it on.

This fragmentation of file formats and protocols seemed to work for the manufacturers for a while.  It created a significant barrier to customer migration from one product to another.  On the other hand, it also meant that a manufacturer which developed a superior product faced a significant barrier to increasing its market share: users of inferior products were reluctant to make the switch and orphan their message archives.

Both the manufacturers and the customers were mostly stuck in a rut until a software developer with no financial stake in the status quo  (the late Mark Crispin) published a messaging service standard that incorporated essentially all the features of the various proprietary products, the Internet Message Access Protocol (IMAP), and the university which employed him (the University of Washington) made available to the public, at no cost, a software suite to implement it.

It still took quite a while for IMAP to be universally adopted.  Not so much because the commercial purveyors of proprietary products felt disadvantaged by it—they remained and still are free to offer competing user agents with different looks and feels in an attempt to keep existing customers and appeal to new ones—but because ... well ... it's bureaucratically difficult for many companies to change the assumptions that underlie their products, even when the technical obstacles to doing so are modest.

What finally did the trick was market demand.  Mostly from large corporate and governmental customers, which didn’t want to be locked into a particular messaging product and began migrating to ones that conformed to the superset international standard.

This is where the historical analogy begins to break down, I'm afraid, because there isn't the kind of corporate customer presence in photography that there is in enterprise IT.  Although the government archivists really ought to be making more of a stink about the lack of standardization than they seem to be.  However, I suspect a sufficient grass-roots groundswell (I don’t think that’s a mixed metaphor) from many individual photographers eventually will bring the camera manufacturers around.

Funny thing is, it actually would be to their benefit to coalesce their raw formats around a public standard.  It must cost a lot of money to maintain the different formats and I never heard of anyone who claimed to have purchased a particular camera because he preferred NEF to CR2, or vice-versa.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 21, 2013, 10:53:22 pm
Yes there are plenty of stories in the world of electronics that have a similar ring.

In our community or industry - photographic - there is a lot of misinformation and confusion about what things mean and what they don't.
Currently as any reading of the threads dealing with this issue show it is clear that there is no consensus that there is a problem, much less what should be done about it.
If there are any stakeholders out there who have a vested interest in maintaining the status quo they must be well satisfied.
Even on a forum such as this, apparently populated by many well informed individuals, we appear unable to even agree that the sky is blue!

Perhaps, despite ourselves, useful change may occur, but I doubt that it will.
We, the consumers are generating a lot of heat, but only by random Brownian motion right now, not a hint of cohesive movement.
Maybe Michael (Reichman) will be able to achieve something with a word in a strategic ear but if he is expecting the backing of the forum members as a whole it seems he must be disappointed right now.

I for one will maintain a consistent stand on this.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 11:36:07 pm
Chris, nice review of the historical beginnings of mail on the internet.

I think your last paragraph sums up where this situation is quite different, where you say, "...I never heard of anyone who claimed to have purchased a particular camera because he preferred NEF to CR2, or vice-versa."

If the end users were only able to use the camera manufacturers RAW conversion software, the email analogy would be quite close.

It is Open source and 3rd party RAW conversion software availability which makes it different.  End users do not need to disrupt their workflow to change from one RAW format to another.  It is also, as dpBestFlow says, the reason why the risk of inaccessibility (loss of reading RAW images) is significantly less.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 21, 2013, 11:53:01 pm
As for camera makers getting no benefit, actually, that's not really true...each and every new camera requires additional work for the camera makers to update their own software. That could be mitigated of they adopted a standard because new cameras would enjoy auto support. Yes, they still need to do the work specifying the metadata and the sensor responses–which they have to do anyway...you are overstating the amount of work that would be required...why are you doing that? Simply to be devil's advocate? Or do you know something I don't know?

You are absolutely correct.  Changes to the RAW format must drive cost into the camera maker's RAW conversion software.  Manufacturers such as Canon do not charge for their software, so that cost needs to be recovered in the product cost, which drives up the cost.

If Adobe had a situation like this, where sloppiness or poor planning on one part of the development team for a product caused increase cost in another area, thus driving overall cost for the product under development, wouldn't they fix it?  Particularly if this problem reoccurred again and again with each new release  of the product?

So, now back to Canon.  Do you think their financial people are asleep at the switch?  Why do they keep repeating this over and over again, driving cost from their RAW software?  You are smart....if you are honest, you would admit that the logical conclusion must be that there are other factors involved, probably not driven by costs, but resulting in cost, where it is more costly to produce the same RAW format for each camera and safe the cost of updating their RAW software.

If the logic is wrong...where?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Rhossydd on April 22, 2013, 04:02:07 am
First of all, it's not an entirely new file format.
But different enough that third party software won't read them, so in effect totally different to the end user.
Quote
There were technological advances from 12.8 ...... It only seems logical that some changes were required in the Raw data that was recorded.
The issue is providing a standard file format that allows a degree of future proofing.
Quote
Why people resist technological progress in favor of uniformity, beats my sense of logic.
From what others have written on this I've yet to see that end users have actually seen any benefit from propriety file formats.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on April 22, 2013, 05:05:50 am
The progress in favor of uniformity
Is evident and necessary.
No, not necesary, urgent.

Taking a simple example that can be
Extrapolate to still: I'm interested
in this pocket BM camera
That uses Cinema DNG and supposed
To be the attempt of RAW standart in
Motion imagery. Fine so far.

But then, the workflow has to go through
Resolve. It's not that Resolve is bad, it's
That I don't like it and I have other system
I paied for. If you are on smoke2013 for
Ex, Cinema DNG isn't supported. Media Composer?
Not supported. Etc etc...so first, I'm on
Bondage with some software: no choice.
2) metadatas. So if we got standardized metadatas
Protocol, the way we can apply the workflow
Will be extremely more simplify and user friendly,
And the archiving preservation ] guarantee.
So if I buy this camera, not only I buy the
Camera, but a specific workflow that gives
No choice to the user. Then, if I mix with Red footage,
I have to apply also another Type of workflow
Within the same project, wich isn't rare.
The result of this proprietary mess is an over
Complicated workflow, issues within roundtrippings,
More time spent in order to acheive tasks,
No support during some time each time a
New format is released etc etc...

WHO wants that?

Progress has always been about simplifying.

Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Vladimirovich on April 22, 2013, 03:15:07 pm
Uh, ok...so some changes are recorded as metadata in the raw file...so? What's to say that data can't be recored in a meaningful instead of undocumented, proprietary method? Do you think there is intellectual property in proprietary raw files? I don't...(and neither do the guys I know who decode them). The big secret is there are no secrets...

so the push really shall be just for documenting of the information... which can be perfectly achieved w/o DNG at all.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: gerryrobinson on April 22, 2013, 03:15:48 pm
 Nothing more sinister than trying to keep the message out, that I think adopting a standard format, as an option, just makes more sense, in the long term, then to keep generating new generations of ORFs,CR2s,NEFs etc.
I'm a bit under-informed as the better method of doing that, short of writing to the camera companies directly, In my case I'll be sending a note off to Canon. If others feel the same as I do maybe that's what we need to do.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Vladimirovich on April 22, 2013, 03:20:19 pm
Let's just be clear here...there is no know issue with patents as it relates to files written to media. Exactly what patents are you referring to or are you just imagining there are patent problems?

As for camera makers getting no benefit, actually, that's not really true...each and every new camera requires additional work for the camera makers to update their own software.

That could be mitigated of they adopted a standard because new cameras would enjoy auto support. Yes, they still need to do the work specifying the metadata and the sensor responses–which they have to do anyway...you are overstating the amount of work that would be required...why are you doing that? Simply to be devil's advocate? Or do you know something I don't know?

and where did you the exact amount of work required which is different from creating/testing camera profiles, adding/testing new features to that software, etc...  plus as you perfectly know (and keep silent) rarely new format features are added to the format itself, so you argument is lame
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 22, 2013, 03:41:58 pm
and where did you the exact amount of work required which is different from creating/testing camera profiles, adding/testing new features to that software, etc...  plus as you perfectly know (and keep silent) rarely new format features are added to the format itself, so you argument is lame

Huh? Do you mean where did I learn about how much work is required by the camera companies to add new camera raw files? Is that what you are asking? If so, I've talked with engineers at Phase One and Leaf...and adding support for new cameras or backs takes a lot of work, but it's work they would have to do regardless of whether or not they do/don't add DNG support. Taking the sensor and lens data and putting that into a usable form for DNG would not be much extra work. Some, but not much over what they have to do anyway.

As for the lame argument comment, again, I'm not sure what you are talking about...are you referring to new features of proprietary raw file formats?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 22, 2013, 05:13:24 pm
Huh? Do you mean where did I learn about how much work is required by the camera companies to add new camera raw files? Is that what you are asking? If so, I've talked with engineers at Phase One and Leaf...and adding support for new cameras or backs takes a lot of work, but it's work they would have to do regardless of whether or not they do/don't add DNG support. Taking the sensor and lens data and putting that into a usable form for DNG would not be much extra work. Some, but not much over what they have to do anyway.

As for the lame argument comment, again, I'm not sure what you are talking about...are you referring to new features of proprietary raw file formats?

But it is addtional work.  How do you define a little?  Did these engineers just look at their effort or did someone cost out the total effect of their work on DNG, which is much more that just writing the code and unit testing it.

Was this DHG work replacing their native RAW or side by side?  Do they have to contend with timing factors to support 10 or 12 fps?  Or other similar factors?

Your offhand comments, while I am sure factually relating the conversations you had, are quite typical of one who has not brought an entire complex hardware/software product to market.  Concept is easy, execution and the effect on the pricing and P&L can be significant.  With your experience you should know it.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: PeterAit on April 22, 2013, 06:06:41 pm
Time to call the euthanasia people for this thread.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 22, 2013, 07:34:15 pm
Uh, ok...so some changes are recorded as metadata in the raw file...so? What's to say that data can't be recored in a meaningful instead of undocumented, proprietary method? Do you think there is intellectual property in proprietary raw files? I don't...(and neither do the guys I know who decode them). The big secret is there are no secrets...


If there are no secrets, there are no problems.  Just work effort for people who want to be in the business of decoding/reverse engineering the RAW data.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 22, 2013, 08:21:35 pm
Time to call the euthanasia people for this thread.

Well, you are under no obligation to keep reading and posting...but some useful discussion and new information has come out of the thread. So I agree with Mike, I wouldn't kill it off prematurely. I'm just not going to respond to certain posts and poster. You could consider exercising the same discipline.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 22, 2013, 08:44:27 pm
Well, you are under no obligation to keep reading and posting...but some useful discussion and new information has come out of the thread. So I agree with Mike, I wouldn't kill it off prematurely. I'm just not going to respond to certain posts and poster. You could consider exercising the same discipline.

So....you think everyone is on your side.   :D

I find it interesting that you choose not to respond to posts which poke holes in you ill thought out logic.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: rasterdogs on April 22, 2013, 11:28:33 pm
Well, you are under no obligation to keep reading and posting...but some useful discussion and new information has come out of the thread. So I agree with Mike, I wouldn't kill it off prematurely. I'm just not going to respond to certain posts and poster. You could consider exercising the same discipline.
This is a good point but seems to have little impact on certain parties who seem intent on argument for the sake of argument.  And you were correct on the other thread on this topic, pigs can fly.
I continue to doubt that they can be taught to sing.   :o
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 23, 2013, 12:10:28 am
This is a good point but seems to have little impact on certain parties who seem intent on argument for the sake of argument.  And you were correct on the other thread on this topic, pigs can fly.
I continue to doubt that they can be taught to sing.   :o

Gosh....I guess some people cannot understand and/or honor Michael's request....

BTW...if you have not noticed, the disagreement (argument, in your terms) goes both ways....with more than one person on each side.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: rasterdogs on April 23, 2013, 12:24:03 am
Gosh....I guess some people cannot understand and/or honor Michael's request....

BTW...if you have not noticed, the disagreement (argument, in your terms) goes both ways....with more than one person on each side.
Seriously? I'm betting that you will have the last word here.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 23, 2013, 12:42:02 am
Seriously? I'm betting that you will have the last word here.

If you have value to add to the discussion, feel free to post.  If all you have is insults, please find a playground.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on April 23, 2013, 03:56:12 am
Mmmm...

It smells more and more like
Tópic's closed.

So before it does, I put My final
Words:
Should camera companies adopt
A non proprietary RAW format?
Yes!
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: peterv on April 23, 2013, 10:50:45 am
No, this thread should not be closed, it should be moderated.

As a regular reader to this forum, I find this an interesting subject and I'd like to hear sensible reasoning that lead to a conclusion from which we as photograpers can all benefit in the future.

So far, it hasn't been made very clear why DNG would not be a good idea.

Please no name calling, just sane discussion.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 23, 2013, 11:10:54 am
No, this thread should not be closed, it should be moderated.

As a regular reader to this forum, I find this an interesting subject and I'd like to hear sensible reasoning that lead to a conclusion from which we as photograpers can all benefit in the future.

So far, it hasn't been made very clear why DNG would not be a good idea.

Please no name calling, just sane discussion.

Peterv, I agree.

I have also not been against the CONCEPT of DNG, or other open RAW standard.  I think it would be a good idea.

I have been disagreeing that the camera manufacturers are malicious, or whatever, not falling in line.

It is not "giving them a pass" to attempt to understand their side of the equation.  How does it effect them?  What does it cost them to implement and/or maintain it.  Is it total conversion or side by side versions of RAW from them.  What if any competitive exposures would this create for them?  Will the "standard" constrain their ability to innovate in the future?

These considerations are belittled by (most of) the DNG proponents.  Unfortunately, without understanding them and finding a value proposition that solves them, I do not believe will ever be overcome by the handwringing which is going on.  All business, including Adobe, are driven by P&L.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 23, 2013, 01:44:26 pm
So far, it hasn't been made very clear why DNG would not be a good idea.

Yep...it isn't clear why DNG would not be a good idea because, well, DNG is a good idea thought up by one of the bright minds of digital imaging–Thomas Knoll who basically jump started this whole industry by writing Photoshop, then Camera Raw and DNG. Thomas is kinda an expert on these matters, having decoded over 300 different cameras raw file formats.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: jrsforums on April 23, 2013, 03:03:05 pm
Yep...it isn't clear why DNG would not be a good idea because, well, DNG is a good idea thought up by one of the bright minds of digital imaging–Thomas Knoll who basically jump started this whole industry by writing Photoshop, then Camera Raw and DNG. Thomas is kinda an expert on these matters, having decoded over 300 different cameras raw file formats.

Why do you keep repeating points that no one has an issue with?

DNG could/would be good...for many.  Is it critical?  Is it best for the manufacturers?  who covers the cost for some to implement, that which is a cost savings for others?  How does it really effect the average photographer vs. status quo?  Those are the points we have been discussing.

Also, no one questions Thomas Knoll's knowledge or contributions to this industry.  

If nothing else, DNG seems to build internal efficiencies in ACR/LR processing, currently and with LR Smart previews in LR5.  The good news is that this is under the covers, not requiring any end user to implement or convert, which might cause users to seek out a different solution.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: peterv on April 23, 2013, 04:09:26 pm
who covers the cost for some to implement, that which is a cost savings for others?

I think this is the second time you bring up the aspect of the costs of reverse engineering the proprietary raw files. In one of their videos Michael and Jeff talk about the fact that the ACR team is actually quite small. I can't believe Adobe makes a case for a universal DNG in order to save on the salaries of a few good man.

The same goes for the camera makers. The costs of the man-hours for making DNG available are not relevant to the total costs of marketing a new model.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on April 23, 2013, 04:39:51 pm
It's very possible that softwares like Lightroom, not in a so long future (could be very short), are going to be a raw dev plus a file manager for both still and motion imagery.
DNG is the only Raw possibility to date that can cover the needs to come in imagery very very soon. (that are already there but still in infancy)

They will have to do it yes or yes. The only question is when.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 23, 2013, 04:46:50 pm
The same goes for the camera makers. The costs of the man-hours for making DNG available are not relevant to the total costs of marketing a new model.

Yep, the issue is not related to costs, nor to technical limitations of standardizations, not really–any short term technical issues can be dealt with and the cost of adding a standard raw file format would be a drop in the bucket for the development of the chips and cameras.

It's all about the politics...Nikon and Canon simply don't want to adopt any standards at this time. However, I hope that over time and with grassroots support, one of the two main players might blink and either adopt a standard format or make a standardized raw file format an option like JPEG already is–in fact, the Lossy DNG is a potential replacement for JPEG that would offer the potential file savings of JPEG while making editing of the scene referred image data far more flexible.

Once that happens, it will be a tipping point for the industry. And while they are a niche "rich man's camera" one very successful camera maker is doing just that. Leica has successfully adopted DNG as their native raw file format. And Leica is doing great. They literally can sell avery camera/lens they make and have backorders for lots of their stuff. They enjoy the benefits of DNG which includes automatic support for new cameras on current and older software platforms as well as pushing the image quality as far as possible by including lens profiles to address aberrations. Leica is a shining example of how DNG can benefit a camera maker. So, if Leica can afford to adopt DNG (since they are such a small company) one presumes there should be no economic roadblock to Nikon or Canon doing so. No, it ain't about the money...
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: davidgp on April 23, 2013, 05:55:27 pm
So, I have been reading this topic, maybe a bit heated, but I think I got some pieces of information about how Adobe or other software companies works... nice information to know (thanks!!!).

My two cents. My background is more of a software programer than a photographer, basically an amateur photographer that earns its living working 8 hours a day designing a programming software (no Image processing software of any short to leave that clear).

As a programmer, for me that the camera companies adopt or not DNG is not a matter of cost, for them it must be less of 1% of the work they employ to write the software, and as difficult, again for them, than to add the new RAW format the camera engeniers decide to adopt for the new Canon 1D Marl XX or the new Nikon D900e, I mean, they just get a full specification of a file that need to read, translate to the internal format the program uses (like Adobe does to DNG), and the rest of the code of their RAW software its the same. They don't write an entire pipeline of RAW processing for each new RAW format their company produces. Well, at least in my mind, that will be a very inefficient way to work.

I'm more of the opinion that this is a political reason as Jeff Schewe says than a software problem. The real cost is to convert the data coming from the sensor into RAW data, and that happens in the camera, then if the camera stores it in DNG, NEF, or whatever... it is just a very little piece of the puzzle.

Other topic that I read in this thread and in the previous one in the Lightroom sub-forum is the argument that the camera vendors give their software for free. Really? I thought I paid for that when I bought the camera, in the price of the camera you are not paying only for the little pieces of metal, plastic and so on... and the people that worked in the factory building it. You are paying for the design of the camera, paying for the people that wrote the warranty information, paying for the people that wrote the manual of the camera, and paying for the software that comes with it, everything adds... after all, they are giving you software that only works with their cameras, it is not useful for anybody RAW files.

(well, there are some exceptions, some camera makers from time to time add software that can work with other cameras, if memory does not fail, I think Leica used to bundle a copy of Lightroom with some of its cameras, not sure if Phase One gives you a copy of Capture One with their backs (they are over my pay-rate so I don't look to much to them)... but anyway, the price of that software it is included in the Leica camera or in the Phase One back...).
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 23, 2013, 06:59:43 pm
(well, there are some exceptions, some camera makers from time to time add software that can work with other cameras, if memory does not file, I think Leica used to bundle a copy of Lightroom with some of its cameras, not sure if Phase One gives you a copy of Capture One with their backs (they are over my pay-rate so I don't look to much to them)... but anyway, the price of that software it is included in the Leica camera or in the Phase One back...).

Yes, when you buy that expensive Leica, you get a copy of the then shipping Lightroom (upgrades are the responsibility of the user). And Phase One (and Hasselbad) "gives you" the software to drive the camera and process raw images from the back. In the case of Capture One, you get the digital back (DB) version of Capture One, not Capture One Pro which also processes other supported cameras like DSLRs. Not sure about Hasselbad's Focus software, but I know their software will process any raw capture from their backs.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 24, 2013, 12:02:15 am
Yep, the issue is not related to costs, nor to technical limitations of standardizations, not really– any short term technical issues can be dealt with and the cost of adding a standard raw file format would be a drop in the bucket for the development of the chips and cameras.

It's all about the politics...Nikon and Canon simply don't want to adopt any standards at this time. However, I hope that over time and with grassroots support, one of the two main players might blink and either adopt a standard format or make a standardized raw file format an option like JPEG already is–in fact, the Lossy DNG is a potential replacement for JPEG that would offer the potential file savings of JPEG while making editing of the scene referred image data far more flexible.

Once that happens, it will be a tipping point for the industry. And while they are a niche "rich man's camera" one very successful camera maker is doing just that. Leica has successfully adopted DNG as their native raw file format. And Leica is doing great. They literally can sell avery camera/lens they make and have backorders for lots of their stuff. They enjoy the benefits of DNG which includes automatic support for new cameras on current and older software platforms as well as pushing the image quality as far as possible by including lens profiles to address aberrations. Leica is a shining example of how DNG can benefit a camera maker. So, if Leica can afford to adopt DNG (since they are such a small company) one presumes there should be no economic roadblock to Nikon or Canon doing so. No, it ain't about the money...

This is true.
A simple reading of the DNG specification, last updated in December 2012, will rebut many of the fallacies currently been pushed.

Claiming that adopting DNG, or perhaps some other similar format, as a universal RAW standard somehow 'limits' technical advances on the hand of the camera-makers simply does not stack up.
As a principle it is a bit like claiming that the CD or Blue-ray disc that one buys limits the creativity of musicians, mixing specialists and music producers in the studio.
No-one even thinks about that as an issue per se.
Is it possible to change how that music sounds when we listen to it - sure.
Both different players (and the technicalities of the amplifiers and speakers may make adifference) as well your tastes with regard to treble and bass etc.
In principle, a/d conversion of light in the camera and the subsequent number crunching is what happens in the 'studio'.
The finished product on the CD or Blue-ray disc is the RAW file written to a particular format.
Finally, the player and the sound we eventually hear is analogous to the combination of our RAW conversion and our tastes.

Truly understanding what a RAW file format actually does as well as the TRUE similarities as well as differences between proprietary RAW formats and DNG will immediately squash any claims that DNG stifles the creativity of camera-makes in order to give us better cameras (read sensors and processors in the immediate context).
The DNG specification is published and freely available from Adobe's website.
It is an interesting read.
My reading of the specification, anyway, most emphatically confirms that DNG has space in the metadata component for proprietary information should it be required. (At least one post tried to take me to task on that one.) The key is that propurietary information is not placed in other areas of metadata not designed for the purpose (a bad habit especially prevalent in, now, more dated proprietary formats).
The entire concept of DNG is designed to be extensible, not only in volume, but also in technology.
Also, if one is prepared to acknowledge this, the very fact that the specification has been updated several times to take into consideration potential, or actual, technological advances, shows that not only the attitude of the designers, but the actual format design - it is there to be changed and updated as required.

Additionally, Adobe has always, as far as their published intent goes anyway, expressed a desire to make sure that the format be kept current, but in reality the plan has always been for the International Standards Organisation to take on the format, in which case the prime responsibility would not be Adobe's even if their involvement did persist.

There are no proprietary RAW formats currently in production that cannot be accurately and fully documented in a DNG file.
In fact most proprietary RAW formats follow the fundamental outline of the DNG design (which uses the TIFF-EP specification as its basis).
Most proprietary RAW formats have also adopted TIFF-EP as the basis of the format - and this appears to have only occurred since the release of the first iteration of the DNG specification.

Mentioning the fact that DNG has already been adopted by several manufacturers has stimulated comments along the line that images files from these models are crap and the implication is that DNG somehow is the limiting factor.
I was pleased that Jeff mentioned Leica.
Are there any Leica users out there using models where DNG is the RAW format generated who feel that the image files are crap, and are they prepared to finger DNG as the culprit, and what is the evidence then that DNG might be the problem?
Lets see if anyone is prepared to stick their hand up on this one.
Somebody mentioned that at least one Pentax model using DNG has crap RAW files. Maybe you are correct in your assessment of the file quality. What evidence do you have that DNG is to blame?

Currently, RAW file conversion, includes a demosaicing process, omongst others, that requires information about the sensor and A/D conversion. The demosaicing process is there to give us the colour that we see, sensel for sensel, that is not present in the actual recorded information, sensel for sensel, since sensels only record grey tones.
Most sensors use a Bayer array placed over the sensor the filters incoming light using three colour filters - red, green, and blue.
Generally the pattern is a two-by-two repeating pattern with two green and a single red and a single blue filter.
The demosaicing process is an estimation process of what the actual colour, sensel for sensel, that each sensel should be based on how much light has struck that sensel and the surrounding sensels with due regard for the colour filter governing light transmission for each sensel.
So, the decisions made by the RAW converter are governed by what is known about the sensor and some of the A/D processing.
There are two ways this information can be obtained.
Either the information is provided by the manufacturer/designer or the information is obtained by reverse engineering.
Almost all the required information, for the third-party RAW converters, has, historically anyway, been got via reverse engineering.
I have some recent information that I have no personal way of verifying that Adobe may currently be getting their data from the camera-makers directly. Apparently this courtesy has not been extended to other software companies. As I have said I cannot verify this personally.
However, part of the design of the DNG format is to make possible a camera-model independent RAW processing experience where should any camera model use DNG as its RAW format, that no RAW converter would need the information currently required to accurately demosaic the RAW file as well as make other decisions. The process does not risk any proprietary secrets.

Some, but not all of the information in this post, is from the latest DNG specification that I accessed straight off Adobe's website. All the other information has been published on sites well known to the photographic community, and last, but not least LuLa itself.

There has been a hugh amount of heat generated based on complete furfies.
Perhaps you may not want, or feel the need, for a universal RAW format.
You may not want DNG adopted.
These are valid positions per se.
What is not so tenable though is the 'don't confuse me with facts approach' that has been evident.

My advice is NOT to take at face value anything written above - go and research this stuff yourself.
Draw your own conclusions based on knowledge, not ignorance.

BTW Any RAW converting process based on DCRAW has limitations based on the lack of documented format information. All the conversion information required is reverse engineered, ingeniously, but still a hack. What we really need are RAW files that do not need model-specific information for conversion. DNG does offer this.

Remember clarify this all for yourself - nonetheless my reading of the situation is not so rosy especially looking forward without a universal RAW format.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bryan Conner on April 24, 2013, 12:34:58 am
Thanks Tony.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Schewe on April 24, 2013, 12:56:02 am
There are no proprietary RAW formats currently in production that cannot be accurately and fully documented in a DNG file.

Well, to be accurate, there are almost no proprietary RAW formats currently in production that cannot be accurately and fully documented in a DNG file...sadly, the recent Sigma cameras that use Foveon chips can't...the couple of cameras that can be converted to DNG end up as Linear DNG, not raw DNG. And the recent cameras by Sigma aren't supported at all although Adobe is working with Sigma to change that. Sadly, those people who have bought those cameras have no DNG/ACR/LR support and users are forced to use the Sigma supplied raw processing software :~(
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 24, 2013, 02:01:04 am
Well, to be accurate, there are almost no proprietary RAW formats currently in production that cannot be accurately and fully documented in a DNG file...sadly, the recent Sigma cameras that use Foveon chips can't...the couple of cameras that can be converted to DNG end up as Linear DNG, not raw DNG. And the recent cameras by Sigma aren't supported at all although Adobe is working with Sigma to change that. Sadly, those people who have bought those cameras have no DNG/ACR/LR support and users are forced to use the Sigma supplied raw processing software :~(
Thanks for the clarification Jeff.
I will follow that up and read all about it.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on April 24, 2013, 03:55:36 am
Well, memory jogged.

I did read Michael's review of the Sigma SD1 but some of that detail did not really penetrate at the time.

More in depth research of the sensor and how it works clearly indicates why neither Adobe products or any DCRAW application can read the file.
Paradoxically though, my reading of this situation indicates that this is a situation that DNG was born for, not as an intermediate, but as the native out-of-the-camera RAW file.
Here we have a very interesting new type of sensor that is potentially a good step forward in sensor technology. Potentially better colour fidelity because no Bayer-type arrays are required so no demosaicing (and all its attendant assumptions) and no anti-aliasing filter is required so real resolution/detail may be improved. Nice!

But, the only bit of software that can read this baby's files is provided by Sigma.
Given Michael's brief dismissal of that software obviously it is not up to scratch.

However, if Sigma took on DNG as its native RAW format the result could be a RAW file of rare quality that is also simultaneously accessible by any RAW converter that knows DNG (and there is no reason that, that should not be possible).
If Sigma did this I would suggest that any Sigma camera model using Foveon-type sensors would really become hot-ticket items.
Potentially a win for Sigma.

I doubt anything I have written here in the absolute sense is original.
This must have occurred to others.
Nonetheless, this is an excellent potential use-case demonstating the principle for which the DNG format was designed.
Many cameras using very diverse technology can output a camera-independent RAW format that can be read and processed by any application that knows DNG without that application needing to know anything about the camera that originally produced the image.

Whats not to like?

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on April 24, 2013, 04:46:02 am
What would be a gas is that devs
Could provide a separate file (like the RMD)
Where are stored all the metadatas.
The RAW file remaining always untouched.

This system is incredibly flexible.

Red adopted it and it really is taking full advantage
Of the RAW.

You can create a look from the dev, put the file into
The card and shoot with the created look. You film
With the parameters that have been created in their
RAW dev. But the camera settings aren't lost either.

You can mail the file. Batch apply any look in the editor,
In a second, rebuild the looks at any point, name those
Metadatas files so they have a meaning within the
Prod, assure that every system reads the same metadatas
Accuratly.

To simplify, the file structure is
001.r3d. The RAW
001.rmd. the metadatas of the 001, at the same
Directory, plus,
All the created rmd. That could be located anywhere and
Named. Ex: folder name: night-shots-day-1, with inside
Nameofyourchoice.rmd etc...

They can be loaded in-camera or in a proper software
That is not Red proprietary as LUTs.

For the ones who are not familiar with this type of workflow,
I did a short video tutorial recently on the advantage of having
separate standardized metadatas files.
http://vimeo.com/63008611

(ps: very sorry for my english, this microphone experience has been a nightmare
for me as I was looking for my words all the time and, well, it's clumpsy speach
but you got the idea.)
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: AlfSollund on May 01, 2013, 12:04:21 pm
For what its worth, I believe it is in our best interests to support the adoption of a non-proprietary raw format from the camera manufactures. This would benefit us with lower manufacture costs and improved future-prof interoperability.

I thank for all the information given, in particular to Tony.

What really eludes me is the reasoning behind what I consider as personal resistance against a (de-facto) standardized open RAW format. Is it some kind of hidden fan-boy resistance due to the understanding that Canikon and others have made the wrong choices? By wrong I mean long-term both internally as a cost-driwer as well as a part of a photographic community. Sorry if tis offends, its not a personal attack on anyone.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Vladimirovich on May 01, 2013, 06:36:38 pm
But, the only bit of software that can read this baby's files is provided by Sigma.

or Iridient Rawdeveloper for example... which is a one man shop really... which shows how much effort really needed if there is a desire to support...

Many cameras using very diverse technology can output a camera-independent RAW format that can be read and processed by any application that knows DNG without that application needing to know anything about the camera that originally produced the image.

once upon a time Panasonic decided to introduce a certain flavor of software correction parameters to be stored in raw files... do you remember how much time it took Adobe to support that in DNG ? imagine DNG being an ISO standard... shall Panasonic disclose in advance their move to competition ? shall Panasonic seek approvement from competition for what they want ? no, decided Panasonic... so they did what they wanted, when they wanted and how they wanted... and half a year later Adobe had to update the DNG standard... right  ::) ?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Bryan Conner on May 02, 2013, 12:38:28 am

once upon a time Panasonic decided to introduce a certain flavor of software correction parameters to be stored in raw files... do you remember how much time it took Adobe to support that in DNG ? imagine DNG being an ISO standard... shall Panasonic disclose in advance their move to competition ? shall Panasonic seek approvement from competition for what they want ? no, decided Panasonic... so they did what they wanted, when they wanted and how they wanted... and half a year later Adobe had to update the DNG standard... right  ::) ?

I did not know that Adobe in competition with Panasonic.  How is this?  I am by no means an expert on dng, or any raw file format, but does a camera manufacturer have to reveal what they are doing to Adobe in order to make a change to the parameters in the dng file format?  In other words, does Adobe have to know what is going into the container before making changes to the container?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Ellis Vener on May 02, 2013, 08:55:23 pm
Tony Jay: That's a well and dispassionately stated debunking of the F-U-D spreading about DNG.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: Tony Jay on May 03, 2013, 12:36:12 am
Thank you Ellis.

Tony Jay
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on May 03, 2013, 07:25:32 am
We really need standardized international
And independant formats and workflows,
Like the efforts in the as11, with capabilities
To add shims.

The counsiousness of broadcasters in order
To solve the not-tolerable-anymore proprietary
Mess and move from "common practises" to
Real common file format is not a luxury.

We still are in the wild west in digital imagery,
And it needs to be civilized.

This, also concerns still imagery and specially
Raw formats.
When RAWvideo will be the norn, it will not be
Possible to handle with the nonsense current panorama.

It's not a question if standart will occur, it will,
It's more a question of when.

And the very important issue is separate metadatas files
from the viewing datas. Really really key. We need proper versioning
within platforms and proper mastering-archiving etc...
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: NikoJorj on May 06, 2013, 03:54:57 am
Just to cast another "yes" vote...
Beyond the standardisation issue, which is a tad more about principles than pragmatism - even if those are good principles in my book, because for an end-user like me, camera support in "any" (as "any that supports DNG") software would be a appreciable bonus.

once upon a time Panasonic decided to introduce a certain flavor of software correction parameters to be stored in raw files... do you remember how much time it took Adobe to support that in DNG ? imagine DNG being an ISO standard...
DNG supports proprietary metadata, so it would certainly not block innovation, and from what I understand of DNG structure, the file should still be processed in other software not aware of the innovation, only without this particular innovation - the relevant metadata just waiting to be read to add the plus.
From what I can guess and imagine, DNG adoption would really be a win-win situation in such a case.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: madmanchan on May 06, 2013, 09:24:20 am
Pentax and Ricoh (now under the same roof), Leica, Casio, and other adopters of DNG routinely store proprietary metadata in the private DNG MakerNote.  They use such information for some vendor-specific image processing routines, diagnostics, etc.  There does not appear to be any conflict between their desire to keep this information private, and the ability to store the raw image data itself and associated essential metadata (e.g., white balance coefficients, calibration data, opcode corrections, etc.) in a publicly accessible and documented form.
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: digitaldog on May 06, 2013, 10:17:27 am
This was not from digital cameras, but Photo CDs  He did not totally lose them, as there are means to recover them...he has to make decision if the effort is worth it.  With 20/20 hind sight it might have been avoided.

I was hoping to stay out of this since we've been down this rabbit hole but to clarify:

The problem (and it IS a problem for me) is I have both PhotoCD and old camera file formats I can't access today. Cameras from early Kodak, Leaf, Phase and others while doing reviews of those cameras. It is NOT impossible for me to access that data but it would be very difficult and very expensive to do so. WHY should I have to suffer this problem when a camera built and used during the same time that captures an open format (say JPEG) suffers none of these issues? The reason this problem exists is the proprietary raw format, nothing more. There is zero, none, nada reason for this. It's totally political. But I said all this to you jrs about a month ago, so why repeat myself again?

20/20 has nothing to do with it. There was no DNG at the time these images were captured nor a means of converting to an open raw format. The same ugly situation could happen in 10 years with the same results. Why should we allow this to be such an unfortunate option?
Title: Re: Should Camera Companies adopt non-proprietary raw?
Post by: fredjeang2 on May 06, 2013, 01:05:18 pm
The situation is that, as users, we have no
Control on brands politics and are on bondage.

 there isn't to date a universal adoption of dng
Neither for stills nor motion.  This is a very
Unfortunate situation but that's what is.

I can't wait to see the holy mess comming in motion
Because the post is so more complex and when
Will have red RAW, arriraw, Cinema dng, canon raw, nikon
Raw, panasonic raw, Sony raw, phantom raw,
This is going to be the HUGE mess.

The editing rooms are going to be in flames. As if it
Wasn't enough mess with the non RAW formats, but
No, this is nothing with what's coming slowly but surely.

The wild west is going to feel like a refined classical palace.