Luminous Landscape Forum

Raw & Post Processing, Printing => Digital Image Processing => Topic started by: albytastic on February 08, 2019, 09:38:06 am

Title: JPEG or RAW?
Post by: albytastic on February 08, 2019, 09:38:06 am
With the various arguments about whether Topaz Labs new product "JPEG to RAW" actually can turn a JPEG into a 16 bit TIFF or DNG file.

The usual argument in favour of using RAW is that a JPEG, being only 8 bits, has lost too much digital information for it to become a 16 bit file.

So here is a little test - I have posted 2 images on my Flickr site, can you tell the difference?

A little hint - both were processed in J2R then edited exactly the same way.

They are both 100% crops.

https://www.flickr.com/photos/20926615@N05/46303221754/in/dateposted-public/

https://www.flickr.com/photos/20926615@N05/46303183134/in/dateposted-public/

Look forward to your replies.
Title: Re: JPEG or RAW?
Post by: Slobodan Blagojevic on February 08, 2019, 10:01:35 am
... both were processed in J2R then edited exactly the same way.

You started with a raw file and processed it in J2R!?
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 10:08:29 am
I subscribe to Ask Tim Grey's daily newsletter in which he answers readers' questions, and today's email is about Topaz JPEG to RAW. Here is Tim's text:

<<Today's Question
I got an email about software from Topaz Labs that claims to convert JPEG images to raw captures. Is that even possible, and if so will it provide all of the benefits of raw?

Tim's Quick Answer:
The "JPEG to RAW AI" software from Topaz Labs does not provide the same benefits as a raw capture, and frankly I feel that their marketing around this software is misleading.

More Detail:
"JPEG to RAW AI" from Topaz Labs enables you to batch process JPEG images and convert them to a DNG or TIFF image with a 16-bit per channel bit depth. As part of the processing, various enhancements are applied to the image. The claim is that the result will be greater dynamic range, a larger color space, higher bit depth, reduced artifacts, and increased detail.
 
To begin with, converting a JPEG image to a DNG or TIFF file format with a different color space and higher bit-depth setting does not provide any quality benefit for the image all by itself. The only real benefit from these changes would be the potential for better image quality after applying strong adjustments. The exact same results could be achieved by changing the color space and bit depth for an image in Photoshop, for example, with no visible change in appearance for the photo.
 
After testing a variety of images with JPEG to RAW AI, I did not find that there was any significant improvement in the level of detail in the photos. Some photos showed evidence of contrast enhancement and sharpening in certain areas, which obviously could also be applied using other software.
 
While some of the visible artifacts in JPEG images I tested with JPEG to RAW AI were reduced, in areas where artifacts were reduced overall sharpness and detail were also reduced. In some cases detail enhancement in certain areas of an image actually increased the visibility of artifacts in the image.
 
Overall I was not impressed with the results I achieved with the JPEG images I processed with JPEG to RAW AI. More worrisome to me, however, is that I feel the way the product is being marketed is misleading. While I do feel that some of the software products from Topaz Labs are very good, I would not recommend JPEG to RAW AI.
 
If you'd like to check out JPEG to RAW AI for yourself, you can get more info on the Topaz Labs website here:
 
https://topazlabs.com/jpeg-to-raw-ai/ref/273/  >>

It seems that Tim tested the application with at least several sample photos to come to these conclusions.

Beyond what Tim reports, I wonder just how much quality enhancement inventing data can provide. To some extent and depending on the operations being performed, it can be useful; but we need to be mindful that for many of our cameras, bit depth exceeds JPEG bit-depth and the JPEG has already eliminated much of the scene referred raw data that a camera sensor records, so any processing already starts with that disadvantage, which would most likely make a visible difference to image quality the more the photo is processed and magnified. Of course the only way to know for sure whether this application does good things for us is to try it in the context of our own requirements and see. I won't be doing so because none of my photography uses JPEG as the initial capture format. I only record raw files and periodically convert some photos to JPEG as useful for transmission over the Internet.
Title: Re: JPEG or RAW?
Post by: 32BT on February 08, 2019, 10:18:28 am

Look forward to your replies.

One swallow does not a summer make...
Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 11:41:37 am
You started with a raw file and processed it in J2R!?

Wrong!
Title: Re: JPEG or RAW?
Post by: Slobodan Blagojevic on February 08, 2019, 11:46:15 am
Wrong!

So what exactly are you asking us? Can we tell the difference between what? Your thread title and subsequent explanation implies that one of the two images started as a raw file.

Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 11:49:03 am
So what exactly are you asking us? Can we tell the difference between what? Your thread title and subsequent explanation implies that one of the two images started as a raw file.

Wrong again!
Title: Re: JPEG or RAW?
Post by: Garnick on February 08, 2019, 11:49:55 am
With the various arguments about whether Topaz Labs new product "JPEG to RAW" actually can turn a JPEG into a 16 bit TIFF or DNG file.

The usual argument in favour of using RAW is that a JPEG, being only 8 bits, has lost too much digital information for it to become a 16 bit file.

So here is a little test - I have posted 2 images on my Flickr site, can you tell the difference?

A little hint - both were processed in J2R then edited exactly the same way.

They are both 100% crops.

https://www.flickr.com/photos/20926615@N05/46303221754/in/dateposted-public/

https://www.flickr.com/photos/20926615@N05/46303183134/in/dateposted-public/

Look forward to your replies.


First of all, your first sentence is irrelevant.  Photoshop can do this as well, has been able to do so for many versions.

The second sentence is also somewhat irrelevant.  A JPEG file has 8 bits per channel.  All of this has been previously explained but I will do so one more time.  If you convert a JPEG file to a 16bit file the container is only half full.  You have therefore accomplished nothing.  As far as I'm aware the only way to convert an 8bit file to a true 16bit file is through interpolation, which is never the best approach.  Since we do not know how you have processed each of your example files that is also an irrelevant exercise in my opinion. 

Andrew has many times asked for some real time proof of the claims made by Topaz concerning J2R.  I agree with Andrew to some extent, that being the fact that the initial promotion of this new Topaz app is very misleading.  So basically they're playing a word game here.  If the product is as good as they claim, why not invent a new word to use in their advertising campaign that does not imply that a JPEG  file can magically become a RAW 16bit file? 

Bart has just put up two examples as well and I will say it is starting to look like something to be investigated further.  I have and use some of the Topaz plugins and for the most part they do what is advertised, and with no hyperbole. 

Now here's a test for you.  Open two new files in Photoshop, one 8bit and one 16bit at a size of approximately 10x3".  Now create a gradient from absolute black to absolute white across the 10' dimension on each file, 8 & 16bit.  Then posterize the gradient on each file with 21 steps, or as many steps as you like.  Now look at each file.  The demarcation between steps should look quite different, rather blurred and dirty in the 8bit file as opposed to the clean demarcation between steps in the 16bit file.  Now here's the tricky part.  Convert the 8bit file to 16bits.  Do the demarcations between the steps look any better now?  And of course the answer is NO.  That's because all you have done is put an 8bit file into a 16bit container.  Nothing more and nothing less.  You have not increased the bit depth of the 8bit file.  Now do the same with the 16bit file, convert it to 8bit.  You will notice no difference in the demarcation between steps because all you have done is put the 16bit file into a smaller 8bit container.  In other words, all of the data in the 16bit file has been squished down to fit the 8bit container.  However, if you now try to make some rather difficult adjustment to that new 8bit file you will see problems, because you have discarded some of the actual bit depth you had in the 16bit file and you can never get it back again. 

Perhaps at some point AI will indeed be able to do a much higher quality interpolation routine.  And as I mentioned, Bart's examples are very interesting and I may try the demo as well, but I will not jump on board for full $$ commitment until there's more proof of absolute quality.

Gary
Title: Re: JPEG or RAW?
Post by: Slobodan Blagojevic on February 08, 2019, 11:57:40 am
Wrong again!

WTH!? Cat got your tongue!? Can't you answer a simple question?
Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 12:09:17 pm

First of all, your first sentence is irrelevant.  Photoshop can do this as well, has been able to do so for many versions.

The second sentence is also somewhat irrelevant.  A JPEG file has 8 bits per channel.  All of this has been previously explained but I will do so one more time.  If you convert a JPEG file to a 16bit file the container is only half full.  You have therefore accomplished nothing.  As far as I'm aware the only way to convert an 8bit file to a true 16bit file is through interpolation, which is never the best approach.  Since we do not know how you have processed each of your example files that is also an irrelevant exercise in my opinion. 

Andrew has many times asked for some real time proof of the claims made by Topaz concerning J2R.  I agree with Andrew to some extent, that being the fact that the initial promotion of this new Topaz app is very misleading.  So basically they're playing a word game here.  If the product is as good as they claim, why not invent a new word to use in their advertising campaign that does not imply that a JPEG  file can magically become a RAW 16bit file? 

Bart has just put up two examples as well and I will say it is starting to look like something to be investigated further.  I have and use some of the Topaz plugins and for the most part they do what is advertised, and with no hyperbole. 

Now here's a test for you.  Open two new files in Photoshop, one 8bit and one 16bit at a size of approximately 10x3".  Now create a gradient from absolute black to absolute white across the 10' dimension on each file, 8 & 16bit.  Then posterize the gradient on each file with 21 steps, or as many steps as you like.  Now look at each file.  The demarcation between steps should look quite different, rather blurred and dirty in the 8bit file as opposed to the clean demarcation between steps in the 16bit file.  Now here's the tricky part.  Convert the 8bit file to 16bits.  Do the demarcations between the steps look any better now?  And of course the answer is NO.  That's because all you have done is put an 8bit file into a 16bit container.  Nothing more and nothing less.  You have not increased the bit depth of the 8bit file.  Now do the same with the 16bit file, convert it to 8bit.  You will notice no difference in the demarcation between steps because all you have done is put the 16bit file into a smaller 8bit container.  In other words, all of the data in the 16bit file has been squished down to fit the 8bit container.  However, if you now try to make some rather difficult adjustment to that new 8bit file you will see problems, because you have discarded some of the actual bit depth you had in the 16bit file and you can never get it back again. 

Perhaps at some point AI will indeed be able to do a much higher quality interpolation routine.  And as I mentioned, Bart's examples are very interesting and I may try the demo as well, but I will not jump on board for full $$ commitment until there's more proof of absolute quality.

Gary

An interesting idea and I will try it when I have time.

But the actual purpose of the 2 test images was simple - how much infomation is there in a JPEG that could be used by JPEG to RAW?

And the answer is a lot more than most people realise.

They were both JPEGs from my Canon 1Ds MkII which gives 16 MP images.

Test A is the original JPEG, just over 10MB in size.

Test B is a reduced version of that - 833Kb in fact!

The original JPEG was reduced by Easy Thumbnails and kept the image size but reduced the filesize (JPEG Quality set to 60%)

Yet it still produced a good image despite the huge loss of data.

So if it is possible to produce a good image from such a small filesize it means there is a huge amount of available data in JPEGs.

So I see no reason why Topaz Program cannot use this data to do exactly what it says.

Both full size images remained at 16MP regardless of the filesize.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 12:50:21 pm
With the various arguments about whether Topaz Labs new product "JPEG to RAW" actually can turn a JPEG into a 16 bit TIFF or DNG file.
What arguments? There's nothing at all unique or special about converting a JPEG to a TIFF into DNG and upping the bit depth. Been doable for couple of decades.
What's bogus is the idea Topaz Labs converts a JPEG to raw data.
Converting a JPEG to DNG absolutely doesn't make it raw! You were told this fact in the other post: DNG like TIFF is simply a container. A JPEG converted to DNG is a JPEG in the DNG wrapper. You can open a JPEG in Photoshop, change the mode to 16-bit and make a DNG. It's still not raw. You didn't add any additional data, you simply provided a finer encoding of the existing numbers!
Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 12:53:56 pm
What arguments? There's nothing at all unique or special about converting a JPEG to a TIFF into DNG and upping the bit depth. Been doable for couple of decades.
What's bogus is the idea Topaz Labs converts a JPEG to raw data.
Converting a JPEG to DNG absolutely doesn't make it raw! You were told this fact in the other post: DNG like TIFF is simply a container. A JPEG converted to DNG is a JPEG in the DNG wrapper. You can open a JPEG in Photoshop, change the mode to 16-bit and make a DNG. It's still not raw. You didn't add any additional data, you simply provided a finer encoding of the existing numbers!

Not what I was actually getting at.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 12:58:26 pm
Not what I was actually getting at.
What are you trying to get at? Use your words. Here are some of your words which are not correct (there's no such argument):
The usual argument in favour of using RAW is that a JPEG, being only 8 bits, has lost too much digital information for it to become a 16 bit file.
Got very little to do with bit depth and a lot to do with the difference between a raw, un-rendered piece of digital clay and a JPEG which is baked by something else you have no control over (compared to raw). The extra bit depth is simply icing on the cake. I'll post this again here for you to examine facts about the differences in raw plus JPEG (and there's more but this is a lot):

The JPEG engine that processes the raw massively clips and compresses highlights. We often don't when editing the raw. This compression can clump midtones as much as 1 stop while compressing shadow details! People incorrectly state that raw has more highlight data but the fact is, the DR captured is an attribute of the capture system; it's all there in the raw but maybe not in a camera proceed JPEG.
A raw capture that's 10 or 11 stops of dynamic range can be compressed to 7 stops from this JPEG processing which is a significant amount of data and tonal loss! So when we hear people state that a raw has more DR than a JPEG, it's due to the poor rendering or handling of the data to create that JPEG. The rendering of this data and the reduction of dynamic range is from the JPEG engine that isn't handling the DR data that does exists as well as we can from the raw! Another reason to capture and render the raw data, assuming you care about how the image is rendered!

Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 01:07:32 pm
What arguments? There's nothing at all unique or special about converting a JPEG to a TIFF into DNG and upping the bit depth. Been doable for couple of decades.
What's bogus is the idea Topaz Labs converts a JPEG to raw data.
Converting a JPEG to DNG absolutely doesn't make it raw! You were told this fact in the other post: DNG like TIFF is simply a container. A JPEG converted to DNG is a JPEG in the DNG wrapper. You can open a JPEG in Photoshop, change the mode to 16-bit and make a DNG. It's still not raw. You didn't add any additional data, you simply provided a finer encoding of the existing numbers!

Exactly, and what I was getting at in Reply #2 though in a different way. By the time an image becomes a JPEG it is encoded as pixels and cannot be raw. The only raw data is what the camera captures at the time of capture. Taking your thought a bit further, this is not the first time the same error has been made, with certain developers claiming that the files their software produces are raw when all they've done is to encase fully rendered, pixel-based data in a DNG container.

Now, turning to the 8 to 16-bit part of it, Andrew, you say this is providing a finer encoding of the existing numbers. In a way, yes, the encoding may be finer because 8 depth becomes 16 depth, but is the real content of it any finer? Information is being invented to do this. Is it safe to say that invented information is in one way or another largely a clone of existing information, and therefore doesn't contribute anything incrementally useful to how the file can be edited or purposed?
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 01:16:28 pm
Exactly, and what I was getting at in Reply #2 though in a different way. By the time an image becomes a JPEG it is encoded as pixels and cannot be raw. The only raw data is what the camera captures at the time of capture. Taking your thought a bit further, this is not the first time the same error has been made, with certain developers claiming that the files their software produces are raw when all they've done is to encase fully rendered, pixel-based data in a DNG container.
I think both are points are falling on deaf ears that need a lot more education on this topic.

Quote
Now, turning to the 8 to 16-bit part of it, Andrew, you say this is providing a finer encoding of the existing numbers. In a way, yes, the encoding may be finer because 8 depth becomes 16 depth, but is the real content of it any finer? Information is being invented to do this. Is it safe to say that invented information is in one way or another largely a clone of existing information, and therefore doesn't contribute anything incrementally useful to how the file can be edited or purposed?
It's quite possible subsequent editing now on high bit data could be useful but there are too many factors to declare one way or the other: what kind of edit, what condition was the JPEG in, in the first place.
As you know, LR and ACR process JPEGs in high bit, wide gamut. That's how their processing engines operate. There are advantages to parametric editing and at least in terms of Adobe, that the edits are applied in a fixed order, not user order. So there are possible difference in just the order edits are applied (Photoshop vs. LR) that alone could factor into this. Too many variables.
What people need to understand, most certainly the OP, is there's no free lunch. Raw, high bit, wide gamut potential (let alone huge options for rendering) isn't anything like a baked, sRGB JPEG in 8-bits per color.
A carrot cake isn't the same as the individual ingredients used to make that cake and you can't extract them and start over again. Raw equates to the ingredients and how you deal with those ingredients plays a massive role in the outcome. JPEG is the baked cake. You can add more icing perhaps or sprinkles. But if you used salt instead of sugar to make that cake, NOTHING will make it taste right. No matter how much additional sugar you pour on top of it.
These facts are ignored by some who are looking for magic software with algorithm's that contain unicorns, coded by the tooth fairy. Those that think one can use a product to convert a JPEG into raw data. We can't get them out of that unreality bubble easily.  :-[
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 01:27:54 pm
Exactly, and what I was getting at in Reply #2 though in a different way. By the time an image becomes a JPEG it is encoded as pixels and cannot be raw. The only raw data is what the camera captures at the time of capture. Taking your thought a bit further, this is not the first time the same error has been made, with certain developers claiming that the files their software produces are raw when all they've done is to encase fully rendered, pixel-based data in a DNG container.

Hi Mark,

Quite true, if that were all that happens here.

With all the risk of failure that's inherent with analogies, maybe the following concept is of some use to those unfamiliar with A.I.:
The challenge is not unlike trying to unscramble an omelet.

It is not possible with traditional means, but we can try by starting over with fresh image fragments eggs and through a process of training and learning (scrambling and comparing the new result with the omelet we had), one might be able to approach the result of the omelet at hand, and maybe improve it in certain aspects. If improved, we achieved that by using (newly synthesized) Raw files eggs.

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 01:41:10 pm
Hi Bart,

OK, Andrew posted at 1:16 and you posted at 1:27 so presumably you saw what he wrote. Does this boil down to the proposition that only the tooth fairy can unscramble omelettes, or does A.I. really make it possible to reverse-engineer or parse what is an already heavily compromised data base into anything resembling what the original raw data would have been like? I have to say that regardless of the amazing things A.I. can do these days, I remain thoroughly unconvinced about this one.
Title: Re: JPEG or RAW?
Post by: faberryman on February 08, 2019, 01:46:10 pm
Does this boil down to the proposition that only the tooth fairy can unscramble omelettes, or does A.I. really make it possible to reverse-engineer or parse what is an already heavily compromised data base into anything resembling what the original raw data would have been like? I have to say that regardless of the amazing things A.I. can do these days, I remain thoroughly unconvinced about this one.
Content aware fill frequently works surprisingly well.
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 01:55:33 pm
Content aware fill frequently works surprisingly well.

Correct, and that's why near the bottom of Reply 2 I said invented information can be useful.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 01:58:33 pm
Hi Mark,

Quite true, if that were all that happens here.

With all the risk of failure that's inherent with analogies, maybe the following concept is of some use to those unfamiliar with A.I.:
The challenge is not unlike trying to unscramble an omelet.
The challenge is telling the truth with a software manufacturer that tells us that's what it's doing; converting a JPEG into raw.
They don't have to say that, since it's untrue. But they have decided to go that route. I find it inexcusable. But some don't mind being lied to.
A.I. isn't converting the JPEG to raw.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 02:00:13 pm
Correct, and that's why near the bottom of Reply 2 I said invented information can be useful.
And one could suggest the clone tools does this as well. Content Aware in PS does work well often (and fails as well) but it's simply cloning existing data. It isn't pretending to take the existing data and make it 3D, or increase it's DR, or turn it into raw data.
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 02:06:27 pm
And one could suggest the clone tools does this as well. Content Aware in PS does work well often (and fails as well) but it's simply cloning existing data. It isn't pretending to take the existing data and make it 3D, or increase it's DR, or turn it into raw data.

Exactly, but I would suggest the word "simply" could be misunderstood. I have been impressed with the extent of complex infills it can create, so there's some very clever math going on under the hood, but in the final analysis, yes correct, it is essentially borrowing and transposing existing information intelligently.
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 02:17:22 pm
Hi Bart,

OK, Andrew posted at 1:16 and you posted at 1:27 so presumably you saw what he wrote. Does this boil down to the proposition that only the tooth fairy can unscramble omelettes, or does A.I. really make it possible to reverse-engineer or parse what is an already heavily compromised data base into anything resembling what the original raw data would have been like?

As I said Mark, it is not possible to unscramble the omelet, or unbake Andrew's carrot cake.

The goal of A.I. is not to unscramble/unbake, but scramble/bake again starting with other/better Raw ingredients. The way that is attempted here is by using different Raw files/fragments and resulting JPEGs, trying many combinations of Raw conversion deterioration until one can produce the same effect, and stop doing it that way! When a better result is produced, one probably found a better Raw input that survived the deterioration process (lossy compression, reduced dynamic range, partially clipped highlight), or a way to undo the deterioration. That better recipe it then tried on other image fragments, until it more often than not also creates better output on other ingredients.

This is an extremely complex process (if it were easy it would have been done already), and it can (as TopazLabs have mentioned) take months/weeks and many example image pairs before something better is found.

Result is also dependent on the training data set. A small set is easy to train on, but the resulting recipe will only work well for that specific set, and fail on anything else. To achieve more general usefulness, many many representative images are required.

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 02:29:05 pm
As I said Mark, it is not possible to unscramble the omelet, or unbake Andrew's carrot cake.
So we now agree it's not possible for Topaz Labs to convert a JPEG to raw? Or that TIFF I provided for you?
Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 02:31:12 pm
As I said Mark, it is not possible to unscramble the omelet, or unbake Andrew's carrot cake.

The goal of A.I. is not to unscramble/unbake, but scramble/bake again starting with other/better Raw ingredients. The way that is attempted here is by using different Raw files/fragments and resulting JPEGs, trying many combinations of Raw conversion deterioration until one can produce the same effect, and stop doing it that way! When a better result is produced, one probably found a better Raw input that survived the deterioration process (lossy compression, reduced dynamic range, partially clipped highlight), or a way to undo the deterioration. That better recipe it then tried on other image fragments, until it more often than not also creates better output on other ingredients.

This is an extremely complex process (if it were easy it would have been done already), and it can (as TopazLabs have mentioned) take months/weeks and many example image pairs before something better is found.

Result is also dependent on the training data set. A small set is easy to train on, but the resulting recipe will only work well for that specific set, and fail on anything else. To achieve more general usefulness, many many representative images are required.

Cheers,
Bart

Excellently put Bart - but at the end of the day what so many people are desperately striving to avoid mentioning - IT WORKS!

And at the end of MY day that's all I'm really interested in!
Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 02:41:34 pm
So we now agree it's not possible for Topaz Labs to convert a JPEG to raw? Or that TIFF I provided for you?

But they do create RAW file - DNG

"DNG is also considered to be a RAW image file. It is Adobe’s proprietary image standard that was created to store image data in a generic, highly-compatible format, unlike RAW files that have specific formats based on manufacturer and camera type. Although DNG was invented by Adobe and is supported in all Adobe applications, there are other camera manufacturers such as Leica, Hasselblad and Pentax that adopted this standard and use it in their cameras as their native and supported RAW file format."

So 3 major manufacturers actually use DNG as their RAW files.

https://expertphotography.com/dng-file-vs-raw-file/

sorry FOUR.
Title: Re: JPEG or RAW?
Post by: Jeremy Roussak on February 08, 2019, 02:51:20 pm
But they do create RAW file - DNG

Alby, may I, with respect, suggest that until you understand what you're talking about, you defer to those who do?

Jeremy
Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 02:52:33 pm
Alby, may I, with respect, suggest that until you understand what you're talking about, you defer to those who do?

Jeremy

But I am, the actual camera manufacturers.
Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 02:54:13 pm
Alby, may I, with respect, suggest that until you understand what you're talking about, you defer to those who do?

Jeremy

But you are quite right and I feel this topic is now exhausted.

I don't know what time it is in the USA but her in Britain it's dinner time.

So good night and sleep well.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 02:56:35 pm
But they do create RAW file - DNG

"DNG is also considered to be a RAW image file. It is Adobe’s proprietary image standard that was created to store image data in a generic, highly-compatible format, unlike RAW files that have specific formats based on manufacturer and camera type. Although DNG was invented by Adobe and is supported in all Adobe applications, there are other camera manufacturers such as Leica, Hasselblad and Pentax that adopted this standard and use it in their cameras as their native and supported RAW file format."

So 3 major manufacturers actually use DNG as their RAW files.

https://expertphotography.com/dng-file-vs-raw-file/ (https://expertphotography.com/dng-file-vs-raw-file/)

sorry FOUR.
You're confused again. DNG may equal raw, DNG may not equal raw. You've been told over and over again, and provided outside references that DNG is a container for image and metadata. You refuse to accept that a JPEG placed INTO a DNG is still a JPEG and NOT raw data.
A camera may indeed write raw data into a DNG. That data IS raw.
A proprietary raw can be converted to DNG, that sensor data is raw (and again, all that is converted is simply metadata).
A JPEG or TIFF can be converted to a DNG. That absolutely doesn't make that raw data.
I'd provide the actual DNG spec that talks of this but I suspect it would over your head as you simply cannot accept or understand the multiple outside references I've already provided to you that clearly state what happens when you convert a JPEG to DNG. What it absolutely does NOT do is make that JPEG raw data. If you believe that, you're stuck in an unreality bubble.



You've been tricked by misunderstanding and assumptions about DNG. DNG may equal raw. DNG may not equal raw. As for JPEG to DNG, read, learn:




http://asktimgrey.com/2015/09/01/jpeg-to-dng/
https://forums.adobe.com/thread/1049709
https://forums.adobe.com/thread/2189544
https://forums.adobe.com/thread/968134
https://photo.stackexchange.com/questions/73475/how-to-convert-jpeg-to-raw-in-photoshop-or-similar
https://www.lightroomqueen.com/community/threads/jpg-to-dng.35456/
Title: Re: JPEG or RAW?
Post by: Slobodan Blagojevic on February 08, 2019, 02:57:08 pm
But I am, the actual camera manufacturers.

You really, really, really do not understand the subject matter.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 03:00:41 pm
You really, really, really do not understand the subject matter.
I'd like to and yet another "really".  :-[
Title: Re: JPEG or RAW?
Post by: Eric Myrvaagnes on February 08, 2019, 03:14:45 pm
You really, really, really do not understand the subject matter.
+1000.
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 03:16:52 pm
So we now agree it's not possible for Topaz Labs to convert a JPEG to raw? Or that TIFF I provided for you?

Selective quoting of only part of my answer, only demonstrates unwillingness to understand what A.I. is about.

Or to put it differently (with a wink to the late Douglas Adams), the answer to your question is: 42.
If the answer seems to make little sense to someone, maybe the question was ill-posed, or that someone is incapable of understanding?

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 03:27:07 pm
As I said Mark, it is not possible to unscramble the omelet, or unbake Andrew's carrot cake.

The goal of A.I. is not to unscramble/unbake, but scramble/bake again starting with other/better Raw ingredients. The way that is attempted here is by using different Raw files/fragments and resulting JPEGs, trying many combinations of Raw conversion deterioration until one can produce the same effect, and stop doing it that way! When a better result is produced, one probably found a better Raw input that survived the deterioration process (lossy compression, reduced dynamic range, partially clipped highlight), or a way to undo the deterioration. That better recipe it then tried on other image fragments, until it more often than not also creates better output on other ingredients.

This is an extremely complex process (if it were easy it would have been done already), and it can (as TopazLabs have mentioned) take months/weeks and many example image pairs before something better is found.

Result is also dependent on the training data set. A small set is easy to train on, but the resulting recipe will only work well for that specific set, and fail on anything else. To achieve more general usefulness, many many representative images are required.

Cheers,
Bart

Bart, going back to reply 16, while I did mention "unscrambling the omelette" (or is it omelet?) I also said, more meaningfully: ".....reverse-engineer or parse what is an already heavily compromised data base into anything resembling what the original raw data....". I think we're on the same page if you interpret my use of the term reverse-engineer to your term "bake again with other/better raw ingredients". However in this case, there are no other/better raw ingredients because those were all trashed with the conversion to JPEG. One is only working with "JPEG ingredients", to pursue the analogy. Now, you are explaining to us how you think they can parlay those JPEG ingredients into better output. I'm not positioned to argue it's impossible, but I'll remain agnostic about how likely and how useful until I see it convincingly demonstrated in this particular case by an expert  commercially disinterested third party; the O/P has not accomplished that. However, no matter how it's done and no matter how convincingly it can be demonstrated for a range of images processed and magnified to various extents, the fact remains - purely by definition - that a DNG containing rendered pixels is not RAW. Period. So calling it JPEG to RAW is nonsensical and misleading before we get into the technicalities of what A.I can or cannot do. Therefore, my bottom line to this point is that what the O/P showed doesn't begin to demonstrate whatever utility the application may have beyond the one or two samples he/she shows, and the premise from TOPAZ makes no technical sense.
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 03:40:05 pm
But they do create RAW file - DNG

"DNG is also considered to be a RAW image file. It is Adobe’s proprietary image standard that was created to store image data in a generic, highly-compatible format, unlike RAW files that have specific formats based on manufacturer and camera type. Although DNG was invented by Adobe and is supported in all Adobe applications, there are other camera manufacturers such as Leica, Hasselblad and Pentax that adopted this standard and use it in their cameras as their native and supported RAW file format."

So 3 major manufacturers actually use DNG as their RAW files.

https://expertphotography.com/dng-file-vs-raw-file/

sorry FOUR.

I think you are misunderstanding something a bit subtle but very important.

DNG is an Adobe open raw file format (means Digital Negative Graphic) that Thomas Knoll spearheaded in an effort to try to get camera manufacturers to standardize their raw file formats on this one specification. Several manufacturers have done so, hence your three or four camera makers who use the DNG format as their default raw file format.

Now here's where it gets a bit subtle: it is also possible to use the DNG specification for encapsulating JPEG or TIFF files, however this does not make them raw files. They are fully rendered pixel-based image files with a DNG "hat" on them. They are not raw. Raw data is undemosaiced/unrendered, whereas pixel-based data is. I think this is where you are getting confused. Because raw data can only come from the camera at the time of making the photograph (as a camera maker you should know this), nothing you do to a JPEG after the fact turns it back into a raw file - even putting a DNG hat on it and calling it a DNG file.
Title: Re: JPEG or RAW?
Post by: Alan Klein on February 08, 2019, 03:54:50 pm
You can't make a silk purse from a sow's ear.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 03:57:32 pm
Selective quoting of only part of my answer, only demonstrates unwillingness to understand what A.I. is about.
So with A.I (or not), Topaz converts JPEG to raw? Yes or no?
Title: Re: JPEG or RAW?
Post by: Telecaster on February 08, 2019, 04:19:47 pm
I think the heat of this discussion could be dialed down if the notion of returning processed data (JPEG) to an *unprocessed state (Raw) were abandoned, and instead we focused on whether or not the Topaz software actually increases the quality (at least tonally but maybe spatially as well) of the source JPGs. Like Bart says, you can't unbreak an egg…but some methods of attempting this may get you closer than others, and the amount of "closer" may have actual value. Taking shots at hype is easy. But does the Topaz process do any good? That's the question I'd be interested in.

-Dave-

*Relatively unprocessed, that is.
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 04:25:27 pm
You can't make a silk purse from a sow's ear.

https://libraries.mit.edu/archives/exhibits/purse/

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 04:27:01 pm
So with A.I (or not), Topaz converts JPEG to raw? Yes or no?

42

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 04:30:48 pm
42

Cheers,
Bart
You must have been watching Attorney General Matt Whitaker today, yes or no answers should not be so difficult to answer.
Title: Re: JPEG or RAW?
Post by: Alan Klein on February 08, 2019, 04:33:15 pm
https://libraries.mit.edu/archives/exhibits/purse/ (https://libraries.mit.edu/archives/exhibits/purse/)

Cheers,
Bart

I wonder if they tried tasting the purses???  :)
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 04:43:04 pm
https://libraries.mit.edu/archives/exhibits/purse/

Cheers,
Bart

This is a most interesting story Bart - thanks for the reference; but on reading it, it struck me that "silk" has a specific etiology - it is made from the salivary glands of silk worms. In fact I saw the whole manufacturing process from the worm to the finished product at a silk mill in Thailand. So very much like true Cognac can only come from the Cognac region of France, true silk can only come from the silk worm, and RAW files can only come from a digital camera so enabled, not a JPEG. So the obvious answer to Andrew's question isn't "42", it's NO.
Title: Re: JPEG or RAW?
Post by: TomFrerichs on February 08, 2019, 05:28:19 pm
I think the heat of this discussion could be dialed down if the notion of returning processed data (JPEG) to an *unprocessed state (Raw) were abandoned, and instead we focused on whether or not the Topaz software actually increases the quality (at least tonally but maybe spatially as well) of the source JPGs. Like Bart says, you can't unbreak an egg…but some methods of attempting this may get you closer than others, and the amount of "closer" may have actual value. Taking shots at hype is easy. But does the Topaz process do any good? That's the question I'd be interested in.

-Dave-

*Relatively unprocessed, that is.

Thank you. You have clearly stated the question I'd like to have answered, too. 

Before I get the standard comments...
1. The name is misleading, but the name has nothing to do with how it actually performs. If your objection is based solely on how it's being marketed, then you have nothing new to add. You aren't addressing the question.
2. I agree starting with a raw file is preferred to trying to clean up a JPEG.
There are circumstances where it is impossible to get an original raw file. Some devices do not supply a raw file, or the original photographer may not have chosen that option. If you are fortunate enough to never encounter that problem, then this application is not for you. You don't even need to know that it exists, and the answer to Dave's question has no value to you.

It may be possible to duplicate the action of this application using Photoshop or other image editing software. This has not been established as far as I know. However, even if it is possible, does this application make it easier or faster to obtain the same results. If it does, then it may still have value for certain users.  Though not those who don't ever have to "polish turds."

TG Frerichs


Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 07:30:07 pm
This is a most interesting story Bart - thanks for the reference; but on reading it, it struck me that "silk" has a specific etiology - it is made from the salivary glands of silk worms. In fact I saw the whole manufacturing process from the worm to the finished product at a silk mill in Thailand. So very much like true Cognac can only come from the Cognac region of France, true silk can only come from the silk worm, and RAW files can only come from a digital camera so enabled, not a JPEG. So the obvious answer to Andrew's question isn't "42", it's NO.

Hi Mark,

As you have correctly understood, it totally depends on one's definition of "Raw", or RAW. ;)

BTW, I'm glad that the result is not merely a 12-14 bit single channel per pixel Bayer Mosaic, or a 4-2-2 (or worse) lossy compressed file.
Instead, as long as the A.I. has found better alternative local features (which is the goal), I'm happy with a much more robust 48-bit per pixel RGB file.

The output quality is determined by the A.I., not whether it becomes an intermediate Raw.

Cheers,
Bart

Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 07:35:25 pm
Hi Mark,

As you have correctly understood, it totally depends on one's definition of "Raw", or RAW. ;)

BTW, I'm glad that the result is not merely a 12-14 bit single channel per pixel Bayer Mosaic, or a 4-2-2 (or worse) lossy compressed file.
Instead, as long as the A.I. has found better alternative local features (which is the goal), I'm happy with a much more robust 48-bit per pixel RGB file.

Cheers,
Bart

The future is already here:

https://www.theverge.com/2018/12/17/18144356/ai-image-generation-fake-faces-people-nvidia-generative-adversarial-networks-gans
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 07:35:30 pm
Hi Mark,
As you have correctly understood, it totally depends on one's definition of "Raw", or RAW. ;)
https://www.adobe.com/digitalimag/pdfs/understanding_digitalrawcapture.pdf (https://www.adobe.com/digitalimag/pdfs/understanding_digitalrawcapture.pdf)
A raw file is a record of the data captured by the sensor. While there are many different ways of encoding this raw sensor data into a raw image file, in each case the file records the unprocessed sensor data.

Read on, doesn't matter if you want to use RAW or raw; what raw image data is, is pretty established and understood by many but not all.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 07:38:11 pm
The output quality is determined by the A.I., not whether it becomes an intermediate Raw.
Intermediate Raw, seriously? Do provide a lick of evidence that's occurring at any state of this products processing of JPEG data.
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 07:39:05 pm
https://www.adobe.com/digitalimag/pdfs/understanding_digitalrawcapture.pdf (https://www.adobe.com/digitalimag/pdfs/understanding_digitalrawcapture.pdf)
A raw file is a record of the data captured by the sensor. While there are many different ways of encoding this raw sensor data into a raw image file, in each case the file records the unprocessed sensor data.

Read on, doesn't matter if you want to use RAW or raw; what raw image data is, is pretty established and understood by many but not all.

And it still says nothing about image quality. A Raw turd remains a turd.

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 07:40:51 pm
Intermediate Raw, seriously? Do provide a lick of evidence that's occurring at any state of this products processing of JPEG data.

If you think that a Raw file is the final result instead of an intermediate result, dream on.

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 07:43:46 pm
And it still says nothing about image quality. A Raw turd remains a turd.

Cheers,
Bart

Not after I've polished it! ;)
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 07:45:26 pm
If you think that a Raw file is the final result instead of an intermediate result, dream on.

Cheers,
Bart

We're getting into semantics here, but a raw file is a final result of a raw capture process. But it is not the final result of making the photograph. The next stage is post-capture processing. Final is final. No matter what you do with the raw file in a post-capture processing context, with ANY raw processors I know about, the raw file remains completely unaffected.
Title: Re: JPEG or RAW?
Post by: albytastic on February 08, 2019, 07:48:48 pm
We're getting into semantics here, but a raw file is a final result of a raw capture process. But it is not the final result of making the photograph. The next stage is post-capture processing. Final is final. No matter what you do with the raw file in a post-capture processing context, with ANY raw processors I know about, the raw file remains completely unaffected.

Wanna bet? - use exiftool on it to strip out the metadata and see how unchanged it is!

But make a copy first!
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 07:49:34 pm
And it still says nothing about image quality. A Raw turd remains a turd.

Cheers,
Bart
No. Raw data has to be rendered. That rendering could be a turd. But the point you've made is rather pointless and seems a digression on what the product you're so fond of actually does, case in point:
Quote
If you think that a Raw file is the final result instead of an intermediate result, dream on.
As I suspected, your assumption this product produces a so called 'intermediary raw' is pure supplication and like my TIFF you were going to reverse engineer to a raw, simply another assumption. It's getting more difficult to take you seriously when you can't answer simple yes or no answers about the product you are cheerleading actually does, then you suggest wildly there might be some intermediary raw.
Does it convert JPEG into raw at any point of the processing, yes or no? Seems the obvious answer is no. If yes, prove it does, provide outside reference from Topaz it does and don't wiggle around suggesting there's a differing interpretation of what raw data is, or if it's written as raw or RAW.
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 07:50:04 pm
We're getting into semantics here, but a raw file is a final result of a raw capture process. But it is not the final result of making the photograph.

Correct, which is why I called it an intermediate, not the final goal.

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 07:50:39 pm
We're getting into semantics here, but a raw file is a final result of a raw capture process. But it is not the final result of making the photograph. The next stage is post-capture processing. Final is final. No matter what you do with the raw file in a post-capture processing context, with ANY raw processors I know about, the raw file remains completely unaffected.
A tactic to divert from facts that don't fit their assumptions I fear.
Title: Re: JPEG or RAW?
Post by: faberryman on February 08, 2019, 07:51:04 pm
If you think that a Raw file is the final result instead of an intermediate result, dream on.
How does that pertain to the question asked?
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 07:54:54 pm
I think it's time to call a truce - or even an armistice - on whether that application produces raw files. It doesn't, and Topaz is remiss in its representation of the product.

The real debate is whether whatever "A.I." Topaz uses in its application is good enough to ingest a JPEG file and output a result that has equivalent photographic quality to a well-processed true raw file. The fact is that we don't know the answer to that question first hand because it would seem that none of the contributors to this thread other than the OP have tested it, and the O/P's testing seems very limited. That is why I brought in the message from Tim Grey, so we have his word, which I take seriously, but not his test results. So the jury is out on what this application can do with what kind of JPEG images under what processing requirements and objectives. I don't think there's much point discussing it further without additional, carefully developed first-hand evidence. BTW, yours truly will not be doing that.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 07:57:09 pm
Correct, which is why I called it an intermediate, not the final goal.

Cheers,
Bart
Of course, you'll provide proof that an intermediate raw is produced in the product, yes?
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 07:57:30 pm
Wanna bet? - use exiftool on it to strip out the metadata and see how unchanged it is!

But make a copy first!

Read my post - I was talking about post-capture processing, not declaring war on the file.
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 07:58:51 pm
The real debate is whether whatever "A.I." Topaz uses in its application is good enough to ingest a JPEG file and output a result that has equivalent photographic quality to a well-processed true raw file.

Well summarized.

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 07:59:25 pm
Wanna bet? - use exiftool on it to strip out the metadata and see how unchanged it is!

But make a copy first!
Will not alter the raw data one bit. But then you incorrectly assume, after many outside references, that a DNG always equals raw (wrong) and that converting a JPEG to DNG makes it raw (wrong again).
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 08:00:38 pm
Well summarized.

Cheers,
Bart
Yes, and the answer is, it cannot.
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 08:02:04 pm
Correct, which is why I called it an intermediate, not the final goal.

Cheers,
Bart

Oops - you said "intermediate raw" in post 45, not intermediate goal, and we've been discussing whether the concept of intermediate raw makes sense, so let's not shift the basis", otherwise the discussion gets confusing. There is a difference between "intermediate raw" which doesn't exist and "intermediate goal" which can well exist.
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 08:02:42 pm
Well summarized.

Cheers,
Bart

Whew! Thank you -:)
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 08:04:58 pm
Yes, and the answer is, it cannot.

Cannot? What law of physics prohibits the possibility to improve the final image?

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 08:06:12 pm
Whew! Thank you -:)
Doesn't change the new goal posts moved about this product, now, the assumption there is an intermediate raw. I didn't make it Bart, this is your exact writings.

The output quality is determined by the A.I., not whether it becomes an intermediate Raw.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 08:08:51 pm
Cannot? What law of physics prohibits the possibility to improve the final image?

Cheers,
Bart
The same physics you admitted to, with the cake and omelet analogy.
Try to be consistent and please, try to prove what you state, otherwise it seems just an assumption.
The product doesn't convert a JPEG into raw or RAW if you must. It therefore cannot produce the same results from utterly different data.
But maybe, maybe you can prove there is a conversion to raw data, intermittent or otherwise. In all the posts about this product from day one, you haven't. Kind of telling IMHO.
You have the floor. But will you use it to back up what you state we should believe?

As I said Mark, it is not possible to unscramble the omelet, or unbake Andrew's carrot cake.
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 08:10:12 pm
Doesn't change the new goal posts moved about this product, now, the assumption there is an intermediate raw. I didn't make it Bart, this is your exact writings.

OK, but I was trying to get us all beyond this kind of stuff in Reply #58. I.O.W. - isn't it beaten to death for the time being?  :-)
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 08:11:56 pm
OK, but I was trying to get us all beyond this kind of stuff in Reply #58. I.O.W. - isn't it beaten to death for the time being?  :-)
It can be settled when those who insist the product is producing raw from JPEG can prove it. Just writing that it does, doesn't make it so.
Let's see what Bart and other's come up with.
Otherwise, why perpetrate the lie from Topaz?
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 08:19:38 pm
OK, but I was trying to get us all beyond this kind of stuff in Reply #58. I.O.W. - isn't it beaten to death for the time being?  :-)

It's beaten to Raw ;) pulp. Good luck trying to demosaic that.

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 08:23:18 pm
It's beaten to Raw ;) pulp. Good luck trying to demosaic that.

Cheers,
Bart

Would it be about as challenging as reconstructing your omelette (or omelet)?  :)
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 08:26:47 pm
Would it be about as challenging as reconstructing your omelette (or omelet)?  :)

Maybe even harder to achieve ...  :-\

It would take real intelligence, not the Artificial kind.

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: nirpat89 on February 08, 2019, 08:28:38 pm
Reading this thread, I am getting dizzy and hungry at the same time.... :)
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 08, 2019, 08:30:07 pm
Reading this thread, I am getting dizzy and hungry at the same time.... :)

LOL

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: Slobodan Blagojevic on February 08, 2019, 08:33:00 pm
... The real debate is whether whatever "A.I." Topaz uses in its application is good enough to ingest a JPEG file and output a result that has equivalent photographic quality to a well-processed true raw file...

Correct, Mark, that is absolutely the crux of the matter.

It shouldn't be too hard to design that test:

1. set the camera to raw+jpeg
2. shoot several scenes under varied lighting conditions, e.g., deep shadows, bright highlights, light source of various temperature, etc.
3. process raw files to taste
4. send the jpeg to J2R
5. further process it to taste
6. compare the results

That should determine if J2R is capable of:

1. recovering the shadow/highlight detail and to which extent, relative to raw
2. correcting white balance and to which extent, relative to raw

Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 08:38:08 pm
Maybe even harder to achieve ...  :-\

It would take real intelligence, not the Artificial kind.

Cheers,
Bart
Harder or not possible as you indicated? I don't expect an answer as it's obvious for both. ;)
Title: Re: JPEG or RAW?
Post by: digitaldog on February 08, 2019, 08:40:37 pm
Correct, Mark, that is absolutely the crux of the matter.

It shouldn't be too hard to design that test:

1. set the camera to raw+jpeg
2. shoot several scenes under varied lighting conditions, e.g., deep shadows, bright highlights, light source of various temperature, etc.
3. process raw files to taste
4. send the jpeg to J2R
5. further process it to taste
6. compare the results

That should determine if J2R is capable of:

1. recovering the shadow/highlight detail and to which extent, relative to raw
2. correcting white balance and to which extent, relative to raw
Indeed! I suggested something similar but with an incorrect white balance which would make one ugly JPEG. Fix that with A.I. No takers. I found that telling.
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 08:58:02 pm
Correct, Mark, that is absolutely the crux of the matter.

It shouldn't be too hard to design that test:

1. set the camera to raw+jpeg
2. shoot several scenes under varied lighting conditions, e.g., deep shadows, bright highlights, light source of various temperature, etc.
3. process raw files to taste
4. send the jpeg to J2R
5. further process it to taste
6. compare the results

That should determine if J2R is capable of:

1. recovering the shadow/highlight detail and to which extent, relative to raw
2. correcting white balance and to which extent, relative to raw

Are you volunteering to do this for us?
Title: Re: JPEG or RAW?
Post by: Slobodan Blagojevic on February 08, 2019, 09:03:14 pm
Are you volunteering to do this for us?

Ha! I already ruined one Netflix evening trying to download the damn program. It is like half a gygabite and slowed my Netflix to a crawl. Worse, it froze it every 5 minutes 😉
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 08, 2019, 09:46:03 pm
Ha! I already ruined one Netflix evening trying to download the damn program. It is like half a gygabite and slowed my Netflix to a crawl. Worse, it froze it every 5 minutes 😉

AH, good (I mean not the trouble, but the fact you are downloading it). Once this painful download completes and assuming it installs, should we look forward to your take on the results?
Title: Re: JPEG or RAW?
Post by: Slobodan Blagojevic on February 08, 2019, 10:15:30 pm
... should we look forward to your take on the results?

Maybe, but I can't promise. I am usually not the guy to shoot brick walls and test targets for fun.
Title: Re: JPEG or RAW?
Post by: fdisilvestro on February 09, 2019, 07:36:10 am
The real debate is whether whatever "A.I." Topaz uses in its application is good enough to ingest a JPEG file and output a result that has equivalent photographic quality to a well-processed true raw file.

My answer to that is NO, the output of Topaz Jpeg to Raw is not at the level of a true raw file, understanding "true raw" as the original raw from which the JPEG was produced.

I would change the debate to:

whether whatever "A.I." Topaz uses in its application is good enough to ingest a JPEG file and output a result that has better quality than any other current way to edit a JPEG file. (and I don't have the answer to this)

Is the output a raw file? First consider that what this application claims to do is not a simple file conversion, but apply A.I. and try to recreate missing information that is coherent to the JPEG file, not necessarily to the real scene that was photographed. As a reference, consider the Google photos A.I. ability to colorize B&W images: would the colors be accurate to the original scene? Not necessarily, but the color information was "recreated" and is believable.

Having said that, the output file has some characteristics of a raw file: Linear raw, 16 bit per channel, uncompressed data, linear tone curve; while other characteristics are as in rendered files: color space encoded (ProPhotoRGB).

If you consider the processing of an image from "true raw" to rendered in tiff as a series of steps, I would say that the output from Topaz JPEG to RAW is just one step short of TIFF, so even if you can open the DNG in any raw processing application, there are some limitations to what you can do. For instance, in Lightroom/ACR there is no difference processing these DNGs instead of TIFFs. You cannot apply a DCP profile nor adjust white balance as in a true raw.

One more observation: JPEG to RAW tries to maximise detail by sharpening aggressively, so if you open the DNG with Capture one with its default settings the image will look really oversharpened. This is another big difference to "true raws", which in my experience are softer than processed images, especially if you use a camera with low pass filter.

Following the results of a test of recovering shadows (you may call it contrived)
Photo taken with a Nikon D800 in Raw + Jpeg mode, 14 bit raw and large, fine, optimal quality JPEG. Underexposed 6 stops.

The first image shows how the JPEG out of camera looks:

(https://www.frankdisilvestro.com.au/img/s/v-3/p3287601258-4.jpg)

The following is what I could do with the "True raw" (NEF) file in LR:

(https://www.frankdisilvestro.com.au/img/s/v-3/p3287601260-4.jpg)

And what I was able to do with the DNG output from JPEG to RAW + LR

(https://www.frankdisilvestro.com.au/img/s/v-3/p3287601259-4.jpg)
Title: Re: JPEG or RAW?
Post by: nirpat89 on February 09, 2019, 08:36:32 am
My answer to that is NO, the output of Topaz Jpeg to Raw is not at the level of a true raw file, understanding "true raw" as the original raw from which the JPEG was produced.

I would change the debate to:

whether whatever "A.I." Topaz uses in its application is good enough to ingest a JPEG file and output a result that has better quality than any other current way to edit a JPEG file. (and I don't have the answer to this)

Is the output a raw file? First consider that what this application claims to do is not a simple file conversion, but apply A.I. and try to recreate missing information that is coherent to the JPEG file, not necessarily to the real scene that was photographed. As a reference, consider the Google photos A.I. ability to colorize B&W images: would the colors be accurate to the original scene? Not necessarily, but the color information was "recreated" and is believable.

Having said that, the output file has some characteristics of a raw file: Linear raw, 16 bit per channel, uncompressed data, linear tone curve; while other characteristics are as in rendered files: color space encoded (ProPhotoRGB).

If you consider the processing of an image from "true raw" to rendered in tiff as a series of steps, I would say that the output from Topaz JPEG to RAW is just one step short of TIFF, so even if you can open the DNG in any raw processing application, there are some limitations to what you can do. For instance, in Lightroom/ACR there is no difference processing these DNGs instead of TIFFs. You cannot apply a DCP profile nor adjust white balance as in a true raw.

One more observation: JPEG to RAW tries to maximise detail by sharpening aggressively, so if you open the DNG with Capture one with its default settings the image will look really oversharpened. This is another big difference to "true raws", which in my experience are softer than processed images, especially if you use a camera with low pass filter.

Following the results of a test of recovering shadows (you may call it contrived)
Photo taken with a Nikon D800 in Raw + Jpeg mode, 14 bit raw and large, fine, optimal quality JPEG. Underexposed 6 stops.


What do you get if you took the jpg out of the camera (image #1,) convert to Prophoto 16 bit tiff, put it in LR and/or PS, and give it the best treatment possible.  How would it compare with the JPG to RAW + LR output (image #3)?   How big of a difference?
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 09, 2019, 08:45:08 am
My answer to that is NO, the output of Topaz Jpeg to Raw is not at the level of a true raw file, understanding "true raw" as the original raw from which the JPEG was produced.

I would change the debate to:

whether whatever "A.I." Topaz uses in its application is good enough to ingest a JPEG file and output a result that has better quality than any other current way to edit a JPEG file. (and I don't have the answer to this)

..............

(https://www.frankdisilvestro.com.au/img/s/v-3/p3287601259-4.jpg)

Given the context set by the Topaz claim, I wouldn't change my question; rather I would call your's another equally valid question.

The three examples you show are very informative in respect of my question, clearly showing that there is acceptable photographic adjustment potential, at least in difficult raw files, that far exceed what one can do with a JPEG in the Topaz application. Again this is a sample of one, but nonetheless of indicative value.
Title: Re: JPEG or RAW?
Post by: Bart_van_der_Wolf on February 09, 2019, 09:43:02 am
What do you get if you took the jpg out of the camera (image #1,) convert to Prophoto 16 bit tiff, put it in LR and/or PS, and give it the best treatment possible.  How would it compare with the JPG to RAW + LR output (image #3)?   How big of a difference?

Exactly, that's the test to perform. Does this application allow to produce better results from JPEG input than a straight JPEG edit? Is the J2R preprocessed JPEG more robust for postprocessing than the original JPEG?

I think we can only judge that for general subjects when a better A.I. model is trained, with less noise suppression and less sharpening applied than the 'Normal' setting does. I'll repeat, the same was the case with Gigapixel A.I., the initial models were too aggressive in noise suppression, but the current 'None' setting does very well for reasonable quality input.

Cheers,
Bart
Title: Re: JPEG or RAW?
Post by: TomFrerichs on February 09, 2019, 11:50:02 am

The real debate is whether whatever "A.I." Topaz uses in its application is good enough to ingest a JPEG file and output a result that has equivalent photographic quality to a well-processed true raw file.
 …
Mark, I am aware that several posters have suggested that you have correctly summarized the question. I don't think you have, though.

The (admittedly overblown) claims made by Topaz marketing say: Use machine learning to convert JPEG to high-quality RAW for better editing. Prevent banding, remove compression artifacts, recover detail, and enhance dynamic range. The application is supposed to process a JPEG file into a form that is more amenable to further processing by reducing artifacts, recover highlight and shadow detail, etc. Nowhere is there a claim that it will result in a "like raw" image, only that the output would have fewer artifacts, better detail in highlight and shadow areas, and be less susceptible to  additional editing problems like posterization.

I think the proper question to ask, based upon what Topaz claims, is this: Does a JPEG file processed in this application exhibit fewer compression artifacts, have recovered or enhanced detail, and is more tolerant of additional editing compared to the original.

TG Frerichs
Title: Re: JPEG or RAW?
Post by: faberryman on February 09, 2019, 11:54:24 am
The (admittedly overblown) claims made by Topaz marketing say: Use machine learning to convert JPEG to high-quality RAW for better editing. Prevent banding, remove compression artifacts, recover detail, and enhance dynamic range. The application is supposed to process a JPEG file into a form that is more amenable to further processing by reducing artifacts, recover highlight and shadow detail, etc. Nowhere is there a claim that it will result in a "like raw" image, only that the output would have fewer artifacts, better detail in highlight and shadow areas, and be less susceptible to  additional editing problems like posterization.
What part of their saying "Use machine learning to convert JPEG to high quality RAW" don't you understand. It is that false claim that has people's hackles up. You gloss over it.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 09, 2019, 12:09:35 pm
What part of their saying "Use machine learning to convert JPEG to high quality RAW" don't you understand. It is that false claim that has people's hackles up. You gloss over it.


What's more, they are VERY clear about what they claim:


Edit JPEG as if you shot it in RAW

As IF you shot it in raw!
And yet, those of us who shoot raw can easily see that if (as an example), one's WB is utterly wrong, it has zero effect on processing that raw, yet where and when will we see an incorrectly captured JPEG with respect to WB be fixed as if it were shot in raw? ??? ?


The proof of the lie (again):


Title: Re: JPEG or RAW?
Post by: TomFrerichs on February 09, 2019, 12:29:34 pm
What part of their saying "Use machine learning to convert JPEG to high quality RAW" don't you understand. It is that false claim that has people's hackles up. You gloss over it.

If you have been reading the entire thread, you would have noticed that in reply #58, Mr. Segal began with: I think it's time to call a truce - or even an armistice - on whether that application produces raw files. It doesn't, and Topaz is remiss in its representation of the product. I didn't include that in my quotation of his post in the interest of saving space.

I don't think anyone here will disagree that there's no way any program can create an original camera sensor capture (if that's the meaning of a raw) from a JPEG. Topaz marketing is nonsense.

Yes, I glossed over that. At this point I think we want an answer to Mr. Segal's question posed in #58 or my restatement. In other words, it's not a question of me ignoring the marketing idiocy but of getting past it to ask other questions.

As I was typing this, I noticed that Andrew was replying.  In response...I don't care how they do it, with artificial intelligence or fairy dust, or how they market it. They can claim that they use alien technology found on a crashed spaceship in Roswell, New Mexico. I don't give a damn if their marketing is dodgy. 

All I really want to know is this: Does the program output a file that can be edited better than the original JPEG? Can you answer that question by direct experience?

TG Frerichs

Title: Re: JPEG or RAW?
Post by: digitaldog on February 09, 2019, 12:51:34 pm
Quote
I don't think anyone here will disagree that there's no way any program can create an original camera sensor capture (if that's the meaning of a raw) from a JPEG. Topaz marketing is nonsense
Not so, read the OP's confused idea about DNG. Try reading a lack of admission from some, after multiple questions with yes or no answers IF the product converts JPEG to raw.
Title: Re: JPEG or RAW?
Post by: albytastic on February 09, 2019, 01:08:11 pm
If you have been reading the entire thread, you would have noticed that in reply #58, Mr. Segal began with: I think it's time to call a truce - or even an armistice - on whether that application produces raw files. It doesn't, and Topaz is remiss in its representation of the product. I didn't include that in my quotation of his post in the interest of saving space.

I don't think anyone here will disagree that there's no way any program can create an original camera sensor capture (if that's the meaning of a raw) from a JPEG. Topaz marketing is nonsense.

Yes, I glossed over that. At this point I think we want an answer to Mr. Segal's question posed in #58 or my restatement. In other words, it's not a question of me ignoring the marketing idiocy but of getting past it to ask other questions.

As I was typing this, I noticed that Andrew was replying.  In response...I don't care how they do it, with artificial intelligence or fairy dust, or how they market it. They can claim that they use alien technology found on a crashed spaceship in Roswell, New Mexico. I don't give a damn if their marketing is dodgy. 

All I really want to know is this: Does the program output a file that can be edited better than the original JPEG? Can you answer that question by direct experience?

TG Frerichs

I already have - on my Flickr site - and at the present moment am processing several hundred JPEGs using this program and every one is producing better results than I had before.
Title: Re: JPEG or RAW?
Post by: Ray Harrison on February 09, 2019, 01:19:39 pm
I already have - on my Flickr site - and at the present moment am processing several hundred JPEGs using this program and every one is producing better results than I had before.

Can you be specific on what you mean by "better than before"? Are you processing any of them afterwards?
Title: Re: JPEG or RAW?
Post by: Alan Goldhammer on February 09, 2019, 01:27:52 pm
I already have - on my Flickr site - and at the present moment am processing several hundred JPEGs using this program and every one is producing better results than I had before.
While this might be a true statement for JPEGs it is a totally false statement when one is talking about RAW files.  An 8 bit file sacrifices a considerable amount of data relative to a 14 bit file (assuming correct WB of the two).  The fair trial is to capture a RAW and high def JPEG on the same exposure.  Do the AI transformation on the JPEG and then compare how the two images can be processed.  My Nikon Z 6 can do this type of capture though I have no intent on doing any comparative trials as I can't be bothered to download another piece of software whose marketing claim bypasses the normal laws of physics.  I know that Thom Hogan has looked at both RAW and JPEGs in his new manual on the new Nikon Z cameras and has shown that there is a difference in quality, particularly around edges.
Title: Re: JPEG or RAW?
Post by: albytastic on February 09, 2019, 02:02:33 pm
Can you be specific on what you mean by "better than before"? Are you processing any of them afterwards?

Of course I am, and a few previous ones are on my Flickr site:

https://www.flickr.com/photos/20926615@N05/albums/72157678279709128


Title: Re: JPEG or RAW?
Post by: albytastic on February 09, 2019, 02:06:07 pm
While this might be a true statement for JPEGs it is a totally false statement when one is talking about RAW files.  An 8 bit file sacrifices a considerable amount of data relative to a 14 bit file (assuming correct WB of the two).  The fair trial is to capture a RAW and high def JPEG on the same exposure.  Do the AI transformation on the JPEG and then compare how the two images can be processed.  My Nikon Z 6 can do this type of capture though I have no intent on doing any comparative trials as I can't be bothered to download another piece of software whose marketing claim bypasses the normal laws of physics.  I know that Thom Hogan has looked at both RAW and JPEGs in his new manual on the new Nikon Z cameras and has shown that there is a difference in quality, particularly around edges.

So, in other words, and like a lot of people, you condemn this program without even trying it out?

Seems a bit ridiculous to me.

Title: Re: JPEG or RAW?
Post by: digitaldog on February 09, 2019, 02:26:31 pm
So, in other words, and like a lot of people, you condemn this program without even trying it out?

Seems a bit ridiculous to me.
He condemns nothing and simply provides facts you refuse to accept. Like your inability to accept DNG may not equal raw! That seems ridiculous to many Here.
Title: Re: JPEG or RAW?
Post by: Mark D Segal on February 09, 2019, 02:27:10 pm
Mark, I am aware that several posters have suggested that you have correctly summarized the question. I don't think you have, though.

The (admittedly overblown) claims made by Topaz marketing say: Use machine learning to convert JPEG to high-quality RAW for better editing. Prevent banding, remove compression artifacts, recover detail, and enhance dynamic range. The application is supposed to process a JPEG file into a form that is more amenable to further processing by reducing artifacts, recover highlight and shadow detail, etc. Nowhere is there a claim that it will result in a "like raw" image, only that the output would have fewer artifacts, better detail in highlight and shadow areas, and be less susceptible to  additional editing problems like posterization.

I think the proper question to ask, based upon what Topaz claims, is this: Does a JPEG file processed in this application exhibit fewer compression artifacts, have recovered or enhanced detail, and is more tolerant of additional editing compared to the original.

TG Frerichs

The Topaz problem starts with the first phrase you recited: "Use machine learning to convert JPEG to high-quality raw for.....". Machine learning cannot do this, so it is a false claim. Their claim is also that their application will give similar results to a well-processed raw file from a JPEG file. That was the basis of my definition of the issue, and I remain comfortable thinking it's valid to put it that way. Furthermore, Frank diSilvestro's three image demonstration above I think answers the question, in the sense that he clearly shows the superiority of just processing a raw file in Lr.
Title: Re: JPEG or RAW?
Post by: Ray Harrison on February 09, 2019, 03:19:04 pm
Of course I am, and a few previous ones are on my Flickr site:

https://www.flickr.com/photos/20926615@N05/albums/72157678279709128

I guess I am not clear though on what you find better than before from looking at the images. What’s a good before and after image?

Sharpening and noise handling are available elsewhere, of course, and presumably if you are shooting jpeg, you’ve got your camera set to optimal jpeg output for the situation and include some level of sharpening, for example. When I tried this application on a couple of images that were challenging (shadow detail needed bringing out) I found it was over sharpened and the colors were off. If I had to answer the question of “does the application produce a better way to edit a jpeg than other ways” my answer would be “no” for my use cases. It doesn’t obviously produce raw or really, even a “raw-like experience”. Raw shouldn’t be part of its name but I think we all agree to that. It does produce a sharper seeming image if your images aren’t sharp, though it has to be dialed down in my case. I find it’s color treatment off, perhaps pilot error on my part though. It quite obviously isn’t going to be able to add details in shadows that I could pull using a standard raw processing approach. I obviously won’t be rushing out to buy it since it adds no real value to my workflow. I respect that others find value.

I will try and post image results later this weekend.
Title: Re: JPEG or RAW?
Post by: Telecaster on February 09, 2019, 04:11:29 pm
This thread reminds me of Twitter: some folks are here for potentially interesting & useful discussion and/or info while others are here to get lit (or off) on outrage. Give the latter an opportunity to try a more positive tack…and they simply won't.  ::)

-Dave-
Title: Re: JPEG or RAW?
Post by: albytastic on February 09, 2019, 06:15:37 pm
I guess I am not clear though on what you find better than before from looking at the images. What’s a good before and after image?

Sharpening and noise handling are available elsewhere, of course, and presumably if you are shooting jpeg, you’ve got your camera set to optimal jpeg output for the situation and include some level of sharpening, for example. When I tried this application on a couple of images that were challenging (shadow detail needed bringing out) I found it was over sharpened and the colors were off. If I had to answer the question of “does the application produce a better way to edit a jpeg than other ways” my answer would be “no” for my use cases. It doesn’t obviously produce raw or really, even a “raw-like experience”. Raw shouldn’t be part of its name but I think we all agree to that. It does produce a sharper seeming image if your images aren’t sharp, though it has to be dialed down in my case. I find it’s color treatment off, perhaps pilot error on my part though. It quite obviously isn’t going to be able to add details in shadows that I could pull using a standard raw processing approach. I obviously won’t be rushing out to buy it since it adds no real value to my workflow. I respect that others find value.

I will try and post image results later this weekend.

Well if you looked at the several examples provided it reduced noise by an incredible amount, and also recovered detail and sharpened where appropriate.
As far as I'm concerned the NR properties alone make it worth the money I paid because it is so much easier than any other NR programs I've tried.
Title: Re: JPEG or RAW?
Post by: albytastic on February 09, 2019, 06:23:15 pm
He condemns nothing and simply provides facts you refuse to accept. Like your inability to accept DNG may not equal raw! That seems ridiculous to many Here.

Well why don't you try and find out for yourself if it improves the original JPEG.

Take a JPEG of a increased color gamut scene in JPEG.

Then, without doing anything else to that JPEG put it through J2R and check the results with your programs and compare it to the original JPEGs.

See if it does in fact increase the colour gamut.

I for one would be very interested in the results as I'm sure everyone here would.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 09, 2019, 06:30:23 pm
Well why don't you try and find out for yourself if it improves the original JPEG.
You don't listen or read very well. I don't have original JPEGs. I told you this at least two times. Like I told you twice that converting something to DNG doesn't necessarily make it raw. You still haven’t accepted these absolute facts.
The JPEGs I produce are from raw and need no further editing.
No, it can't increase color gamut. It can provide a larger color space container then you might futz around further attempting to polish a turd by increasing saturation. But I'm not ready to even attempt to explain those facts to you as you've been utterly unable to understand a much simpler concept about converting data into DNG yet.
Quote
I for one would be very interested in the results as I'm sure everyone here would.
Rather difficult to accept or believe since you refuse to accept simple facts such as what happens when you convert a JPEG into a DNG. Ain't raw data bud. Understand that yet?  :P
Title: Re: JPEG or RAW?
Post by: Ray Harrison on February 09, 2019, 07:30:57 pm
Well if you looked at the several examples provided it reduced noise by an incredible amount, and also recovered detail and sharpened where appropriate.
As far as I'm concerned the NR properties alone make it worth the money I paid because it is so much easier than any other NR programs I've tried.

OK. So your work flow is to shoot JPEG (without, apparently, using camera sharpening, white balance, noise reduction, etc) and then run it through this program to create your TIFF file and then spit it back out as a JPEG? Is this what you do normally or is this stuff you're doing on old files?

I'm seeing lots of sharpening for sure in your example images. Is the level of sharpening you get out of the application acceptable to you? Do you dial it back at all?

Title: Re: JPEG or RAW?
Post by: fdisilvestro on February 09, 2019, 08:02:01 pm
... what happens when you convert a JPEG into a DNG. Ain't raw data bud. ...

The approach of A.I. is not to convert (as you would do in LR), but to re-create data which is consistent with the output. Paraphrasing what Bart said a few posts ago, it is not trying to unscramble the egg, but having a taste of it and trying to recreate the recipe that would lead to the final plate (which a good chef would easily do, applying his knowledge and intelligence).

In an hypothetical scenario of having infinite computer power and infinite raw image and final jpeg samples under all imaginable cases, it would theoretically be possible to re-create (not convert) a "True raw" from  a jpeg file using A.I.

We are very far from there, but let's for a moment consider that this is a newborn baby that has to grow and learn, so let's not kill him yet.

What I think is the problem is that Topaz marketing went crazy and oversold the product and its capabilities. Maybe they thought that it was necessary to sell the product, but the risk is that it could backfire and die prematurely. For instance, this is a quote from the Topaz web site:

Quote
JPEG to RAW’s machine learning models expand the sRGB colorspace to ProPhoto RGB, which is even better than a regular RAW file!

This is nonsense taken to a whole new level, at least that's what I think. Why do that? Unless the target of this product is not the knowledgeable crowd

Now, back to the tests, In my previous posts I showed that a Jpeg to Raw DNG is not even close to a Raw file with extreme underexposure, here is the result of trying to edit the underexposed jpeg compared to the A.I. generated DNG (in my opinion one is bad, the other is worse)

Trying to get shadow detail with the DNG generated from JPEG to RAW:

(https://www.frankdisilvestro.com.au/img/s/v-3/p3287601259-4.jpg)

This with the original out of camera JPEG:

(https://www.frankdisilvestro.com.au/img/s/v-3/p3287601261-4.jpg)




Another test, where the product shows improvement is in reducing JPEG artifacts: This is a coparison of an original rendered NEF, a low quality (full size) JPEG and a DNG generated from the JPEG. The DNG shows a big reduction of jpeg artifacts, reduced banding in the sky, at the expense of creation of artificial detail as can be seen in the church windows.

-- Edit: The following are 100% crops --

Original image from NEF:

(https://www.frankdisilvestro.com.au/img/s/v-3/p3288184301-5.jpg)


JPEG full size, but low quality factor, (I'm not sure if the banding in the sky can be appreciated)

(https://www.frankdisilvestro.com.au/img/s/v-3/p3288184300-5.jpg)


Finally, the DNG out of Jpeg to Raw:

(https://www.frankdisilvestro.com.au/img/s/v-3/p3288184299-5.jpg)

IMHO this DNG is better than the JPEG but not better than the original.

I have made other tests with extremely off white balances, and no, you cannot get a DNG which can compare with the original raw. Some colors will be off, not matter what you do.

My conclusion: it is not a raw file, the application it is capable of creating a tiff file with better quality than the jpeg in some cases. Topaz is creating expectations that cannot be met, at least for now.
Title: Re: JPEG or RAW?
Post by: albytastic on February 09, 2019, 08:16:44 pm
OK. So your work flow is to shoot JPEG (without, apparently, using camera sharpening, white balance, noise reduction, etc) and then run it through this program to create your TIFF file and then spit it back out as a JPEG? Is this what you do normally or is this stuff you're doing on old files?

I'm seeing lots of sharpening for sure in your example images. Is the level of sharpening you get out of the application acceptable to you? Do you dial it back at all?

These are all old files, and the sharpening was me, normally the sharpening is less than that.

My normal monitor went kaput so I had to rely on an old one or my TV set.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 09, 2019, 08:21:32 pm
The approach of A.I. is not to convert (as you would do in LR), but to re-create data which is consistent with the output. Paraphrasing what Bart said a few posts ago, it is not trying to unscramble the egg, but having a taste of it and trying to recreate the recipe that would lead to the final plate (which a good chef would easily do, applying his knowledge and intelligence). =
I understand that's what they (Topaz) want to express. I understand what they say they are doing that's not true (Convert JPEG to Raw).


My comments were directed at the utter misunderstandings of the OP of this thread and the original where he stated, incorrectly, that conversion to DNG produces a raw. With or without A.I. that's simply just wrong. He posted in both locations this severe misunderstanding:

But they do create RAW file - DNG


"DNG is also considered to be a RAW image file. It is Adobe’s proprietary image standard that was created to store image data in a generic, highly-compatible format, unlike RAW files that have specific formats based on manufacturer and camera type. Although DNG was invented by Adobe and is supported in all Adobe applications, there are other camera manufacturers such as Leica, Hasselblad and Pentax that adopted this standard and use it in their cameras as their native and supported RAW file format."


So 3 major manufacturers actually use DNG as their RAW files.


https://expertphotography.com/dng-file-vs-raw-file/


sorry FOUR.
The "they' above is Topaz. He believes they convert JPEG to raw because a DNG is involved. He continues to apparently believe that DNG equals raw. At least he's yet to admit that's not always the case, that a conversion of JPEG to DNG, everywhere, doesn't produce raw data. Or that any conversion to DNG equals raw data as he's expressed in two threads. It is simply not always true. No wonder he accepts what Topaz claims about this product. As it's simply not true when Topaz converts a JPEG into a DNG. That A.I. may be used to improve further processing (seems sharpening is a big part of this process), it's not got a lick to do with raw data or processing raw data, DNG or otherwise. As we both agree. As do others. I'm not sure he understands that yet. As such, when he asks me "Well why don't you try and find out for yourself if it improves the original JPEG" despite having told him twice there's absolutely no reason for me to do so, or he states "for one would be very interested in the results as I'm sure everyone here would", it's simply disingenuous dialog based on his inability to yet accept what happens when data is converted to DNG. So I suggest he decide first, what really happens with differing conversions of data to DNG occurs before going any farther.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 09, 2019, 08:22:53 pm
These are all old files, and the sharpening was me, normally the sharpening is less than that.

My normal monitor went kaput so I had to rely on an old one or my TV set.
That explains the over sharpening which is rather ugly. At least on my NEC P271Q SpectraView color reference display system.
Title: Re: JPEG or RAW?
Post by: albytastic on February 09, 2019, 08:27:10 pm
You don't listen or read very well. I don't have original JPEGs. I told you this at least two times. Like I told you twice that converting something to DNG doesn't necessarily make it raw. You still haven’t accepted these absolute facts.
The JPEGs I produce are from raw and need no further editing.
No, it can't increase color gamut. It can provide a larger color space container then you might futz around further attempting to polish a turd by increasing saturation. But I'm not ready to even attempt to explain those facts to you as you've been utterly unable to understand a much simpler concept about converting data into DNG yet. Rather difficult to accept or believe since you refuse to accept simple facts such as what happens when you convert a JPEG into a DNG. Ain't raw data bud. Understand that yet?  :P

You may not have original JPEGs, but you have a camera capable of taking JPEGs surely?

I do understand about DNG, a similar thing happens when you put a 14 bit output from your camera into the 16 bit space of a TIFF or DNG file.

Or the 8 bit output of a JPEG into a 16 bit space.

What I wanted to know was there any difference in the final output from J2R compared to the original JPEG, in the colour gamut - which you claim can't be done.

I already know that J2R really does work in recovering details and in noise reduction, so whether or not it does that also is really just an interest, nothing more.

But if you're not willing to do it, ok no real problem.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 09, 2019, 08:34:05 pm
You may not have original JPEGs, but you have a camera capable of taking JPEGs surely?
Of course it does. And I can go out of my way to under expose anything, set the incorrect white balance and produce out of focus images. But I'm a professional photographer (at least I was, that's what I was paid to do, to feed my family). There's absolutely NO reason to shoot JPEGs when I can and do shoot raw. To futz with a product based on a lie.


Quote
I do understand about DNG, a similar thing happens when you put a 14 bit output from your camera into the 16 bit space of a TIFF or DNG file.

Or the 8 bit output of a JPEG into a 16 bit space

No, you don't understand about DNG, or bit depth. Or color gamut based on your writings. Or how to process images ideally by over sharpening them by doing so on a TV and then presenting them as being improvements.
You don't understand that very, very, very few devices actually capture 16-bit data. Or that few products actually edit 16-bit data (Photoshop for one doesn't). You probably believe that a 16-bit document is 'better' than a 12-bit document because it has 'more colors' (nope) or simply because it's a bigger value. Or that more bits means more DR.
You're not willing to listen to facts. It's why a few here are not taking you very seriously.
You still believe a DNG is always raw, no matter the source data prior to conversion? Yes or no? IF no, you're learning. That's at least some progress.  ???
Title: Re: JPEG or RAW?
Post by: albytastic on February 09, 2019, 09:08:23 pm
Of course it does. And I can go out of my way to under expose anything, set the incorrect white balance and produce out of focus images. But I'm a professional photographer (at least I was, that's what I was paid to do, to feed my family). There's absolutely NO reason to shoot JPEGs when I can and do shoot raw. To futz with a product based on a lie.


No, you don't understand about DNG, or bit depth. Or color gamut based on your writings. Or how to process images ideally by over sharpening them by doing so on a TV and then presenting them as being improvements.
You don't understand that very, very, very few devices actually capture 16-bit data. Or that few products actually edit 16-bit data (Photoshop for one doesn't). You probably believe that a 16-bit document is 'better' than a 12-bit document because it has 'more colors' (nope) or simply because it's a bigger value. Or that more bits means more DR.
You're not willing to listen to facts. It's why a few here are not taking you very seriously.
You still believe a DNG is always raw, no matter the source data prior to conversion? Yes or no? IF no, you're learning. That's at least some progress.  ???

Of course I understand about 16 bit data capture which is what I meant when I said about 8 bit and 12 bit.

And I don't believe that DNG is always raw, bit if camera manufacturers are using DNG as raw then DNG can be raw.

And I do understand about DR and bits, certainly not what you ascribed to me.

And despite what you may think I understand very well about bits, A/D conversion and what it achieves in the camera, and a great deal more.

I also understand about politeness.
Title: Re: JPEG or RAW?
Post by: digitaldog on February 09, 2019, 09:25:17 pm
Of course I understand about 16 bit data capture which is what I meant when I said about 8 bit and 12 bit.

And I don't believe that DNG is always raw, bit if camera manufacturers are using DNG as raw then DNG can be raw.

And I do understand about DR and bits, certainly not what you ascribed to me.

And despite what you may think I understand very well about bits, A/D conversion and what it achieves in the camera, and a great deal more.

I also understand about politeness.
1. If it's raw, it's raw, not maybe.
2. Bit depth isn't related to DR. More bits doesn't mean more DR.
You think a "16-bit" Prophoto image of a gray card has more colors than an sRGB 8-bit of a sunset?
You think humans can see 16.7 million colors let alone 65 million colors?
You think numbers are colors?
Title: Re: JPEG or RAW?
Post by: azmike on February 09, 2019, 10:52:14 pm
This discussion and the associated "Topaz Edit JPG to Raw" are just sad. Andrew Rodney is just full stop correct -you can't remake the raw from the jpg.  The raw data file is a 2-dimensional file of the voltages values from the sensor array.  Nothing more. It ain't a photo yet.  A jpg development of that file goes through numerous non-deterministic and destructive steps.  Topaz should have said they have a software that makes jpg's better, even "AI" better, but to suggest that it recreates the raw is not truthful.  AI will not trump optics, physics and thermodynamics (and a few more fundamentals).  The numerous discussion participants that are posting:  "yeah, but look at my comparison shots"  are living in a world enamored of smart-phone magic where computational wizardry (and marketing) obfuscate the basics of photography. 

I have been a forum member for over a decade; met Andrew at a printing seminar in Chandler; Rodney and others (MR, JS)  have been pioneers in explaining the transition from film to digital.  Style aside, they know their stuff.  Try to learn from them before their patience gives out.

Mike Coffey
Prescott, AZ
Title: Re: JPEG or RAW?
Post by: albytastic on February 10, 2019, 03:05:35 am
1. If it's raw, it's raw, not maybe.
2. Bit depth isn't related to DR. More bits doesn't mean more DR.
You think a "16-bit" Prophoto image of a gray card has more colors than an sRGB 8-bit of a sunset?
You think humans can see 16.7 million colors let alone 65 million colors?
You think numbers are colors?

And that is your idea of a discussion?

It's quite obvious to me that despite anything I say you hate the idea of J2R so any further discussion with your arrogance and condescension is a waste of my time.

Another contributor on her has shown that J2R reduces banding, reduces JPEG artefacts - Bart has shown, and so have I, that J2R recovers lost data, and reduces noise to an incredible degree.

All of which you, and others, prefer to ignore just because of the term "RAW"

I will simply go on posting more images on my site and investigate the capabilities of this program further.

A great pity you have such a closed mind.

Have a nice day.

But as far as I'm concerned, further discussion on this matter is useless.

Title: Re: JPEG or RAW?
Post by: digitaldog on February 10, 2019, 04:28:19 am
And that is your idea of a discussion?

But as far as I'm concerned, further discussion on this matter is useless.
That's my idea of facts.
Your question here "JPEG or RAWS?"
My answer: raw.
Useless?
“If my answers frighten you then you should cease asking scary questions.”
― Quentin Tarantino, Pulp Fiction
Title: Re: JPEG or RAW?
Post by: Garnick on February 10, 2019, 08:56:22 am
albytastic,

"But as far as I'm concerned, further discussion on this matter is useless."  I tend to agree when trying to look at this thread from your point of view.  It's really quite obvious that you are losing this one, so why bother continuing any further?  For the most part, right or wrong, you have been out numbered by a rather large margin.  In your 49th post you suggest that you've been a member of LuLa for a decade, and yet your participation seems to belie that statement.  However, simply lurking is also a way of gathering very useful information.  Hopefully you will join in here on other topics as well. 

I do have one rather important question before I leave.  It goes something like this -- In your opinion, what is Andrew Rodney's main opposition to the concept of converting a "JPEG to RAW"?  A simple question, but one I would like for you to answer, should you chose to do so.  And of course that question would apply to not only Andrew, but also to others who view this thread in the same light.

Gary       

Title: Re: JPEG or RAW?
Post by: Ray Harrison on February 10, 2019, 09:34:15 am
Another contributor on her has shown that J2R reduces banding, reduces JPEG artefacts - Bart has shown, and so have I, that J2R recovers lost data, and reduces noise to an incredible degree.

All of which you, and others, prefer to ignore just because of the term "RAW"

Albytastic - language is important and raw (and even "RAW"  :) ) has meaning. Claims on the website and marketing mean something. It is, in fact (as you have seen), difficult for me and others to get past that meaning to even figure out what this product really is since it is hidden behind misleading language. It clearly does not live up to its name, nor to the marketing claims. It may use AI but it is simply a tool to enhance jpegs in certain circumstances.

As enthusiastic as you may be for what it does to your images, my own conclusion from what I have seen from you, Bart, Frank, others and my own admittedly limited testing is that it has a fairly narrow scope: It seems to (overly) sharpen details in the jpeg, tries to do AI-based cloning of certain detail areas and stuffs more bits into the channels to write out the tiff/dng. I have not seen it "recover data" nor would I use adjective "incredible"  in its ability to handle noise well. In its conversion from sRGB to ProPhotoRGB (I believe they claim to make AI-based guesses as to the original colors and what they'd be in the new space), I've only seen poor results in difficult images.

I agree that people may obtain what for them are pleasing results on some of their images. There may be use cases that is is helpful in, for example from scans, perhaps old low-quality jpegs from a variety of sources, as Bart has pointed out. Over time, as they improve their models and train on more data, they may be able to do more jpeg image enhancement.

In general I have liked Topaz for certain things, I have nothing against AI in the right hands and there is a lot of scope for it in the image space and is indeed fairly pervasive and not just related to them. But they completely missed an opportunity to position the tool for what it really seems to be: JPEG image enhancing (AI or otherwise). Yes, "RAW" bothers me.
Title: Re: JPEG or RAW?
Post by: Garnick on February 10, 2019, 10:46:19 am
"Yes, "RAW" bothers me."

Thank you Ray.  It seems to bother the majority of those participating in this thread, including myself.  But let me explain further why I posed my question to "Albytastic".  I have once again scanned all of Andrews posts, and unless I missed something, he has never denied the possibility that this most recent offering from Topaz can perhaps in some way(s) help to enhance the apparent visual quality of a JPEG file.  From my vantage point Andrew's one and only objection to J2R is the "R" reference and what he sees as misleading and in general a totally false advertising campaign on Topaz' part.  And he seems to have a lot of support from the rest of the community, including me obviously.  I am far from being a techie and I'm the first to admit that some of the information Andrew and others have put forth can at times zoom over my head and cease to enter my knowledge bank, much to my chagrin.  However, that's what this forum is all about and it does indeed offer a bank of information for those who seek it.  We are all richer when we share our thoughts and on occasion our knowledge, no matter the degree.

Gary     
Title: Re: JPEG or RAW?
Post by: Peter_DL on February 10, 2019, 01:55:47 pm
"Yes, "RAW" bothers me."

Thank you Ray.  It seems to bother the majority of those participating in this thread, including myself.

-1.

A Raw file (a) can be processed to b) a high quality 16 bit image, be it in Tiff or DNG format, with rich shadow details, a smooth gradient in the sky, etc. Or, the Raw data (a) can be processed to c) a low-quality Jpeg. The Topaz software tries to transform c) to b), supported by an A.I. which is trained on the correspondence by a large number of images and pairs of c) and b).

The gist is sufficiently clear, and I find it a subordinate point that Topaz mixes up a) and b) in their descriptions.

Peter
--
Title: Re: JPEG or RAW?
Post by: digitaldog on February 10, 2019, 02:35:29 pm
-1.

A Raw file (a) can be processed to b) a high quality 16 bit image, be it in Tiff or DNG format, with rich shadow details, a smooth gradient in the sky, etc. Or, the Raw data (a) can be processed to c) a low-quality Jpeg.
--
Indeed. And neither changes what the raw data actually is. It's read only. The raw is a source for producing either but remains raw.
Quote
The Topaz software tries to transform c) to b), supported by an A.I.
It may try to do lots of things. But what it doesn't do is produce raw data nor can it edit a JPEG as if it were raw. The evidence if this was provided in the attempt to correct white balance of a JPEG that wasn't set correctly. Something that could very easily be conducted if shot as raw, where WB plays zero role because again, it doesn't affect that data, as it does a JPEG whatsoever. Topaz claims, without any evidence that they can Edit JPEG as if you shot it in RAW. At least one person has proven that's not true from actually testing, not that it was necessary for anyone who knows that JPEG and raw data is.
Title: Re: JPEG or RAW?
Post by: fdisilvestro on February 10, 2019, 03:18:02 pm
The Topaz software tries to transform c) to b), supported by an A.I. which is trained on the correspondence by a large number of images and pairs of c) and b).


The issue, and the main point of the recent discussions, is that Topaz claims to transform c) to a)
Title: Re: JPEG or RAW?
Post by: Ray Harrison on February 10, 2019, 04:23:11 pm
-1.

A Raw file (a) can be processed to b) a high quality 16 bit image, be it in Tiff or DNG format, with rich shadow details, a smooth gradient in the sky, etc. Or, the Raw data (a) can be processed to c) a low-quality Jpeg. The Topaz software tries to transform c) to b), supported by an A.I. which is trained on the correspondence by a large number of images and pairs of c) and b).

The gist is sufficiently clear, and I find it a subordinate point that Topaz mixes up a) and b)
in their descriptions.

Peter
--

I get a->b, a->c. I haven’t found the experience of Topaz going from c->b the same as giving me something that’s as if I got to b going from a->b. There’s some sky improvements, sharpening (way overdone in some cases), some AI based detail filling and a few other things, but you can’t really pull shadow detail for example. It’s a JPEG improvement tool. Nothing wrong with that. One can posit use cases for such a tool, indeed Bart has indicated several great ones. With the AI models and training sets getting better over time it will likely evolve to an even better JPEG improvement tool. I am all for being clever at image enhancing. But for me, the only time that DNG is relevant to raw is when starting with raw capture to get to the DNG. Starting with jpeg and backtracking to a tiff (AI-based or otherwise) and wrapping in a dng and calling it raw because “AI JPEG to TIFF” doesn’t sound as sexy or sell as well is disingenuous to me.

Naming does matter if people are trying to decide on the tool, and Topaz had the option to be clear. Now we are injecting things into conversations such as “names don’t matter, just look at what it does” and variations on the theme. This does, in fact, bother me, especially when it didn’t need to be this way.  :)

All that said, I think the horse has been well and truly beaten 😀. Time to get out into the fantastic Colorado sunshine.
Title: Re: JPEG or RAW?
Post by: Rajan Parrikar on February 11, 2019, 04:05:16 am
After going through and carefully weighing the arguments on both sides of this dispassionate, reasoned, erudite and civil discourse, I decided to go ahead and purchase JPG to RAW by Topaz for $67 after discounts.