Pages: [1]   Go Down

Author Topic: Will .H265, HEVC and HEIF drive the demise of JPEG?  (Read 3058 times)

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
Will .H265, HEVC and HEIF drive the demise of JPEG?
« on: June 10, 2017, 11:42:20 am »

Aside to Moderators: I am not sure where a technical topic like this belongs, so feel free to move it—or to create a new sub-forum for the technical discussions that some here love but others revile!

For about two decades, the original JPEG format has stuck around despite the arrival of multiple superior formats (including two newer JPEG standards) due to the great inertia of JPEG support and familiarity everywhere.  But it now seems that this immovable object might soon be pushed aside by the irresistible forces of demand for better video compression, convergence between motion and still imaging, and the iPhone.

The vehicle is H265, the successor of the dominant H264 video format, and its components HEVC (High Efficiency Video Coding) and HEIF (High Efficiency Image File format) with the latter intended both for intra-frame format in video and as as still image format. The iPhone comes in because iOS 11 and newer iPhones will use these new formats in place of H264/MPEG-4 and JPEG, with macOS 11.13 "High Sierra" also supporting them of course.

It looks like a good move as far as features, including far better compression for a given image quality, options for up to 16 bit depth and lossless compression, and some nice features in the growing overlap between motion and still imaging. Things like recording video with a sequence of full quality still (HEIF) images embedded in a single file and storing a burst in a single file, usable for things like focus stacking.

There is also support for non-destrucive editing by adding editing instructions into the original image file.  I wonder if combining this with the options of 16-bit depth and efficient lossless compression could make this an alternative in most cases to raw formats and DNG.

I am sure that there will be a moral debate about it being patent-encumbered (like H264; maybe more so) and whether it is better to wait for the open source rival AV1 being developed by the Alliance for Open Media.
Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #1 on: June 10, 2017, 11:58:46 am »

P. S. Samsung is also supporting H265/HEVC, including hardware support in its recent Exynos phone processors, and Apple has had hardware support in place since the iPhone 6S (A9 processor), so maybe that irresistible force is not just the iPhone but the two companies now driving the smart phone market, and so more or less driving the mainstream of digital image making and sharing.
Logged

rdonson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3263
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #2 on: June 10, 2017, 01:08:30 pm »

JPEGs might only be dethroned if there is widespread adoption of H265. That means it needs to be widely available in browsers, cameras, mobile devices, photo processing editors and on social media.
Logged
Regards,
Ron

Telecaster

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3686
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #3 on: June 10, 2017, 04:48:47 pm »

I'd say it's well past time 8-bit JPEG got the boot as an image creation format. I'd expect it to stick around for quite awhile as a legacy display format, though…

-Dave-
Logged

dreed

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1715
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #4 on: June 13, 2017, 08:58:18 am »

To get any benefit from a file format that has more bit depth than JPEG first requires a screen that can reproduce that.

How many computer monitors can do HDR (10 bit)?

Right now, very few.

Even amongst Apple computers, there are very few monitors that do more than 24bit color (8 bit per RGB.) Few indeed.

Once you pass that hurdle, then you need a graphics card that can generate that sort of color. Again, today very few are capable.

Finally you need software that can work with 10bit (or higher) color. AFAIK, only Photoshop can do this but from what I've read, currently requires various sacrifices to the one god, Adobe.

If you are lucky enough to master all of the above (which is to say it can be done), then you can start looking at what file format to put that image in. H.265 is one but it requires newer hardware and associated licensing fees.

In terms of file formats, you have...
TIFF (16bit)
JPEG2000 (16bit or 32bit)
PNG (16bit)

PNG is already widely supported in viewing devices, what's the need for another?
Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #5 on: June 13, 2017, 03:42:31 pm »

Ron and DReed,

    I agree that there are many things that need to happen before HEIF (part of H.265, aka HEVC) can challenge JPEG. Some comments:

- Video is the main driving force, for reduced transmission bandwidth and storage requirements: I believe that HEVC is the chosen standard for coming 4K broadcast TV. (And for example, Apple is already using HEVC for Facetime video chat, to reduce bandwidth needs.)

- Hardware support is coming first, and is already well established: HEVC is supported in several recent generations of Intel chips for PCs, the SOCs that Apple (since the A9 as in the iPhone 6S from 2015) and Samsung (Exynos chips from as far back as 2014) use in their phones, and also in recent Samsung smart TVs for example.

- With newer computers and phones and some browsers (at least Safari!) offering better video quality and/or lower data consumption for video steaming, other browsers will be pushed to adopt it too.

- Initial usage might be for capture (to save storage space on phones) and transmission, even if then converted to older formats for display and sharing.


Perhaps the main question is to what extent this will spread from HEVC video to use of HEIF for still images, as an alternative/rival to JPEG.
Given the fondness for transferring photos to and from mobile devices, I expect that the advantages in speed and in-phone storage usage for a given image quality will push HEIF forward.

@DReed:
(1) it seems strange to be skeptical about a demand for higher than 8-bits, given the debates over 12 vs 14 vs 16 bit raw files and interest in HDR and exposure stacking! One use is as a capture recording format, for post-processing of images of high dynamic range down to a viewable final product. I also suspect that linear encoding (no gamma compression) is an option, but I have to read more. Losslessly compressed high bit-depth HEIF files might then compete with RAW files and DNG, with the advantages of hardware support, and more generally piggy-backing on resources provided for H.265 video.

(2) I mentioned in the OP, HEIF has a number of features (like stacks of images for HDR, panoramas, focus stacking or such in a single file, together with explanatory meta-data, and support for non-destructive editing) that none of TIFF, PNG, JPEG or the failed JPEG2000 offer.
« Last Edit: June 13, 2017, 03:56:29 pm by BJL »
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #6 on: June 13, 2017, 04:04:03 pm »

(2) I mentioned in the OP, HEIF has a number of features (like stacks of images for HDR, panoramas, focus stacking or such in a single file, together with explanatory meta-data, and support for non-destructive editing) that none of TIFF, PNG, JPEG or the failed JPEG2000 offer.

tiff has options to store a number of images inside it...
Logged

rdonson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3263
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #7 on: June 13, 2017, 05:01:20 pm »


- With newer computers and phones and some browsers (at least Safari!) offering better video quality and/or lower data consumption for video steaming, other browsers will be pushed to adopt it too.

- Initial usage might be for capture (to save storage space on phones) and transmission, even if then converted to older formats for display and sharing.

Perhaps the main question is to what extent this will spread from HEVC video to use of HEIF for still images, as an alternative/rival to JPEG.
Given the fondness for transferring photos to and from mobile devices, I expect that the advantages in speed and in-phone storage usage for a given image quality will push HEIF forward.

I have no problem adopting a new standard.  Heck, I was all for JPEG2000 but that never materialized.  I'm hoping this fares better.  I don't do much video but who knows what the future holds there for me.   ;D   
Logged
Regards,
Ron

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #8 on: June 13, 2017, 05:14:25 pm »

tiff has options to store a number of images inside it...
But TIFF does not offer the excellent compression options of HEIV; instead TIFFs are huge storage hogs. Likewise, PNG has some of the attributes listed (and the advantage of being patent-free), but has far more limited compression options. More generally, HEIV seems to offer a combination of attractive features that no _one_ existing format offers, and beats all previous ones on compression/bandwidth.

I am not sure that HEIV will become a viable and widely-used alternative to JPEG; I am just optimistic enough about the possibilities to start a discussion, due in part to wide industry and hardware support that JPEG2000 and JPEG XR never got.

P. S. What do people think about the idea of creating a sub-forum for technical discussions like this one, that bore many photographers?
« Last Edit: June 13, 2017, 05:20:05 pm by BJL »
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #9 on: June 13, 2017, 05:16:13 pm »

But TIFF does not offer the excellent compression options of HEIV;

I was commenting on a particular item about multiple images inside a container, not in general
Logged

dreed

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1715
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #10 on: June 13, 2017, 05:32:42 pm »

Ron and DReed,

    I agree that there are many things that need to happen before HEIF (part of H.265, aka HEVC) can challenge JPEG.

To put this in context, the idea for using H.265 for image encoding is not new yet it hasn't happened. You don't just need to update encoders but decoders. What good is an image on (say) the LuLa front page in 10bit H.265 if no browser can read it? The problem is more than just "but video uses it." A large percentage of devices out there will need to support HEIF encoded images before people start delivering it.


Quote
@DReed:
(1) it seems strange to be skeptical about a demand for higher than 8-bits, given the debates over 12 vs 14 vs 16 bit raw files and interest in HDR and exposure stacking! One use is as a capture recording format, for post-processing of images of high dynamic range down to a viewable final product.

When you are mapping 14 or 16 bit images into 8bit, there's  a lot more scope for moving things around in HDR processing than 14 to 12.

Quote
(2) I mentioned in the OP, HEIF has a number of features (like stacks of images for HDR, panoramas, focus stacking or such in a single file, together with explanatory meta-data, and support for non-destructive editing) that none of TIFF, PNG, JPEG or the failed JPEG2000 offer.

But what does focus stacking, etc, in a HEIF file mean? That whatever device receives that file has to render it for output. That takes a lot more processing time (and battery power) than the relatively simple decompression of JPEG/GIF/PNG - not to mention bigger files.

Generating a specific look for the web with an image is hard enough now because you have to make sure that everything knows to use sRGB.

I'm all for newer file formats that deliver better image quality but it isn't going to change anytime soon - there's just too much inertia.

If you want to start somewhere, start with convincing Apple to support HEIF images on their iPhones.
Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
Re: Will .H265, HEVC and HEIF drive the demise of JPEG?
« Reply #11 on: June 13, 2017, 07:55:11 pm »

To put this in context, the idea for using H.265 for image encoding is not new yet it hasn't happened. You don't just need to update encoders but decoders.
. . .
If you want to start somewhere, start with convincing Apple to support HEIF images on their iPhones.
You seem to have missed the news that inspired my OP: Apple has just announced that coming this Fall, iOS 11 will change to using HEIF as the still image format for iPhone cameras (and HEVC for video) and that "macOS 10.13 adds HEVC and HEIF decode capability." -- https://beta.applebetas.co/notes/macos/10.13/beta1.pdf, page 6.

It might well be though that such images will routinely be converted to JPEG for sharing with non-Apple devices, including any social media posting. According to one source, "From the Platforms State of the Union after the keynote, they clarified that they'll transcode HEIF images to JPEG when sending pictures of devices they don't think can display HEIF."
There is more about 68 minutes into the video of that Platforms State of the Union: https://developer.apple.com/videos/play/wwdc2017/102/

When you are mapping 14 or 16 bit images into 8bit, there's  a lot more scope for moving things around in HDR processing than 14 to 12.
Sure; but is there any disputing the potential advantage of having a delivery format that goes beyond the 8-bits (albeit gamma compressed, so far better than 8-bit linear) of JPEG?

But what does focus stacking, etc, in a HEIF file mean? That whatever device receives that file has to render it for output. That takes a lot more processing time (and battery power) than the relatively simple decompression of JPEG/GIF/PNG - not to mention bigger files.
I certainly do not think that niche uses like having a single that contains a focus stack or a sequence to be stitched into a panorama (plus metadata) will drive HEIF adoption!
I just mention this feature as a potential advantage _if_ HEIF establishes itself. If only for the convenience of having a singe file to move around or drop into an editing program, rather than a collection of files and maybe needing a text file describing how they are related.

But about processing power: that might be a major factor driving these new formats: the algorithms are indeed costlier, due to things like using larger blocks for compression (up to 64x64 instead of JPEG's 16x16) and trying more options when seeking to compress, and the increase in available computing power and demand for wireless transmission of more and higher quality images is pushing the balance towards costlier but more effective compression/decompression methods. (The same happened with DVD vs CD: DVD error correcting encoding/decoding requires more processing than the CD version, but is better for space usage and such. It was probably not feasible on affordable microprocessors available when the CD format was specified, but had become so by the time of the DVD format design.)
« Last Edit: June 13, 2017, 07:58:33 pm by BJL »
Logged
Pages: [1]   Go Up