Progress in panoramic stitching software

[BernardLanguillier]

Team,

Some of you are certainly aware of this already, but both Autopano Pro/Giga and PTgui are continuing their relentless progress:

- Autopano Pro 3.5 was just released with claimed significant additional progress in terms of detection and stitching quality. It is a free upgrade for 3.0 users,
- PTgui Pro 10 beta3 is under testing and shows amazing potential in terms of performance thanks to the new OpenCL and multi-CPU support. I just did a quick test and it did compute a 13 D800 tiff images layered .psd stitch (resulting file is 200 megapixels and 4.5 GB on disk) in about 45 seconds on my, arguably very fast, system (Mac Pro 2013 and TB2 storage)... that includes the time to save the 4.5 GB file to disk. It is a 45 Euro upgrade for PTpui Pro 9 users.

Who wouldn't want to stitch with such tools?

Cheers,
Bernard

[LesPalenik]

Bernard,

what are the main advantages of Autopano/Pro Giga and PTgui when compared with Photoshop's Pano stitching?
Just the execution speed or also the alignment precision?

[Philip Weber]

Thanks for the update Bernard. As someone who's never stitched much, but would like to dabble, which of these two would you recommend as a first program, given my (at least initial) preference for as much automation as possible.

All the best,
Phil

[BernardLanguillier]

I would recommend Autopano pro as a first stitching tool. It does wonders in one click.

The key values compared to PS are:
- Scalability: they manage memory much more efficiently, no need to charge all the images in memory at once. There is basically no limit to the number of images you can stitch,
- Performance: they are much faster,
- Efficiency: they typically require less manual work and get you to a perfect result faster,
- Control: in terms of projections, perspective,...

PS CS is good in terms of doing pattern matching (although autopano pro is still significantly ahead in low contrast images) but that's the only thing where it shines in pure stitching terms.

Another value of PS is the possibility to use smart objects which reduces the cost of re-doing a pano if you need to change, for example, some raw conversion parameters.

This basically almost never happens to me though.

Cheers,
Bernard

[LesPalenik]

One problem I found with Photoshop pano stitching that it takes the easy way out and ignores the best segments of the individual images.
In practical terms, it tries to use as much as possible from the leftmost and rightmost images, and as little as possible from the center images.

The disadvantage of this approach is that both ends contain the weakest parts of the available images, and the center portions of the images which are optically the best, are often discarded. One way to compensate for this flaw is to manually crop each image on both sides, thus forcing the program to use the middle parts of the images rather than the ends.

This technique can be applied, of course, also for processing with other stitching programs, but maybe some programs take the edge distortions into consideration and try to select the most optimal image sections automatically.

[BernardLanguillier]

One problem I found with Photoshop pano stitching that it takes the easy way out and ignores the best segments of the individual images.
In practical terms, it tries to use as much as possible from the leftmost and rightmost images, and as little as possible from the center images.

The disadvantage of this approach is that both ends contain the weakest parts of the available images, and the center portions of the images which are optically the best, are often discarded. One way to compensate for this flaw is to manually crop each image on both sides, thus forcing the program to use the middle parts of the images rather than the ends.

This technique can be applied, of course, also for processing with other stitching programs, but maybe some programs take the edge distortions into consideration and try to select the most optimal image sections automatically.

Hum... ok, but it is key to stitch using a lens with as little corner defect as possible in the first place.

This is why I am currently stitching with a Zeiss Otus 55mm f1.4 at <f5.6 as much as possible.

PTGUI Pro does a great job at automatically correcting distorsion and light fall off, to within a fraction of an RGB value it seems. Autopano Pro was not as good until 3.0, I have not tested yet 3.5 on this. The skies show...

Another option is to use DxO to process images, as its local algos for light fall off, sharpness and distorsion result in images with very little field variation.

Cheers,
Bernard

[Bart_van_der_Wolf]

what are the main advantages of Autopano/Pro Giga and PTgui when compared with Photoshop's Pano stitching?
Just the execution speed or also the alignment precision?

Hi Les,

In addition to what Bernhard said, I'd also like to mention that these programs use much better resampling algorithms than Photoshop (one can even choose an algorithm that fits the subject matter best). That allows to maintain much better resolution despite the resampling/warping (also useful for keystone correction on a single image). That, combined with the much better memory management, are the first things one will notice.

But perhaps the most important benefit is that you (optionally) get a lot of control (if needed when 'auto' fails) to salvage difficult cases which are impossible to stitch with Photoshop. There is nothing more satisfying than getting a good image out of a challenging shooting scenario, where Photoshop usually fails. Even stitching images which were shifted (Tilt/Shift lens) becomes doable with a dedicated Pano-stitcher.

I'm currently even exploring the possibilities to use the pano-stitcher for producing super-resolution images, where sub-pixel accurate displacements are used to get more accurate samples of image detail.

Cheers,
Bart

[BernardLanguillier]

I'm currently even exploring the possibilities to use the pano-stitcher for producing super-resolution images, where sub-pixel accurate displacements are used to get more accurate samples of image detail.

Hi Bart,

This sounds interesting. I guess you start by taking several image at each pano position, align them, reverse compute the displacement that had to be applied and somehow use this information to re-compute higher quality pixels by weighted averaging?

Regards,
Bernard

[hjulenissen]

No-one mentioning the open-source Hugin?

I am missing integration of multi-image techniques in parametric editors (like Lightroom). If I shoot a number of images of a scene in any combination of "identical shots", focus bracketing, pano, exposure bracketing etc, it becomes hard to keep track of the images, to feed them into appropriate tools (raw development, super-resolution/noise averaging, DOF focus synthesis, stitching, HDR fusion etc) at the right order, and even harder to redo any part of the processing.

If Lightroom (or something similar to it) could keep track of that image set, figure out how they relate to each other (look at meta data and image information), do the needed image analysis, take user input, and render the output on the fly, my world would be a lot simpler. As Lightroom would have complete control over all information from the camera (and a sensible description of my intent), it could also update the algorithms with new "process versions" as they go along.

-h

[Bart_van_der_Wolf]

Hi Bart,

This sounds interesting. I guess you start by taking several image at each pano position, align them, reverse compute the displacement that had to be applied and somehow use this information to re-compute higher quality pixels by weighted averaging?

Hi Bernhard,

I knew you'd be interested

Super resolution exploits the tiny sub-pixel movements between otherwise identical images (shot from virtually the same position). Those movements are caused by tiny displacements due to camera movement (even on a tripod one can have them, check a magnified Live View image fragment and gently push the tripod in one direction to see it more clearly) and atmospheric effects (mostly at a distance). Handheld images will always have some small position changes.

The effect is that even tiny details that may fall 'between' two pixels on one image, may fall in the center of a pixel of another image. Spreading the same detail over one or over several pixels will make a big difference in rendered micro-contrast. Taking multiple slightly displaced samples will allow to statistically determine a more probable rendering of the original detail.

There are known methods from astrophotography (e.g. 'drizzle') and made simple to use in e.g. PhotoAcute Studio, but I was wondering if we can approximate the results with the tools we already have.

To allow the sub-pixel displacements to be aligned at the pixel level, we need to produce an enlarged/undersampled version of a stack of a few images (simply increase the output size of the stitched result), while aligning the images with mostly a translation and rotation correction (Yaw, Pitch, Roll), and some individual image shift ('d' and 'e' parameters), and then blend the images into a single result that then can be deconvolution sharpened. I'm looking into the best/simplest blending approach for maximum result.

The result could come close to the alternative of shooting with a 2x focal length, but without the need to stitch NPP rotated tiles for the wider FOV, and without the need to close down the aperture by a stop to maintain the same DOF (which may increase diffraction and subject movement or camera shake induced blur). Another benefit can be, depending on the blending method, that noise is reduced by averaging.

This just shows that Pano-stitching can serve many purposes.

Yesterday I made a reproduction of a large (1.6 x 1.4 metres) painting hanging at a hard to reach corner position in a poor lighting from one side situation, not what I would normally do for a more formal repro setup. PTGUI allowed (also using the Viewpoint optimization control to stitch a flat plane) to reach very decent impression which, with a bit more work on the uneven lighting, is not that far from a formal repro (see a small version, and a crop, as attached).

Cheers,
Bart

[LesPalenik]

Hi Les,

In addition to what Bernhard said, I'd also like to mention that these programs use much better resampling algorithms than Photoshop, ...

Cheers,
Bart

Well, it seems that with all these great plugins and specialized utilities, there will be less and less need to use Photoshop.
Soon, they will have to pay us $9.99 per month to use it.

Super resolution exploits the tiny sub-pixel movements between otherwise identical images (shot from virtually the same position). Those movements are caused by tiny displacements due to camera movement (even on a tripod one can have them, check a magnified Live View image fragment and gently push the tripod in one direction to see it more clearly) and atmospheric effects (mostly at a distance). Handheld images will always have some small position changes.

The effect is that even tiny details that may fall 'between' two pixels on one image, may fall in the center of a pixel of another image. Spreading the same detail over one or over several pixels will make a big difference in rendered micro-contrast. Taking multiple slightly displaced samples will allow to statistically determine a more probable rendering of the original detail.

That is very interesting and it has a lot of potential for increasing the quality of small details. Thanks for sharing this information, Bart.

[Bart_van_der_Wolf]

No-one mentioning the open-source Hugin?

Absolutely worthwhile to have a look at. Its user interface is just not a slick and fast to work with as AutoPano and PTGUI, but the output quality can be just as good.

I am missing integration of multi-image techniques in parametric editors (like Lightroom). If I shoot a number of images of a scene in any combination of "identical shots", focus bracketing, pano, exposure bracketing etc, it becomes hard to keep track of the images, to feed them into appropriate tools (raw development, super-resolution/noise averaging, DOF focus synthesis, stitching, HDR fusion etc) at the right order, and even harder to redo any part of the processing.

This is where the dedicated applications also make a difference. Based on intelligent choices of shooting interval, time, similarity of scene features, and EXIF parameters, a lot of good guesswork can be done to combine image sets.

Cheers,
Bart

[BernardLanguillier]

Did more tests with the same sample pano above:

1. PTGUI Pro 10beta3 saving to TB2 raid: 42 sec
2. PTGUI Pro 10beta3 saving to NAS: 2min 48 sec
3. PTGUI Pro 9 saving to TB2 raid: 1min 22 sec
4. PTGUI Pro 9 saving to NAS: 3 min 12 sec

Cheers,
Bernard
 

[BernardLanguillier]

I knew you'd be interested

 

There are known methods from astrophotography (e.g. 'drizzle') and made simple to use in e.g. PhotoAcute Studio, but I was wondering if we can approximate the results with the tools we already have.

Yep, I must still have a old PhotoAcute Studio license sleeping somewhere. With good optics, I could frankly never quite convince myself that the effort was really worth it compared to good old stitching with a longer lens. But as you point out below, that is indeed not a free lunch.

To allow the sub-pixel displacements to be aligned at the pixel level, we need to produce an enlarged/undersampled version of a stack of a few images (simply increase the output size of the stitched result), while aligning the images with mostly a translation and rotation correction (Yaw, Pitch, Roll), and some individual image shift ('d' and 'e' parameters), and then blend the images into a single result that then can be deconvolution sharpened. I'm looking into the best/simplest blending approach for maximum result.

The result could come close to the alternative of shooting with a 2x focal length, but without the need to stitch NPP rotated tiles for the wider FOV, and without the need to close down the aperture by a stop to maintain the same DOF (which may increase diffraction and subject movement or camera shake induced blur). Another benefit can be, depending on the blending method, that noise is reduced by averaging.

This just shows that Pano-stitching can serve many purposes.

Yesterday I made a reproduction of a large (1.6 x 1.4 metres) painting hanging at a hard to reach corner position in a poor lighting from one side situation, not what I would normally do for a more formal repro setup. PTGUI allowed (also using the Viewpoint optimization control to stitch a flat plane) to reach very decent impression which, with a bit more work on the uneven lighting, is not that far from a formal repro (see a small version, and a crop, as attached).

Very interesting! Thanks for sharing.

Cheers,
Bernard

[Damon Lynch]

Even stitching images which were shifted (Tilt/Shift lens) becomes doable with a dedicated Pano-stitcher.

It's funny you should mention this. Recently I shot some panos here in Tajikistan. I use a nodal slider to avoid parallax. However one I made an image of an interior with much complex detail. I wanted to get the detail from the ceiling as well as the floor. I shifted the lens up and down -- completely forgetting the parallax. Now I need to figure out a way to process the image and keep my sanity. PS CC and PT GUI 9 both fail to handle it. However in my brief test today run with Autopano Pro 3.5 looks much more promising. I might be able to save the image after all without spending hours of thankless work in PS.

Ironically the image is of the interior of a room of museum dedicated to Mubarak-i Wakhani (1839-1903), who was an Ismaili mystic poet, musician, astronomer, and religious scholar. He was a man who emerged from a largely uneducated populace, very remote from any urban centers, to achieve a measure of greatness that seems to me to be seriously impressive. He had to make his own paper to be able to work.

[Isaac]

No-one mentioning the open-source Hugin?

Absolutely worthwhile to have a look at. Its user interface is just not a slick and fast to work with as AutoPano and PTGUI, but the output quality can be just as good.

fwiw Hugin-2013.0.0 has a re-designed UI.

fwiw In one weird example, I've seen hugin create angular artifacts in an area of smoke/steam that didn't overlap any other image. No such problem stitching the same images with Microsoft Image Composite Editor.

[LesPalenik]

How does Microsoft Image Composite Editor compare to Photoshop Pano Stitching or Autopano Giga/Pro?
In the provided link it says that it works only with Windows 7. Is this page out of date or has Microsoft stopped maintaining it?

  

[Isaac]

How does Microsoft Image Composite Editor compare to…

Don't know.

In the provided link it says that it works only with Windows 7.

It says: "Stitch directly from a video (only on Windows 7)".

"Image Composite Editor works with 32-bit and 64-bit versions of Windows XP, Windows Vista, Windows 7, and Windows 8."

…did Microsoft stopped maintaining it?

It's worse than that! It's provided free of charge for personal and non-commercial use - and without official support - courtesy of Microsoft Research.

I find it helpful to throw a bunch of images into MS ICE to see what a composite would be like, and what problems I'll need to deal with.