Sigma Merrill & the Foveon (X3/X-Trans) Sensor

[rdonson]

X-trans it is a Fuji technology that changes the typical bayer array to make the distribution of red, blue and green pixels more random... The idea was to simulate the randomness of film... But it complicates things in the demosacing process of RAW processors.

Not all RAW processors. 

[davidgp]

Not all RAW processors.

Mainly my phrase was poorly written, what I meant, since it is different than typical Bayer Matrix, makes necessary that RAW developers have to update their demosaicing algorithms to consider this case... some more successful than other doing that



http://dgpfotografia.com

[rdonson]

Well said, David.

[Doug Peterson]

Foveon is marketed as having three times the resolution of an equivalent normal camera. That's great marketing, but otherwise hogwosh.

With really sharp lenses, no AA filter, and a great processing engines (e.g. Capture One), we can recover over 93% of the spatial detail (compared to monochrome sensors, the gold standard) in real-world imaging. To get anywhere near the marketed advantage you have to be shooting subject matter that has detail which is exactly one pixel in size, very high in saturation, and completely random in pattern. No process recovers 100% of the spatial detail, so the theoretical maximum difference is, at absolute most, a few percent compared to a single-shot camera with the same number of actual pixels (and the way Foveon has been historically marketed it often required a little digging to find out the actual pixel count, not the inflated "equivalent" number).

The advantages of multi-shot, three-sensor scan backs, color-wheel-capture, and (in theory) Foveon were profound 20 years ago. Most of those advantages have since evaporated with two decades of improvements in single-shot Bayer cameras and software (where 99% of R+D money has been), while the disadvantages of those more esoteric processes remain.

See 16:41 in our Process Control for a comparison of where the math was years ago versus today.

Like with any camera hype, try it yourself before you believe the marketing (including, of course, the marketing we produce; call us on it and we'll be glad to set you up to test it yourself).

[Paul2660]

Doug,

Those are a bit strong don't you think?  Bayer by default is capturing 1 color per pixel sensel as I understand it, thus if you capture red, you are interpolating for green and blue.  That is Bayer.  So anytime you interpolate you lose details. 

No doubt the IQ CCD and CMOS backs are very sharp and can capture a lot of details, but I would stand to think that a 50MP multishot vs 50MP CMOS IQ back, the multi shot would have more details, unless there is movement.  C1 does an excellent job on Phase back files no doubt but you are still working with interpolation for color. 

The Foveon theory is sound, in that you are not interpolating for any color, thus you will not have loss of details.  There issues, as many have mentioned on this forum and other sites, to the current tech in the state it's in, but that IMO is mainly due to poor raw conversion attempts by various companies and the fact that none of the Foveon chips in use by a DSLR can capture DR past base ISO due to noise and they are not a low light capture device optimally.  However in the right light conditions, (kinda like CCD technology) they can make a very clean and detailed capture.

I can speak the advantages of multishot on 35mm, as a Pentax K1 capture in Pixel shift, I feel would  surpasses a similar P45+ capture, I have plenty of both to work with.  36 vs 39 base MP, the K1 will out resolve the P45+ if Pixel shift is in use, both in resolution and Dynamic range. The K1/pixel shift tech also has issues, but advantages are also very appealing:

1.  excellent dynamic range, superior to DR at any ISO setting, due to the multiple sampling.
2.  Superior detail recovery

The main downside being movement, which depending on the raw converter can either ruin the capture or have some adverse issues. 

The Bayer solution is a sound process and has been around for a long time and proven itself, however there are newer technologies that just have not been embraced by camera companies such as Foveon multi layer or multi shot. 

Paul Caldwell

[JKoerner007]

Great post, Paul.

[JKoerner007]

Foveon is marketed as having three times the resolution of an equivalent normal camera. That's great marketing, but otherwise hogwosh.

Hogwash? I don't think so.

With really sharp lenses, no AA filter, and a great processing engines (e.g. Capture One), we can recover over 93% of the spatial detail (compared to monochrome sensors, the gold standard) in real-world imaging.

In other words, with the D810 ... only?

The advantages of multi-shot, three-sensor scan backs, color-wheel-capture, and (in theory) Foveon were profound 20 years ago. Most of those advantages have since evaporated with two decades of improvements in single-shot Bayer cameras and software (where 99% of R+D money has been), while the disadvantages of those more esoteric processes remain.

Yeah but ... how do you think the two systems would compare, in another 20 years, with 99% of R&D being focused on Foveon (and like) technologies ... while Bayer systems stand still?

[Doug Peterson]

Hogwash? I don't think so.

I've got a closet full of top Bayer gear in NYC and LA and studios in both. If you ever want to do a side by side let me know.

In other words, with the D810 ... only?

Or a few dozen models of digital backs.

Yeah but ... how do you think the two systems would compare, in another 20 years, with 99% of R&D being focused on Foveon (and like) technologies ... while Bayer systems stand still?

EXACTLY. If Foveon had came early and strong early in sensor development and won some momentum early on it would have likely stayed ahead because Bayer math would have stood still for want of R+D and the various problems with Foveon would have been the ones receiving continuous R+D. But that wasn't how things went.

The same could be said of many technologies. If all R+D into flight was focused for 100 years on lighter-than-air travel we'd have dirigibles that were fast and cheap and a dirigible docking station on the Empire State building. But airplanes won out (plus that whole Hindenburg thing) and that's where all the R+D went.

As for whether that's going to happen now in reverse ("20 years, with 99% of R&D being focused on Foveon")... very unlikely. Bayer had a big problem: the math to fill in the missing color is really complicated and really CPU intensive. In the 80s we were terrible at it. In the 90s we were bad at it. In the 2000s we were getting pretty good at it. Now we are really stinking good at it. Again: 93% spatial recovery or better in real world pictures as of Capture One 9 as measured by objective standards like FADGI and Metamorfoze. So the one achilles heal of Bayer is now, at best, a heel which is moderately tingly when the weather is changing.

[Doug Peterson]

Those are a bit strong don't you think?  Bayer by default is capturing 1 color per pixel sensel as I understand it, thus if you capture red, you are interpolating for green and blue.  That is Bayer.  So anytime you interpolate you lose details. 

Absolutely! When Bayer algorithms were brand new the amount could approach 66% depending on the color and nature of the subject, and was very likely to cause artifacts. That was a huge difference!

Now, by objective measurements, (see previous post) that loss is around 7% or less.

Foveon marketing would say every one of their "pixels" counts 3x. When the reality, objectively measured is closer to 1.07x, I'm comfortable calling it hogwash.

No doubt the IQ CCD and CMOS backs are very sharp and can capture a lot of details, but I would stand to think that a 50MP multishot vs 50MP CMOS IQ back, the multi shot would have more details, unless there is movement.  C1 does an excellent job on Phase back files no doubt but you are still working with interpolation for color. 

Absolutely. Again, up to around 7% more detail with modern algorithms used for the single-shot.

Back in the day, for example, prestigious museums and libraries were very heavy users of multishot. Now that pendulum has swung hard toward 60mp, 80mp and 100mp single-shot cameras (here's a very partial list to backup the statement).

The Bayer solution is a sound process and has been around for a long time and proven itself, however there are newer technologies that just have not been embraced by camera companies such as Foveon multi layer...

Foveon is, in no way, new. It's the fusion power of camera sensors. I first heard about it in the late 90s (Foveon, not fusion power).

Look, I'm the first to say that things can change. Foveon (or some other currently unheard of technology) may be the sensor of the mid 2020s. After all, at some point it seems likely fusion will happen and it will be awesome.

My only point of coming on to the thread was that if you (like the OP) are reading marketing that says an "APS-H Foveon sensor is equivalent to a 51mpx traditional Bayer sensor." and expecting that to hold up to actual testing... you will find that is absolutely, unequivocally, not even close to right.

[Doug Peterson]

a Pentax K1 capture in Pixel shift, I feel would  surpasses a similar P45+ capture... The K1/pixel shift tech also has issues, but advantages are also very appealing:

I think if you make a direct comparison (same lens, blah blah, then print the same size) you'd find them very similar, with neither distinctly better across the board.

But that just shows microstepping is a great technology in some situations. If nothing is moving, the sensor is small and light, and the res of the sensor is reasonable, it can greatly improve results. That a 2016 Pentax K1, using pixel shift, can compete with a single-shot medium format sensor developed in 2004, when the camera/subject are mostly static, speaks both to the strength of pixel-shift technology on low res sensors, and the strength of a high quality single-shot Bayer sensor, no?

The problem with pixel-shift (beyond that it doesn't work well when camera or subject are moving) is two fold:
- Microstepping is harder with smaller pixel sizes (more precision required)
- Microstepping is harder with larger sized sensors (more mass to move)

Hence microstepping with higher res becomes exponentially harder.

You'll notice neither Hasselblad nor Sinar have come out with a new multishot camera using a sensor with higher res than 50mp, despite higher res sensors being available for 9 years. That speaks to both the lower demand for multishot (already a niche 9 years ago) and the difficulty of moving a sensor minutely enough, with enough precision, quickly enough, to do multishot at really high res.

Pixel shifting is a great tool for the K1. To have that much detail for static-ish subject matter in a camera so small and affordable is fantastic. Pentax is smartly finding features/niches the bigger players can't or won't play in and exploiting them. But if you're expecting that technology to drive the ultra-high res cameras of the future, then I think you're betting on the wrong pony. But I could be wrong! Time will tell.

[BernardLanguillier]

Everybody does marketing that's true.

There is still something special to the Foveon pixels and I think that I see more that 7% improvement over bayer at same megapixel level.

Cheers,
Bernard

[hogloff]

Everybody does marketing that's true.

There is still something special to the Foveon pixels and I think that I see more that 7% improvement over bayer at same megapixel level.

Cheers,
Bernard

Yeh, more like 8.5% improvement I would think.

[Doug Peterson]

Everybody does marketing that's true.

There is still something special to the Foveon pixels and I think that I see more that 7% improvement over bayer at same megapixel level.

Fair enough. I don't agree but reasonable people can disagree on the exact amount; it's dependent on subject type, method of measure, and notably which Bayer pixels in which raw convertor etc etc. But you'd agree the improvement is not 3x or anywhere near it yes?

[JKoerner007]

Fair enough. I don't agree but reasonable people can disagree on the exact amount; it's dependent on subject type, method of measure, and notably which Bayer pixels in which raw convertor etc etc. But you'd agree the improvement is not 3x or anywhere near it yes?

Hi Doug;

The purpose of this post was 1) to call attention to how "nobody seems to care" about this superior Foveon technology; 2) that this technology appears better-suited to static shots (landscape, macro); 3) to envision a "what if" hypothesis regarding Sony or Nikon getting ahold of these sensors and developing them with their superior camera-crafting skills; and 4) to point out that Canon is itself rumored to be coming out with a similar, Foveon-like, 120mp sensor for 2018 ... because it lends itself better to this kind of imagery (still/static, rather than low-light, fast-action sports).

I specifically intimated that, maybe in the future, there will be a bifurcation: namely Foveon-like sensors for huge MP still cameras (think D900, 5D Mark V) ... so you criticizing the lens for its poor handling of movement/low light is letting the cat out of the transparent bag. We already know this.

So back to the point:

If, as you say, this technology was "profound" 20 years ago ... and if, for the last 20 years, 99% of the R&D has been with Bayer sensors ... and, if, after a 20-year advantage, the Foveon sensor is, as you admit, still superior to Bayer in still imagery ... can you not see the conclusion? Foveon is the better still technology.

So, again, the point is ... if companies like Canon are going to spend the next 20 years developing Foveon-type sensors (for huge, detail-oriented sensors) ... that is something to be excited about

That's all ...

[Paul2660]

I think if you make a direct comparison (same lens, blah blah, then print the same size) you'd find them very similar, with neither distinctly better across the board.

But that just shows microstepping is a great technology in some situations. If nothing is moving, the sensor is small and light, and the res of the sensor is reasonable, it can greatly improve results. That a 2016 Pentax K1, using pixel shift, can compete with a single-shot medium format sensor developed in 2004, when the camera/subject are mostly static, speaks both to the strength of pixel-shift technology on low res sensors, and the strength of a high quality single-shot Bayer sensor, no?

The problem with pixel-shift (beyond that it doesn't work well when camera or subject are moving) is two fold:
- Microstepping is harder with smaller pixel sizes (more precision required)
- Microstepping is harder with larger sized sensors (more mass to move)

Hence microstepping with higher res becomes exponentially harder.

You'll notice neither Hasselblad nor Sinar have come out with a new multishot camera using a sensor with higher res than 50mp, despite higher res sensors being available for 9 years. That speaks to both the lower demand for multishot (already a niche 9 years ago) and the difficulty of moving a sensor minutely enough, with enough precision, quickly enough, to do multishot at really high res.

Pixel shifting is a great tool for the K1. To have that much detail for static-ish subject matter in a camera so small and affordable is fantastic. Pentax is smartly finding features/niches the bigger players can't or won't play in and exploiting them. But if you're expecting that technology to drive the ultra-high res cameras of the future, then I think you're betting on the wrong pony. But I could be wrong! Time will tell.

Hi Doug,

I don't have the time or patience to do multiquotes, but you did make some good points for sure.

1.  I agree 100% that the current state of Foveon, in it's APS-C format would have a hard time equaling the output of a 50MP Bayer, as in the IQ350. 
2.  The C1/P1 algorithms have most definitely improved especially with C1 10.x and have drawn the pixel to pixel gap very close.
3.  The Foveon message on total MP output tends to take the three separate sensors and multiply them by 3, so 15MP x 3 45MP output, which IMO is not what you get.  You get a very detailed 15MP image.  If this in now interpolated to 50MP, I agree the 50MP sensor native will odds
      win.
4.  If you add the huge advantage that the 100MP CMOS sensor adds, since you have such large room for cropping, Foveon has a ways to go.


The K1 output in pixel shift can move, unlike other multishots.  This is something most don't understand.  The camera allows for Pixel shfit with Motion correction or no motion correction.  The KEY is that NO raw converter besides Silkypix can handle the images with motion correction.  LR and C1 are worthless here, C1 even more as they have not even tried to work with the multishot pixel shift.  LR made one pass and as usual for Adobe, (once and done) and their solution is less than adequate.  I only blame Pentax for this, it's their camera and they should have worked with Adobe to get a better solution and at least attempted to get something from C1. 

The amount of movement can't be extreme, but Silkypix and K1 can handle quite a bit of wind movement and still make the correction without aliasing.  This link to a post I made on getdpi, has some examples of just how much movement can be accommodated.  It's much more than you seem to believe. 

https://www.getdpi.com/forum/pentax/59030-examples-silkypix-vs-lr-pixel-shift-conversions-cross-posted.html

However Silkypix is not mainstream, doesn't allow for dng out (last time I used it 6 months ago) and their workflow doesn't work for me.

If a 100MP or 120MP Foveon style chip does arrive, then that will be the time to really compare things and as you point out that's a long way off.  However Pixel shift technology is here now and could be used by a lot more companies, including Phase One for some amazing results.

Paul Caldwell

[NancyP]

On the other hand, under the right conditions, you can't get more compact than one of the fixed-lens Merrills, and they don't need much of a tripod either. The truly irritating thing about the cameras is that they don't have wired shutter release. How dumb is that! I keep thinking about jerry-rigging some 3D print-job Merrill fitting attached to an old mechanical shutter release.

[RobertJ]

I had and sold all three DP Merrill cameras, mainly because the files are almost unusable.  Images taken in anything other than sunlight, even at base ISO, have significant banding, color splotches, noise, etc.

Add to that, most of the sharpness came from the software itself, which was adding insane amounts of local contrast, sharpening, and tone mapping.  Though I may not be technically correct, I felt that this whole Foveon thing was really just an illusion, almost a complete sham. 

After using the X3F_Extract tool to create a DNG from the RAW files, I saw that the files, when untouched by the Sigma Raw converter, are nothing special, and that the sharpness everyone was seeing was mostly an illusion from Sigma Photo Pro.

Bayer sensors are superior *at the moment*, IMO.

If Sony took over the technology, then I would be interested again.