A7r4 pixel shift. Rather a resolution eye-opener
Moderators: rjlittlefield, ChrisR, Chris S., Pau
A7r4 pixel shift. Rather a resolution eye-opener
I've recently been using Sony A7riv (61 megapixel full frame) on my main macro rig. Before that I'd used only A7rii (42mpix) and A9 (24mpix) in this context and had no plans to use riv for studio macro. I figured A7rii already captured all the resolution my lenses and objectives could deliver so why over-sample for no benefit (as I thought the riv would)? But after using 4-shot pixel shift on the microscope and seeing an enormous IQ increase from the extra colour resolution, I figured that might have some uses in studio macro too. Hence plugging the A7riv in for a quick look-see. While I was at it, I figured I might as well try 16-shot pixel shift too, even though I doubted it would work well.
Outcome? Well, I couldn't have been more wrong! Using pushed down Mitties on a 135mm tube lens and the camera in APS-C crop mode + pixel shift, the setup can squeeze 104 megapixels of detail onto APS-C with the 5x, 7.5x, 10x and 20x Mitties. At 50x things start to look a bit soft though (NA too low - and excluded that from the test anyway). I've attached a few of the tests consisting of a full APS-C frame and 100% crop for each of the 5x, 10x and 20x Mitties.
For quick routine stacks I think this is a pretty astonishing result in terms of captured resolution. I wonder what it's *really* equivalent to though? Although 4 times as many "positions" are sampled by moving the sensor half a pixel, it's still a whole pixel that captures the light at each half-pixel location (kind of sub-sampling neighbouring pixels). My guess is the outcome is much like what a heavy anti-aliasing filter would do. Although a virtual 240 megapixels are captured, spatially they would resolve less than that (but more than 61mpix). All guesses though, I have no idea if or how the pixel shift algorithm works, maybe it can squeeze extra information from the overlaps. I wish Sony would document such things. But whatever theory and numbers say - in practice, the outcome looks pretty darned impressive to me. I mean, extreme macro onto 104 megapixels in APS-C mode! Barely seems feasible. The laws of physics might be getting nervous soon
The test subject is a single 7.1mm wide rectangle cleaved from the middle of a silicon wafer full of otherwise mundane chips (almost fills the FoV of the 5x as shown in the first pic). The material is a bit tough to light for good contrast and the noise-like granular substrate is a real annoyance, but it does the job. Lights, diffusion and subject remained in the exact same positions for all three lens tests.
5x mitty pushed down to 3.4x plus 100% crop from the main image. Had to use a lot of jpeg compression on these, so quality not too good - but you'll get the idea. 10x mitty pushed down to 6.75x - each pixel is covering about quarter of a micron on the subject! And finally, 20x mitty at 13.5x - diffraction softening getting noticeable, but nothing a bit of standard sharpening won't fix.
Outcome? Well, I couldn't have been more wrong! Using pushed down Mitties on a 135mm tube lens and the camera in APS-C crop mode + pixel shift, the setup can squeeze 104 megapixels of detail onto APS-C with the 5x, 7.5x, 10x and 20x Mitties. At 50x things start to look a bit soft though (NA too low - and excluded that from the test anyway). I've attached a few of the tests consisting of a full APS-C frame and 100% crop for each of the 5x, 10x and 20x Mitties.
For quick routine stacks I think this is a pretty astonishing result in terms of captured resolution. I wonder what it's *really* equivalent to though? Although 4 times as many "positions" are sampled by moving the sensor half a pixel, it's still a whole pixel that captures the light at each half-pixel location (kind of sub-sampling neighbouring pixels). My guess is the outcome is much like what a heavy anti-aliasing filter would do. Although a virtual 240 megapixels are captured, spatially they would resolve less than that (but more than 61mpix). All guesses though, I have no idea if or how the pixel shift algorithm works, maybe it can squeeze extra information from the overlaps. I wish Sony would document such things. But whatever theory and numbers say - in practice, the outcome looks pretty darned impressive to me. I mean, extreme macro onto 104 megapixels in APS-C mode! Barely seems feasible. The laws of physics might be getting nervous soon
The test subject is a single 7.1mm wide rectangle cleaved from the middle of a silicon wafer full of otherwise mundane chips (almost fills the FoV of the 5x as shown in the first pic). The material is a bit tough to light for good contrast and the noise-like granular substrate is a real annoyance, but it does the job. Lights, diffusion and subject remained in the exact same positions for all three lens tests.
5x mitty pushed down to 3.4x plus 100% crop from the main image. Had to use a lot of jpeg compression on these, so quality not too good - but you'll get the idea. 10x mitty pushed down to 6.75x - each pixel is covering about quarter of a micron on the subject! And finally, 20x mitty at 13.5x - diffraction softening getting noticeable, but nothing a bit of standard sharpening won't fix.
Re: A7r4 pixel shift. Rather a resolution eye-opener
I found similar improvement of Pixelshift of A7riv on the Laowa 2,5-5x macro lens
viewtopic.php?f=27&t=43097
But I have no clue how I could handle all those files when stacking.
viewtopic.php?f=27&t=43097
But I have no clue how I could handle all those files when stacking.
-
- Posts: 112
- Joined: Thu Oct 13, 2011 3:56 pm
Re: A7r4 pixel shift. Rather a resolution eye-opener
Here is an article on the topic by Jim Kasson.
https://blog.kasson.com/the-last-word/d ... esolution/
https://blog.kasson.com/the-last-word/d ... esolution/
Re: A7r4 pixel shift. Rather a resolution eye-opener
Beautiful results, in line with what I have been seeing, first in MFT and now in Pentax and Panasonic FF. This isn't pushing against scientific limits. First, you have the Bayer filtering to deal with. Even just a four-image shift that does not increase the number of pixels (as in my Penatx K1) gives a very significant increase in real resolution, as you noted. Second, just because point-spread functions of nearby points overlap significantly (as when using the Rayleigh criterion), I think this does not mean that we cannot resolve them, because there is still a minimum between the maxima. Anything that reduces noise (as pixel-shifting does) should increase resolution. And it always helps to put more pixels under a feature. Three pixels is a good target, and more pixels should help (unless smaller pixels do not generate too much extra noise), though we eventually reach a point of diminishing returns. Rik and I have shown through tests that good lenses and objectives far outresolve our sensors:
https://www.photomacrography.net/forum/ ... 00#p178314
I estimated that an APS-C sensor should have at least 120Mp to do justice to a Mitutoyo 10x.
Regarding the actual real resolution of eight-shot pixel-shifted images, Olympus has thought about this and offers the output in two ways: as an 80Mp raw file with the pixel dimensions of the original pixel-shifted image, and also as a reduced 50Mp jpg which is claimed to efficiently match the "real" resolution of the image. I think this is probably a good estimate of how much real resolution there is in 8-shot shifted images, relative to the nominal resolution. Your 16-shot shift must do better than the eight-shot shift..... Very impressive!
https://www.photomacrography.net/forum/ ... 00#p178314
I estimated that an APS-C sensor should have at least 120Mp to do justice to a Mitutoyo 10x.
Regarding the actual real resolution of eight-shot pixel-shifted images, Olympus has thought about this and offers the output in two ways: as an 80Mp raw file with the pixel dimensions of the original pixel-shifted image, and also as a reduced 50Mp jpg which is claimed to efficiently match the "real" resolution of the image. I think this is probably a good estimate of how much real resolution there is in 8-shot shifted images, relative to the nominal resolution. Your 16-shot shift must do better than the eight-shot shift..... Very impressive!
-
- Posts: 2627
- Joined: Thu Jan 17, 2013 9:34 pm
- Location: United States
- Contact:
Re: A7r4 pixel shift. Rather a resolution eye-opener
Hi Beatsy,
Pixel Shift Multi Shoot mode is fantastic! In my mag.x 8x test with the A7RIV (here: https://www.photomacrography.net/forum/ ... 25&t=43165 + my site: https://www.closeuphotography.com/qiopt ... -objective) I posted some notes on pixel shift mode, pasting them below in case you or anyone missed them:
"On the Sony α7R IV 4 the 16-image Pixel Shift Multi Shoot (16PSMS) 240MP images are bigger spatially, 60 MP vs 240 MP, so this would be an advantage for printing, but they aren't really any sharper on a monitor when compared to 4-image Pixel Shift Multi Shoot (4PSMS) 60MP images.
The biggest gain with either Pixel Shift Multi Shoot mode, 4 or 16, is at least for me, higher color resolution and lack of demosaicing (or de-bayering) artifacts like false color and moiré, and less noise and more dynamic range. Mostly since there is no need for demosaicing (or de-bayering).
With PSMS there is a 1 EV reduction is noise, equiv. to giving it one more stop light, compared to non-PSMS images."
The most amazing gains from Pixel Shift Multi Shoot mode in my eyes? Less demosaicing artifacts which means less stair-stepped lines, and less noise/more dynamic range. I posted some examples in the test somewhere. Was going to post them here but I don't want to hijack your thread.
Best,
Robert
Pixel Shift Multi Shoot mode is fantastic! In my mag.x 8x test with the A7RIV (here: https://www.photomacrography.net/forum/ ... 25&t=43165 + my site: https://www.closeuphotography.com/qiopt ... -objective) I posted some notes on pixel shift mode, pasting them below in case you or anyone missed them:
"On the Sony α7R IV 4 the 16-image Pixel Shift Multi Shoot (16PSMS) 240MP images are bigger spatially, 60 MP vs 240 MP, so this would be an advantage for printing, but they aren't really any sharper on a monitor when compared to 4-image Pixel Shift Multi Shoot (4PSMS) 60MP images.
The biggest gain with either Pixel Shift Multi Shoot mode, 4 or 16, is at least for me, higher color resolution and lack of demosaicing (or de-bayering) artifacts like false color and moiré, and less noise and more dynamic range. Mostly since there is no need for demosaicing (or de-bayering).
With PSMS there is a 1 EV reduction is noise, equiv. to giving it one more stop light, compared to non-PSMS images."
The most amazing gains from Pixel Shift Multi Shoot mode in my eyes? Less demosaicing artifacts which means less stair-stepped lines, and less noise/more dynamic range. I posted some examples in the test somewhere. Was going to post them here but I don't want to hijack your thread.
Best,
Robert
- rjlittlefield
- Site Admin
- Posts: 23626
- Joined: Tue Aug 01, 2006 8:34 am
- Location: Richland, Washington State, USA
- Contact:
Re: A7r4 pixel shift. Rather a resolution eye-opener
Unfortunately the body of that article has one significant flaw, and the subsequent discussion in comments is beyond my patience to try untangling.Doppler9000 wrote: ↑Wed Apr 07, 2021 4:58 pmHere is an article on the topic by Jim Kasson.
https://blog.kasson.com/the-last-word/d ... esolution/
As I read it, the analysis sounds OK. The problem is in the assumptions, specifically at the point where Kasson explicitly assumes that the sensor has 100% fill factor, and implicitly assumes that the sampling function is perfectly uniform. Given those assumptions, I agree that the MTF curve does drop to zero at one cycle every two pixels, and there's no way to undo that by shifting.
However, if the sensor does not have 100% fill factor, or in general if the sampling function is not perfectly uniform, then the MTF curve does not drop to zero at one cycle every two pixels, and after that the whole argument falls apart.
As an extreme example, consider the case where the sensor has only 50% fill factor, with a perfect hat sampling function inside that 50%. I think you'll have no trouble convincing yourself that in that situation, the multi-exposure shifted sampling is exactly the same as what you'd get with a single shot using pixels half the size and 100% fill factor. Resolution would double. Ironically, a sensor that is "perfect" at its native resolution (100% fill factor) can gain far less from shifting than a sensor whose sampling function is lumpy (<100% fill).
So, the article's conclusion that sensor shifting cannot give more resolution was preordained by its assumptions about the nature of the sensor.
I rather doubt that any manufacturer is going to tell us enough about their sensor to permit a really good theoretical analysis.
So my plan is to put a lot more faith in experimental results. If somebody claims to get more resolution from sub-pixel shifting, then I will certainly say "show me". But if the images support what they said, I'll be happy to accept that result. There's no violation of theory in getting more resolution from sub-pixel shifting.
--Rik
Re: A7r4 pixel shift. Rather a resolution eye-opener
Steve,
Any chance we could see a comparison between an image without pixel shift, an image with 4-shot pixel shift, and an image with 16-shot pixel shift? Perhaps made with whichever objective shows the improvement most dramatically? Perhaps resized as necessary for easy comparison?
I fear that's asking a lot to prepare. This said, I can't get a sense of the improvement you're seeing from the "after" images alone--despite the fact that I've looked at my share of image stacks made with Mitutoyo objectives.
The level of improvement you Sony α7R IV users are describing is sobering for someone like me, who is invested in Nikon bodies, accessories, and workflow. That Sony body is going on two years old, and so far, one still hears of nothing similar on the Nikon horizon. Anything similar in the Nikon world is likely years away, or perhaps never. Though I loathe the thought of mixing systems, perhaps it needs to be considered.
--Chris S.
Any chance we could see a comparison between an image without pixel shift, an image with 4-shot pixel shift, and an image with 16-shot pixel shift? Perhaps made with whichever objective shows the improvement most dramatically? Perhaps resized as necessary for easy comparison?
I fear that's asking a lot to prepare. This said, I can't get a sense of the improvement you're seeing from the "after" images alone--despite the fact that I've looked at my share of image stacks made with Mitutoyo objectives.
The level of improvement you Sony α7R IV users are describing is sobering for someone like me, who is invested in Nikon bodies, accessories, and workflow. That Sony body is going on two years old, and so far, one still hears of nothing similar on the Nikon horizon. Anything similar in the Nikon world is likely years away, or perhaps never. Though I loathe the thought of mixing systems, perhaps it needs to be considered.
--Chris S.
Re: A7r4 pixel shift. Rather a resolution eye-opener
Sorry Chris. this really was a quick, qualitative test. I just rattled off the three stacks, transferred the images from card to PC using Imaging Edge (as it combined the shifted shots into single tiffs) and later formatted the card without keeping the individual frames. I know what they look like, you see. My goal here was solely to convey my wonder and surprise at getting genuine 100 megapixel images off an APS-C sensor (well, FF, but in APS-C mode).Chris S. wrote: ↑Wed Apr 07, 2021 10:45 pmSteve,
Any chance we could see a comparison between an image without pixel shift, an image with 4-shot pixel shift, and an image with 16-shot pixel shift? Perhaps made with whichever objective shows the improvement most dramatically? Perhaps resized as necessary for easy comparison?
I fear that's asking a lot to prepare. This said, I can't get a sense of the improvement you're seeing from the "after" images alone--despite the fact that I've looked at my share of image stacks made with Mitutoyo objectives.
...
I did a comparison shot some time ago, when I first got the camera. I was impressed at the time, but it's not quite up to the improvement I saw this time. More likely the setup and my technique though. Anyway, you can see a nice hi-res comparison pic at https://www.dpreview.com/forums/thread/ ... t-63173757 (if you haven't seen it already). The comparison is similarly stark, but I got slightly better end results this time probably due to lighting.
Last edited by Beatsy on Thu Apr 08, 2021 2:36 am, edited 2 times in total.
Re: A7r4 pixel shift. Rather a resolution eye-opener
The size and amount of files can cause issues - particularly with anything but the shallowest stacks. I take the view that the final pixel-shifted frames are my "source images" - and anything captured before that is temporary and disposable (the 4 or 16 raw frames that make up each image).lothman wrote: ↑Wed Apr 07, 2021 3:12 pmI found similar improvement of Pixelshift of A7riv on the Laowa 2,5-5x macro lens
viewtopic.php?f=27&t=43097
But I have no clue how I could handle all those files when stacking.
I don't tether anymore, preferring to "sneaker net" a memory card over to my PC. I transfer the images by running the Imaging Edge software in this case. It takes the frames direct from the card and deposits the combined images on the PC (which also imports them into a Capture One session). I've never had any frame alignment problems (thanks to the stable rig, I guess) and Imaging Edge makes a perfect conversion every time - so I don't need to "attend" the process and the source frames are binned once I have the converted tiffs on disk. After that, the workflow is no different to normal stacking - albeit with larger images in the 16-shot case, or just nicer ones with the 4-shot
Re: A7r4 pixel shift. Rather a resolution eye-opener
Thanks Robert, I have read some, but not all, of your pixel-shift posts - but will be going back to fix that shortly...RobertOToole wrote: ↑Wed Apr 07, 2021 6:29 pmHi Beatsy,
...The most amazing gains from Pixel Shift Multi Shoot mode in my eyes? Less demosaicing artifacts which means less stair-stepped lines, and less noise/more dynamic range. I posted some examples in the test somewhere. Was going to post them here but I don't want to hijack your thread.
Please post pics if you have them - it's not hijacking at all. Happy to see anything that livens and deepens discussion!
Re: A7r4 pixel shift. Rather a resolution eye-opener
Thanks for that Rik (and Doppler9000 for the link). I tend to respect/believe what Jim Kasson says and pretty much swallow it whole. Useful to have the health warning - especially as it supports my direct experience being somewhat beyond what simplistic theories (and Jim Kasson) might dictate.rjlittlefield wrote: ↑Wed Apr 07, 2021 7:50 pmUnfortunately the body of that article has one significant flaw, and the subsequent discussion in comments is beyond my patience to try untangling.Doppler9000 wrote: ↑Wed Apr 07, 2021 4:58 pmHere is an article on the topic by Jim Kasson.
https://blog.kasson.com/the-last-word/d ... esolution/
...
So, the article's conclusion that sensor shifting cannot give more resolution was preordained by its assumptions about the nature of the sensor.
I rather doubt that any manufacturer is going to tell us enough about their sensor to permit a really good theoretical analysis.
So my plan is to put a lot more faith in experimental results. If somebody claims to get more resolution from sub-pixel shifting, then I will certainly say "show me". But if the images support what they said, I'll be happy to accept that result. There's no violation of theory in getting more resolution from sub-pixel shifting.
Cheers
Re: A7r4 pixel shift. Rather a resolution eye-opener
Hi Lou,Lou Jost wrote: ↑Wed Apr 07, 2021 5:45 pm... Rik and I have shown through tests that good lenses and objectives far outresolve our sensors:
https://www.photomacrography.net/forum/ ... 00#p178314
I estimated that an APS-C sensor should have at least 120Mp to do justice to a Mitutoyo 10x.
...
Hmm, it's easy enough to approach testing that limit by *not* pushing mag down, but still use APS-C. I have an old Hoya 200/3.5 sat here on the desk. I got it to use with the 24mpix A9 in full frame mode (with mitties - which worked well) but never thought to use it for crop mode. I think I'll repeat the test using that and see what happens. If it doesn't push the limits, then I have a Soligor 250/4.5 too. I think I remember that vignetted on FF, but again I didn't think to use it on APS-C. Less FoV of course, but nice to lose the extreme corner smush. Back later...
Re: A7r4 pixel shift. Rather a resolution eye-opener
Hi Beatsy, which 135mm lens did you use for this test? I remember you had a fancy 135mm (Samyang/Rokinon? or Zeiss?) which you rejected because of vignetting. In crop mode that wouldn't be a problem. I like my cheap Vivitar-Komine 135mm but my Sigma Art 135mm gives better results, apart from hard vignetting in the extreme corners on FF.
Re: A7r4 pixel shift. Rather a resolution eye-opener
Just tried 5x and 10x Mitties on a 200mm tube lens for nominal rated magnifications. Still shooting 16-shot pixel shifted, crop mode, same specimen, tweaked lights. Pics below. Note: no non-pixel-shifted comparisons as it's too onerous in the workflow. This was all just to see if the extra (virtual) resolution afforded by pixel shift would even be useable...
And it certainly is! I believe I see a bit more detail captured with the 200mm tube lens (at the cost of a smaller FoV) but it's well into diminishing returns territory. Clearly the lenses were already approaching the hairy edge when mag was pushed down on 100mpix (with 135mm tube lens). The extra mag pushes the performance envelope hard and softening shows up. It's still a very impressive result though. I never, ever thought I'd be able to capture 100-megapixel macros with decent detail in them. Now if only I could have "fully resolved" AND "fully sharp" at the same time. Life would be grand...
5x Mitty at 5x onto APS-C + 100% crop from the centre. Lots of jpg compression again, and looks a little worse (softer) than the first anyway, but still shows good, usefully resolved detail from pixel shift images. 10x Mitty at 10x which pretty much resolves all of the smallest details available on the chip. So I didn't bother doing a 20x version as it would be the same detail, twice the size, with just quarter the surface area in view.
And it certainly is! I believe I see a bit more detail captured with the 200mm tube lens (at the cost of a smaller FoV) but it's well into diminishing returns territory. Clearly the lenses were already approaching the hairy edge when mag was pushed down on 100mpix (with 135mm tube lens). The extra mag pushes the performance envelope hard and softening shows up. It's still a very impressive result though. I never, ever thought I'd be able to capture 100-megapixel macros with decent detail in them. Now if only I could have "fully resolved" AND "fully sharp" at the same time. Life would be grand...
5x Mitty at 5x onto APS-C + 100% crop from the centre. Lots of jpg compression again, and looks a little worse (softer) than the first anyway, but still shows good, usefully resolved detail from pixel shift images. 10x Mitty at 10x which pretty much resolves all of the smallest details available on the chip. So I didn't bother doing a 20x version as it would be the same detail, twice the size, with just quarter the surface area in view.
Last edited by Beatsy on Thu Apr 08, 2021 12:19 pm, edited 2 times in total.
Re: A7r4 pixel shift. Rather a resolution eye-opener
Same Vivitar/Komine as you have - from your recommendation many years ago.Lou Jost wrote: ↑Thu Apr 08, 2021 7:54 amHi Beatsy, which 135mm lens did you use for this test? I remember you had a fancy 135mm (Samyang/Rokinon? or Zeiss?) which you rejected because of vignetting. In crop mode that wouldn't be a problem. I like my cheap Vivitar-Komine 135mm but my Sigma Art 135mm gives better results, apart from hard vignetting in the extreme corners on FF.
Yes, I still have that Samyang - languishing somewhere. I bent a stacking rail with it too (weight + imbalanced). I've replaced it for normal togging with a 135/1.8 G-Master. Now *that* is a proper 135mm lens. It vignettes as a tube lens (yes, it had to be tried) but it's way too good for "normal use" to be used as a tube lens anyway. The Vivitar works very well as it is, no need to change...
Cheers