Some recent observations and questions by Steve Valley (in this thread) prompted me to write up this explanation of stack-and-stitch with ordinary optics versus telecentric systems.
The key parts were:
svalley wrote:Larger insects present another problem. They are too large to fit into the field of view of the scope. This means that they have to be shot in overlapping sections and these sections have to be stitched together to create an image of the whole specimen. ... I discovered that with our Leica imaging system, there is too much depth of field at low magnifications, despite always shooting with both internal apertures wide open. At magnifications less than 16x, the size of the specimen changes dramatically within the image frame of each layer in the stack, resulting in terrible fringing around every sharp edge in the composite.
rjlittlefield wrote:Regarding your problems of stitching stacks, you might be interested in http://www.photomacrography.net/forum/v ... php?t=1032 .Each tile is a stack, 11.5 mm wide. The deepest stack is over 25 mm deep, and the total subject in-focus depth is 37 mm. To shoot the stacks, the camera was shifted laterally. But when I flash between stacks, there's no visible parallax. It's pretty cool.
I was confused for a long time about apparent inconsistencies in these various writings, but I think that eventually I figured out how to make everything fit together.svalley wrote:Rik, I had already read that thread and just reread it. ... I did not understand exactly what you are doing.... to shoot the individual tiles are you just moving the camera and rear lens with the front lens remaining stationary, relative to the subject? This is a very interesting concept, but would be hard to adapt to my scope setup at work. I have not had any noticable parallax problems with my stitched images.
Let's consider a simple hypothetical situation. Our subject is a red circle above a green square, in different planes.
Suppose we try to stack-and-stitch this subject using ordinary optics, where the center of perspective (the entrance pupil) is someplace around the center of the lens. Then we get this situation:
Notice that with an ordinary lens, the red circle looks bigger than the green square, regardless of where the lens is focused. If the lens is moved to focus lower, then the ratio of sizes becomes a bit larger, but not much.
In this case, there are essentially two approaches that the software can take. Either 1) it can do nothing except select pixels from the appropriate images, or 2) it can adjust the scale of each image to compensate for the shift in focus, and then select pixels.
In general, the decision of how to select pixels varies from easy to difficult depending on the DOF per frame. At high magnification and large apertures, with very shallow DOF per frame, decisions are easy and the software generally gets them right. At lower magnifications and smaller apertures, with greater DOF per frame, decisions are difficult and the software may get them wrong.
The quality of pixel selection interacts strongly with scaling and stitching.
If the software does not scale (Cases A and B), then bad pixel selection quickly causes bad result quality because the source images just don't line up. Low magnification makes this problem worse because it encourages large focus steps that have correspondingly large changes in scale.
This problem can be almost completely avoided by having the software adjust images to a common scale (align image features) before selecting pixels. The solution is not quite perfect because the relative sizes of image features at different depths actually changes slightly as the focus shifts. But it's pretty good.
So, if you want to do just a single stack, then in most cases scaling the images to compensate for focus shift is the right way to go. (The one serious exception is if you're not prepared to accept the closer=bigger perspective that's naturally produced by this type of lens system.)
But if you try stitching multiple stacks, the picture changes.
In this case, the fundamental problem is that the lens looks outward in a cone. Subject features that appear to line up with the lens in one position, do not line up when the lens is moved to a different position. This is called parallax, and it's a Very Bad Thing for stitching. But what to do, what to do?
The "solution" to parallax in this case is simple: don't adjust image scale to compensate for focus shift. Taking that approach -- and assuming perfect pixel selection -- subject features will line up the same way in adjacent tiles, even though that's not the way the lens originally saw them.
Unfortunately, if there are errors in pixel selection, this becomes a Catch-22 situation. You cannot turn off scaling because subject edges go bad, but you cannot turn it on either because then you get parallax errors between tiles.
Normally I run with scaling turned on. That produces the best image quality in a single stack, but introduces parallax between tiles.
I'm thinking that Steve runs with scaling turned off. That avoids parallax, but causes worse problems around edges at lower magnifications.
OK, with that analysis as background, let's look at telecentric optics.
You can think about telecentric optics as meaning "no change in scale with distance", or "only uses light rays parallel to the axis", or "entrance pupil at infinity". They're all roughly equivalent.
However you think of it, the key feature of telecentric optics for our purposes here is that the alignment of subject features does not change as you move the lens, either to step focus or to shift laterally for tiling.
Because the alignment of subject features does not change -- courtesy the telecentric optics -- there is no need for scale adjustment in software, and there is no parallax introduced between adjacent tiles.
Using telecentric optics makes it much easier to get a clean result from stack-and-stitch.
There is a downside, of course: telecentric optics are uncommon, potentially expensive, and necessarily have a small field of view (no greater than the front lens diameter minus the aperture size).
But dang, they sure are neat to work with for some subject sizes!
Does this explanation help? What remains confusing?
--Rik
Edit: Jan 2, 2007, to add link to earlier discussions.
Edit: Feb 21, 2015, adding links to other closely related threads