Reduced NA gives more detail

A forum to ask questions, post setups, and generally discuss anything having to do with photomacrography and photomicroscopy.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

ChrisR
Site Admin
Posts: 8668
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Reduced NA gives more detail

Post by ChrisR »

Preamble for beginners! [
We know that increasing NA gives better resolution.
A large NA means using a wide cone of view from the lens, with the apex at the point of focus.
We also know that when photographing a 3D structure, out of focus material obstructing the cone of view causes blurred “ghosting”. (The "Transparent foreground" is related, but it's a different issue).
The intervening structure also reduces the working NA.

If looking into a well, half the cone of view is obstructed at the edges of the well.
Image
If the NA is smaller, then more of the floor of the well is seen without obstruction, but the resolution is lower.
The obstructing shoulder marked red, is likely to be well lit. Its out of focus blob covers a lot of the sensor. All the frames in a stack with detail at the edge of the bottom of the well, are severely washed-out by unwanted unfocused light from the shoulder, so very little will show.
]

This was apparent on a recent subject – the peristome of a common-or-garden moss.
The peristome is the developed structure around the opening at the end of the capsule,
Image
when it opens so spores can come out.

A general part-stack view of a peristome
Image
shows its inner and outer teeth, and some escaping spores. Crop, approx 0.5mm fov, 20x objective.

The depth distance between the outer(A) and inner(B) peristome teeth is about 50 microns, measured at the broken tip, (C).
The red square is 50 microns across, just there for scale. The C-A section sketch shows illumination cones for NA 0.42, and stopped down to half diameter.
Squares are 10µm.
Image

A stack was run through the part in question, with an extra exposure after every 4th, at two stops closed, with the flash turned up to compensate.
So on for 10 sets, ie 50 frames which covered the detail I was looking at.
The whole stack was aligned, then partial stacks made from only the full NA shots, then only the reduced NA shots.

Image

Image

Both have USM at 100%, @10 pixels, applied.

The detail on the outer teeth is, as expected, better in the full-NA stack, but on the inner teeth there’s more to see, and certainly less of the objectionable fuzzy voids, in the stopped-down stack.

Animation here , 1.27MB

It’s not too difficult to retouch onto the Normal one, where it's better. Might be better in Photoshop as each layer could then be given a different USM.
An outcome which I’d previously noticed, is that if you want a picture for web use at say 1000 pixels width, you’re better to stop the objective down, because it’ll look better. You don’t see all the fine detail of a higher NA objective anyway, but the high NA ghosting artifacts would still be visible. Same would apply to stereo/rocking images on normal screens.

The workflow is a pain. Exposing a stack with altered lighting and aperture every fifth shot is grim – it’s easy to miscount.
I found that a stack made simply from all frames, didn’t work. I have to use X-Y alignment in the stacker, because my setup is out of line or wobbles too much at 20x. This means that the stopped-down shots were not aligned sufficiently well with the full NA ones, to avoid double imaging.
If the stacks are run separately, then the alignment is further out and it’s harder to retouch, though areas can be stretched in Photoshop.

Ideally, we want something to arrange illumination of selected parts of the subject. A confocal microscope excludes stray light with a pinhole at the sensor. We can't do that but we could reduce the light coming from unfocused parts by not illuminating them.
This could be done above (“Epi”), or subject-side of the objective. But they’re both a little fiddly so I’ll leave them until after lunch.

ray_parkhurst
Posts: 3416
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Post by ray_parkhurst »

A problem with 3D objects is diffraction from any edges present. The edge that is creating that red area shown will create a strong diffraction effect that will likely affect sharpness. In fact the aperture is very small.

I see this effect when I photograph coins. For best lighting results, I prefer to get my lights as high as possible, so high in fact that the body of the light can be between the lens and coin. I must check the setup before taking shots using a very small aperture, to prove the body is not partially obstructing the path. At the taking aperture, the obstruction is not at all obvious because light hits the sensor through other areas of the lens, and the obstruction vanishes if it's not too egregious. But those areas that are blocked create a diffraction pattern that causes unsharpness in those areas of the image especially, and a bit of unsharpness in the whole image. I would expect this effect at the micro level for high NA objectives looking down into a well as you've shown.

Chris S.
Site Admin
Posts: 4042
Joined: Sun Apr 05, 2009 9:55 pm
Location: Ohio, USA

Post by Chris S. »

Ray, though I’m sure you understand diffraction, I’m concerned that your example might be misunderstood in a way that reinforces common misconceptions. To be clear, diffraction isn’t (much) a function of light bending around an obstruction, but a reduction in the opportunity for light waves to interfere with one another due to a reduced number of waves being seen by the sensor. Having one’s light source partially occlude the image cone should certainly reduce resolution—but it does so mostly by decreasing the utilized aperture (that portion of the aperture actually filled with light), not much by creating a bend in the light waves.

It took me a while to disabuse myself of the widely promulgated “light bending around the aperture” creation myth for diffraction. So I’m a bit sensitive about potentially leading others into that misconception. An excellent treatment is Rik’s “What really causes “diffraction blur.”

Chris R.’s demonstration caused me a lot of head scratching. I really didn’t want to believe that reducing NA could elicit details that a larger NA did not. But to my eyes, he’s elegantly demonstrated this point. And he seems to have shown a path for reducing ghosting around foreground obstructions. This has been a significant—and as far as I know largely unsolved—problem.

In a highly automated system, I could see shooting a bunch of additional images through a revolving aperture placed in front of the objective—an aperture that is not centered, but situated against the edge of the entrance cone. The mechanics would be slightly challenging, and the post processing would be significant, but it might allow a lot of freedom in retouching in detail masked by out-of-focus frontal feature.

Nice job, Chris R.!

--Chris S.

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Nice explanation & demo!

To summarize...

When a OOF foreground object intrudes into the entrance cone, it does three things:
1) It contributes to the image some amount of unfocused light bouncing off the foreground.
2) It blocks some light from the focused background.
3) It reduces the utilized aperture for the focused background.

Elements 1 and 2, in combination, act to significantly reduce the apparent contrast of the background. The maximum reduction is roughly 50% for background adjacent to a single foreground edge, if foreground and background have the same average brightness. But if background is dark and foreground is light, the reduction in contrast can be arbitrarily bad. The reduction is also made worse when background is partially hidden behind multiple foreground edges, as when looking through a slit in foreground structures. All of this is well modeled by geometric optics and has nothing to do with diffraction.

Element 3 does bring diffraction into play in a limited manner. For background points near the foreground edge, the utilized aperture is reduced by up to 50%, but only in the direction perpendicular to the edge. In the direction parallel to the edge, the entire aperture continues to be utilized.

My guess is that most of the effects shown by ChrisR are due to elements 1 and 2: the reduction of contrast for the background, particularly in the vicinity of a bright foreground edge. Because focus stacking algorithms work by detecting contrast, anything that reduces that contrast makes it more likely that real detail will be missed. In addition, whatever real detail is retained will be at reduced contrast levels and thus harder to see.

So, overall quality in this case is a balancing act. Narrowing the aperture reduces the width of the contaminated band of background near foreground edges, while simultaneously reducing the resolution everywhere.

Does this sound right?

--Rik

ray_parkhurst
Posts: 3416
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Post by ray_parkhurst »

Chris S. wrote:Ray, though I’m sure you understand diffraction, I’m concerned that your example might be misunderstood in a way that reinforces common misconceptions. To be clear, diffraction isn’t (much) a function of light bending around an obstruction, but a reduction in the opportunity for light waves to interfere with one another due to a reduced number of waves being seen by the sensor. Having one’s light source partially occlude the image cone should certainly reduce resolution—but it does so mostly by decreasing the utilized aperture (that portion of the aperture actually filled with light), not much by creating a bend in the light waves.

It took me a while to disabuse myself of the widely promulgated “light bending around the aperture” creation myth for diffraction. So I’m a bit sensitive about potentially leading others into that misconception. An excellent treatment is Rik’s “What really causes “diffraction blur.”

Chris R.’s demonstration caused me a lot of head scratching. I really didn’t want to believe that reducing NA could elicit details that a larger NA did not. But to my eyes, he’s elegantly demonstrated this point. And he seems to have shown a path for reducing ghosting around foreground obstructions. This has been a significant—and as far as I know largely unsolved—problem.

In a highly automated system, I could see shooting a bunch of additional images through a revolving aperture placed in front of the objective—an aperture that is not centered, but situated against the edge of the entrance cone. The mechanics would be slightly challenging, and the post processing would be significant, but it might allow a lot of freedom in retouching in detail masked by out-of-focus frontal feature.

Nice job, Chris R.!

--Chris S.
I'm not sure it matters what way you look/calculate the blurring effect, the end result is the same. Place an object in the way of your light path, and you get blurring of the resulting image. If the object you place in the path is an aperture, then you affect the size of the airy disc. If the object is the edge of a "hole" as Chris R shows, or in my case the edge of the body of an LED light, you get a similar effect, though with a convex shape such as the LED light the interference patterns are shaped differently.

Imaging the bottom of a hole is difficult to do, as the hole is effectively an aperture that restricts the angles of light that can pass through the lens onto the sensor. Deeper/smaller hole implies narrower angles and more blurring.

ray_parkhurst
Posts: 3416
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Post by ray_parkhurst »

How would telecentricity affect this effect? Telecentric lenses are able to image with good contrast down to the bottom of a hole, ie with strong foreground components.

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Telecentrics can look to the bottom of holes that are perpendicular to the focus plane because their entrance cones are also perpendicular to the focus plane.

It's the match between the orientation of the holes and the orientation of the cones that provides the special behavior.

If we weren't so fond of drilling perpendicular holes, we would be less impressed by telecentrics.

--Rik

Chris S.
Site Admin
Posts: 4042
Joined: Sun Apr 05, 2009 9:55 pm
Location: Ohio, USA

Post by Chris S. »

rjlittlefield wrote:So, overall quality in this case is a balancing act. Narrowing the aperture reduces the width of the contaminated band of background near foreground edges, while simultaneously reducing the resolution everywhere.

Does this sound right?
Almost, I think. Chris R. isn’t suggesting that we accept the reduced resolution everywhere, but that we shoot at two different apertures and apply the effect of the smaller aperture only in those places where its strengths and weaknesses offer an improvement.

--Chris S.

ChrisR
Site Admin
Posts: 8668
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

Ray - if the restriction of the effective aperture near the edge of the well were doing doing very much, I'd expect to see detail there become a little less out of focus. It's not something I've seen. It would be hard to tell I think on this bit of moss. A step arranged by laying a coverslip on a microscope slide micrometer, with lines disappearing under the slip, might show it.

--
Rik wrote:So, overall quality in this case is a balancing act. Narrowing the aperture reduces the width of the contaminated band of background near foreground edges, while simultaneously reducing the resolution everywhere
.
Not quite. I'm seeing it as a balance beween shiny apples and rotten eggs :)
If you increase NA, the sharp gets sharper, but in the crannies, the visible becomes completely invisible.

We can combine the two, or indeed any number, with fortuitous apertures appropriate for the local subject structure, but only if we don't have alignment issues.
Telecentricity would help with that, but only to resolve geometry variation. The problem I'm having with turning alignments off, is almost certainly to do with my set-up. X_Y shifts, not scale.

There is a small alignment query - whether two frames with everything unmoved except for aperture, would align the same way with the rest of the stack. Features appear to move around when they're out of focus, as we've seen elsewhere. I imagine the aligner being guided by that to some extent.


----

Ray - I reckon you could put a small light between objective and subject with relative impunity. As long as it has no light on its upper surface, it's just "dark".
A Mitty glass is about 20mm diameter, say 314 sq mm. Losing 10sq mm wouldn't matter much?
If you took a SMD LED on a couple of fine wires, like the aerial camera at the ball game, you could fly it about and I don't think you'd see it. They're quite small ;)
If you replaced the LED with a mirror and used a collimated side-light, you'd have something more directional. (There are fibre-optic scale mirrors..) You could project an image of a very small spot 1/5th the size of a 250micron led chip, perhaps using your 5xMM lens and its whacky WD.
I'd be worried about colours at the edges of the beam etc, but dunno.



I was previously having visions on an old polarising scope rotation stage with x-y, and a mirror/led/fibre optic strand, on a twistable wire:
Image

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Chris S. wrote:
rjlittlefield wrote:So, overall quality in this case is a balancing act. Narrowing the aperture reduces the width of the contaminated band of background near foreground edges, while simultaneously reducing the resolution everywhere.
Does this sound right?

Almost, I think. Chris R. isn’t suggesting that we accept the reduced resolution everywhere, but that we shoot at two different apertures and apply the effect of the smaller aperture only in those places where its strengths and weaknesses offer an improvement.
Agreed -- good catch.
ChrisR wrote:We can combine the two, or indeed any number, with fortuitous apertures appropriate for the local subject structure, but only if we don't have alignment issues.
Telecentricity would help with that, but only to resolve geometry variation. The problem I'm having with turning alignments off, is almost certainly to do with my set-up. X_Y shifts, not scale.

There is a small alignment query - whether two frames with everything unmoved except for aperture, would align the same way with the rest of the stack. Features appear to move around when they're out of focus, as we've seen elsewhere. I imagine the aligner being guided by that to some extent.
The aligner will definitely be guided by OOF movement. Its concept of "best match" is based on difference in pixel values across the entire image. OOF areas have small differences for each pixel, but there are lots of pixels, so summing them all up can definitely have an effect. That's what causes some of the edge streaks we see in high mag stacks with asymmetric subjects. I have no data on how much misalignment would result from shooting the same stack at two different apertures.
Ray - I reckon you could put a small light between objective and subject with relative impunity. As long as it has no light on its upper surface, it's just "dark".
A Mitty glass is about 20mm diameter, say 314 sq mm. Losing 10sq mm wouldn't matter much?
This is very clever. I agree completely. Notice also that the area you're talking about is considerably smaller than occlusion by the central mirror in a reflecting objective.

The biggest problem I foresee with this type of illumination would be increase in speckle due to the small size of the LED.

--Rik

Horstl
Posts: 32
Joined: Wed Feb 02, 2011 4:14 am
Location: Austria

Re: Reduced NA gives more detail

Post by Horstl »

ChrisR wrote: .... A confocal microscope excludes stray light with a pinhole at the sensor. We can't do that but we could reduce the light coming from unfocused parts by not illuminating them.
This could be done above (“Epi”), or subject-side of the objective. But they’re both a little fiddly so I’ll leave them until after lunch.
Chris,
thank you for this interesting post.
Do you think that some kind of structured Epi-illumination could work?
There are some alternatives to laser confocal fluorescence microscopes on the market, working with a moving grid in the illumination path and forming an image by overlaying several images with the grid in different positions. I was thinking for a wile if this could be done in a simplified DIY-way (for fluorescence). Not to make high-end confocal intruments obsolet :lol: , but as an interesting field for experimenting.

Horst

ray_parkhurst
Posts: 3416
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Post by ray_parkhurst »

ChrisR wrote:Ray - if the restriction of the effective aperture near the edge of the well were doing doing very much, I'd expect to see detail there become a little less out of focus. It's not something I've seen.
That's exactly what it looks like to me near the > in detail A area. In fact the farther right you go in the > the more "diffraction blurry" the edge becomes and the more obscured the detail at bottom of the "hole" becomes. This is consistent with a cobined interference effect coming from both sides of the >.
--
ChrisR wrote:Telecentricity would help with that, but only to resolve geometry variation. The problem I'm having with turning alignments off, is almost certainly to do with my set-up. X_Y shifts, not scale.
Reason I thought of telecentricity is that since the cones are vertical it might reduce the local contrast reduction due to cones that come in at slight angles and are thus most egregious in their reflections and blockages in the OOF areas.

----
ChrisR wrote:Ray - I reckon you could put a small light between objective and subject with relative impunity. As long as it has no light on its upper surface, it's just "dark".
A Mitty glass is about 20mm diameter, say 314 sq mm. Losing 10sq mm wouldn't matter much?
If you took a SMD LED on a couple of fine wires, like the aerial camera at the ball game, you could fly it about and I don't think you'd see it. They're quite small ;)
If you replaced the LED with a mirror and used a collimated side-light, you'd have something more directional. (There are fibre-optic scale mirrors..) You could project an image of a very small spot 1/5th the size of a 250micron led chip, perhaps using your 5xMM lens and its whacky WD.
I'd be worried about colours at the edges of the beam etc, but dunno.
I've tried the individual LED approach, but they tend to be extreme pinpoint sources and create "red and blue sparklies" or "speckle" as Rik pointed out.

What I have found is its not so much a percentage of the open aperture that makes the difference, but distance the blockage is from the direct rays coming from the coin. I have built a "vignetting" type of lighting structure out of a fabric cone, and my initial concept was to vignette the coin fairly closely in order to eliminate stray light coming in from other areas of the field. But if I get "too close" to the coin edges, I see illumination falloff and a simultaneous blurring of the edges of the coin. The center remains bright and sharp, so it is a local effect similar to the blurring you see at the edges of an aperture when you are viewing the aperture with focus at aperture plane.

I first noticed this effect when using two lights (always with black absorbers to keep stray light from hitting the lens) at the standard 10:00 and 2:00 position, but close enough to the coin to obscure some of the field of view. Here are some pics to show the effect. The light on the left is outside the field of view, while the light on the right is blocking between corner of the field and edge of the coin.

f/22 Overall shows the light position clearest
Image
f/22 Left side crop
Image
f/22 Right side crop
Image

f/4 Overall the obscuring light shows up less due to more light paths to create the image
Image
f/22 Left side crop
Image
f/22 Right side crop
Image

Note I critically-focused on the background rather than the coin for this sequence to show effect in focal plane.

If I had focused on the coin, you would see that the OOF background areas distort along the axis of the blocking light. You can see this effect on the images above, but it's less pronounced because of the nature of the coin features.

ChrisR
Site Admin
Posts: 8668
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

I've tried the individual LED approach, but they tend to be extreme pinpoint sources and create "red and blue sparklies" or "speckle" as Rik pointed out.
I thought you liked speckle :? .
Yes sure you'd get it, but not the same as suggested if you read on, using a mirror to illuminate a small area of the subject.
Sorry its drawing wasn't up to much!
Perhaps you could take a bunch of exposures with the thing in slightly different position and average them out or avoid the effects you don't like? I don't know - I don't photograph coins much!

--
ChrisR wrote:Ray - if the restriction of the effective aperture near the edge of the well were doing doing very much, I'd expect to see detail there become a little less out of focus. It's not something I've seen.
Ray wrote:That's exactly what it looks like to me near the > in detail A area. In fact the farther right you go in the > the more "diffraction blurry" the edge becomes and the more obscured the detail at bottom of the "hole" becomes. This is consistent with a cobined interference effect coming from both sides of the >.
I understand what you mean but that's exactly not what I'm referring to.
Less out of focus = not the same as more diffraction blur.
My frames with the lens at full aperture show no detail in the bottom of the >, where the stopped down ones do. That slight softening towards the end is small in comparison. Go left a third of the box width, the large aperture pic is less sharp than the smaller aperture one.
The thread's about trying to get more detail. Yes ultimately the restriction of the edge of the well reduces the effective aperture, as original post,
"The intervening structure also reduces the working NA", but that's not something we can do a lot about here. The point of restricting the aperture is to help with the lighting problem which kills the contrast completely. That's the bad egg problem, which is "no image at all", not the unshiny apple one, "not quite as sharp as it might have been".
Last edited by ChrisR on Mon Apr 29, 2013 11:12 am, edited 1 time in total.

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Chris, perhaps the issues might become more clear if we could see crops from individual frames, as shot, focused on the background where detail is lost at high NA.

--Rik

ray_parkhurst
Posts: 3416
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Post by ray_parkhurst »

ChrisR wrote:
I've tried the individual LED approach, but they tend to be extreme pinpoint sources and create "red and blue sparklies" or "speckle" as Rik pointed out.
I thought you liked speckle :? .
Yes sure you'd get it, but not the same as suggested if you read on, using a mirror to illuminate a small area of the subject.
Sorry its drawing wasn't up to much!
Perhaps you could take a bunch of exposures with the thing in slightly different position and average them out or avoid the effects you don't like? I don't know - I don't photograph coins much!
I don't personally mind the speckle much, but I get a lot of grief when I post pics online with speckle. The blame always goes to LED lighting, which is partially true, just not because of the technology but because of the physical size of the source.

--
ChrisR wrote: I understand what you mean but that's exactly not what I'm referring to.
Less out of focus = not the same as more diffraction blur.
If we could rely on the edges of the well, or the >, to restrict the aperture, then there would be no point stopping the lens down. But my frames with the lens at full aperture show no detail in the bottom of the >, where the stopped down ones do. That slight softening towards the end is small in comparison. Go left a third of the box width, the large aperture pic is less sharp than the smaller aperture one.
The thread's about trying to get more detail. Yes ultimately the restriction of the edge of the well reduces the effective aperture, original post
"The intervening structure also reduces the working NA", but that's not something we can do a lot about here. The point of restricting the aperture is to help with the lighting problem which kills the contrast completely. That's the bad egg problem, which is "no image at all", not the unshiny apple one, "not quite as sharp as it might have been".
I understand the point you are making, but I am still not convinced in the relative proportions of the root causes. The upper stack is with high-NA, while lower is with low-NA, correct? An issue I have is that the high-NA stack looks to have much lower resolution in areas that should not be affected by the ledge-contrast issue. Looking at the green blob in upper left, here is the high-NA vs low-NA stack, cropped from your images, compared with the crops of the corner of the > from same images:

..................high-NA...........................................low-NA.................
Image Image
Image Image

Or am I somehow mixing things up? Because the entire low-NA image seems more detailed. In my view, the reduced contrast in the area to the left of the > could be simply due to the aberrations-induced contrast issues in the high-NA image, while the area near the right side of the > are further exacerbated by "diffraction blur".

I'm not trying to be argumentative, just trying to understand what I'm seeing using the limited optical physics I've been taught...Ray

edited to add that I put the crops of the associated > areas below the green blob crops for comparison...

Post Reply Previous topicNext topic