Zerene DMap - Obtaining "Messy" Results

A forum to ask questions, post setups, and generally discuss anything having to do with photomacrography and photomicroscopy.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

photomac
Posts: 8
Joined: Sat May 03, 2014 10:24 am

Zerene DMap - Obtaining "Messy" Results

Post by photomac »

I have been struggling to obtain satisfactory results using the DMap function within Zerene Stacker. You can see in the attached example that a significant portion of the stacked image remains blurry and out of focus. I could spend hours cleaning it up but that seems to defeat the purpose of using the software. I am hoping that there may be some users that can suggest some helpful changes in my process. For this image, I proceeded as follows:
-imported approximately 140 source images
-organized those 140 images into 28 subgroups
-each subgroup had 7 images, with two of those images overlapping with the next subgroup. So the first subgroup had images 1-7 and the second subgroup had images 6-12.
-Set the DMap estimation radius to 20
-Ran the DMap function on the 28 subgroups
-Finally ran the DMap function on those 28 stacked subgroups to create my final image

Can anyone suggest improvements to produce more reliable DMap results?Image

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Hi, this is the software author speaking. In the image as downsized here, I can't see what's going wrong. In general, an estimation radius of 20 seems unusually large for an image as detailed as I imagine this one would be. Aside from that, there are possibly issues involving the setting of the contrast threshold slider. In places, the 3D structure is complicated enough that I'm not sure DMap will do a great job even at best, so it may be necessary to factor in PMax also. The DMap/DMap slabbing process is a bit unusual and I'm not sure how that plays in.

I suggest posting some actual-pixel crops of source and output images so that we can see more clearly what's going on. Or if you'd like to email that information to support@zerenesystems.com, I'd be happy to take a look at it offline.

--Rik

Chris S.
Site Admin
Posts: 4042
Joined: Sun Apr 05, 2009 9:55 pm
Location: Ohio, USA

Post by Chris S. »

Photomac,

Would you mind also answering several questions:

What is the approximate size of the subject field?
What optics did you use?
What lighting regime did you use?
How did you perform the mechanics of acquiring focus-stacked input images?
What incremental movement did you use between frames?
Did you use any autofocus or autoexposure?
If you used autoexposure, did you close the viewfinder, if you were using an SLR? If not, did you ever put your eye to the eyepiece--or if you kept it mostly there, did you remove it at any time?
Was this shot taken in a studio, as it appears, or outdoors?
Was there opportunity for the subject or camera to move relative to one another during image acquisition?


I certainly see problems with this image, but would be surprised if DMap is the cause. The horizontal band of reduced contrast and sharpness near the top of the image suggests to me that something is going wrong during image acquisition.

And unless this subject is much smaller than my intuition suggests, 140 sources images seems like way too many. But I'll reserve further comment on this, pending your answers.

BTW, what you call "subgroups" seem to be what are more commonly referred to as "slabs" or "sub-stacks" at this forum. You probably already know this, but there are three free or nearly-free utilities to automate the process of making slabs in Zerene Stacker. To find them, you can query "SlabberJockey," "Bugslabber," or "ZereneVS."

--Chris

photomac
Posts: 8
Joined: Sat May 03, 2014 10:24 am

Post by photomac »

Hello Chris, thank you for your reply. I'll answer your questions as best I can from memory - hopefully the additional information will shed some light on areas where my technique can improve:

-Approximate size of subject field: I was shooting an area approximately 1.5' x 2.0'
-Optics: Nikon 60mm 2.8 micro lens on a Nikon D800
-Lighting: two speedlights - one hitting a sheet behind the flowers to create the white background and the second hitting the flower from camera left (with a fill card on the opposite side)
-Mechanics of acquiring stack: I used a Novoflex mini focusing rack
-Incremental movement: I don't recall exactly - I borrowed the focusing rack and used the grid on the rack for guidance, but as you mentioned I shot a lot of photos so I was moving the rack in small increments
-Autofocus or Autoexposure: No, manual for each
-Eye to eyepiece: No, just adjusted the focusing rack as needed
-Shot location: Studio
-Relative camera/subject movement: camera moved progressively closer to the flowers on the focusing rack

And thank you for the suggestion on the slab software - do you know if any of them work with a Mac? I was under the impression that the SlabberJockey did not, but I could be wrong

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

photomac wrote:-Approximate size of subject field: I was shooting an area approximately 1.5' x 2.0'
-Optics: Nikon 60mm 2.8 micro lens on a Nikon D800
-Mechanics of acquiring stack: I used a Novoflex mini focusing rack
-Relative camera/subject movement: camera moved progressively closer to the flowers on the focusing rack
This combination raises a red flag. When you move the camera to change focus with a large subject, you introduce perspective changes that can wreak havoc with stacking. It's far better to leave the camera in one place and change focus by turning the ring on the lens. See http://zerenesystems.com/cms/stacker/do ... versusrail for a discussion of the perspective change issues.

If your lens is AF-capable, then there is tethering software that can automate this process while avoiding any need to physically touch the lens. For more information about tethering, see the tutorials about ControlMyNikon and Nikon Camera Control Pro 2 linked at http://zerenesystems.com/cms/stacker/do ... rialsindex for more information about tethering.

Regarding the required number of frames, see the tables at http://zerenesystems.com/cms/stacker/do ... romicrodof. With a field width of 2 feet on a fullframe camera, your magnification is about 1.5" / 24" ~= 0.06X. At f/8, which would provide a good balance between maximum sharpness and number of frames, a reasonable estimate for DOF is around 45 mm (by interpolating between table entries).

I agree with Chris that you've shot far more frames than needed, but that by itself should not cause difficulties with stacking except for the extra time involved.

At the moment, I'm guessing that the biggest problem is changes in perspective caused by camera movement.

--Rik

photomac
Posts: 8
Joined: Sat May 03, 2014 10:24 am

Post by photomac »

Hello Rik, thank you for your answers. This is great information and may help me nail down the best technique to use going forward. Per your previous request, I've attached a few detailed pics so you can see the type of blury and distorted results I am getting.

Based on your first answer, it seems like I should consider the following:
-smaller estimation radius
-PMax slabs first and then try doing a DMap process on the those sub stacks. I was avoiding doing the PMax slabs because I had read that PMax introduces color variations as well as noise and contrast issues.

However, you second answer suggests that the focus rail may be a larger problem. I have a couple of follow up questions:

-Is there a time when focus rails are appropriate? Perhaps smaller subjects? It seems a lot of stackers enjoy using their rails so I assumed it would help produce better results.
-Will the tethering software help overcome the problem of my lens' very short focus throw? I gave up on manually focusing with the focus ring because my lens simply doesn't afford that much specific/accurate control.

Image
Image
Image
Image

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

photomac wrote:Per your previous request, I've attached a few detailed pics so you can see the type of blury and distorted results I am getting.
Thanks for the images. I'll take a closer look at those later. On quick scan, I see a lot of defects that are typical of DMap in general but possibly are being made much worse with perspective changes. So I'm still at the "can't tell for sure" stage, but the next thing to do is definitely to switch strategies for stepping focus.
-Will the tethering software help overcome the problem of my lens' very short focus throw?
Yes. Tethering software can run your lens at the finest increments of its focus motor, which by design is more than fine enough to focus precisely at any distance even at widest aperture. Typically those finest movements are too small to see the ring turn, even though it really is.

Regarding aperture, my usual recommendation is to stop down as far as possible while retaining whatever sharpness you need, then determine a good step size based on that aperture. Shooting wider than you have to introduces two problems: 1) it provides more opportunities for artifacts to creep in during the automated processing, and 2) it requires more frames, which makes retouching more tedious.
-PMax slabs first and then try doing a DMap process on the those sub stacks. I was avoiding doing the PMax slabs because I had read that PMax introduces color variations as well as noise and contrast issues.
You're correct about the issues with PMax.

However, for the application shown here I would not anticipate a need for slabbing at all. Slabbing is great for high mag stacks with hundreds of frames, but when you get your aperture and step size optimized for a subject this large, the number of frames should be small enough to handle at one go.

At worst, you might need to touch up troublesome overlap regions from a PMax output, as discussed in the retouching tutorials.
-Is there a time when focus rails are appropriate? Perhaps smaller subjects? It seems a lot of stackers enjoy using their rails so I assumed it would help produce better results.
At very high magnification there's no substitute for using a rail or its equivalent such as a microscope focus block. At somewhat lower magnifications they're less than ideal from an optical standpoint, but still the optical problems are not too bad and the convenience is great so the rail is still a good method. As the magnification gets progressively lower, the rail gets progressively worse, and at some point it becomes unusable. The changes are smooth and continuous, so there's no brightline criterion for what's best in each situation. The matrix at the top of the Ring-versus-Rail article tries to summarize the tradeoffs.

--Rik

Chris S.
Site Admin
Posts: 4042
Joined: Sun Apr 05, 2009 9:55 pm
Location: Ohio, USA

Post by Chris S. »

Photomac,

Thanks for answering my lengthy list of questions. For me, your answers ruled out certain potential issues, and highlighted the possibility of others. Rik has already responded on the meaty issues, but I'll venture a few other thoughts.

While taking more shots than necessary isn’t a problem per se, it can add complexity—especially when you add slabbing to the equation. If one is having issues, eliminating unneeded complexity is a good idea while solving those issues. I’m a proponent of slabbing to deal with certain difficult subjects, but would not expect to find it useful in this particular photographic situation. And it seems to me that you have an order of magnitude more shots than you need, here. As Rik said, at f/8, given your subject field size, you’d have about 45mm of DOF—nearly two inches. So if you use a one-inch stacking increment, you should have lots of overlap. How deep is this spray of flowers, front to back—maybe four to six inches? If so, I’d be thinking along the lines of starting with maybe six images, with an inch or so of focus separation between each. Once you get that image with no issues, feel free to try finer increments and see if you find improvement.

I haven’t used a Nikon 60mm micro, but have lots of experience with Nikon 105mm and 200mm micro lenses. These have a nice, long focus throw, and for a short stack with one-inch increments, I’d probably turn the focus knob by hand. I do it often enough in the field, and find it pretty easy; I don’t overworry about having identical increments—just look through the viewfinder and guesstimate by eye.

Your stacked image strikes me as suffering from veiling flare, and thinking about your lighting arrangement, I wonder if the sheet behind the flowers is bouncing too much light onto the front of your lens? A litmus test might be to make a quick stack of the flower spray with no background or background light, possibly even adding a black velvet or similar background, if you have it. If the subject is sharper without the background and background light, you have flare. In that case, I’d try angling the white background to bounce much of the light away from the lens, and see if that clears up the flare.
And thank you for the suggestion on the slab software - do you know if any of them work with a Mac? I was under the impression that the SlabberJockey did not, but I could be wrong.
Sorry, I should have noticed in your original post—and from your forum name—that you work on a Mac. SlabberJockey is something I knocked together for my own use. Once it was in existence, I figured that I might as well offer it to others who might find it useful. But I use Windows PCs, and know very little about MacIntosh computers. SlabberJockey is a Microsoft Access application, and can be run under the free runtime version of Access. To the best of my knowledge, Access is available only for PCs—though perhaps some Mac wizard could teach us otherwise. However, that's probably unnecessary, as Bill Eldridge wrote a very nice program called Bugslabber, which runs on both Macs and Windows computers. The need for a Mac-compatible program was one of the main reasons Bill wrote it. I briefly tried Bugslabber in a beta version, and thought it seemed pretty good. Last I heard, you just need to PM Bill if you want to try his program. As I recall, if you like Bugslabber and intend to keep using it, he asks that you make, on the honor system, a $10 donation to a charitable organization that fosters girls’ science education.

But again I’ll say that for this particular image, I don’t see a clear benefit to be had by slabbing.
PMax slabs first and then try doing a DMap process on the those sub stacks.
I do this frequenty, in order to get the advantages of each approach (PMax and DMap), while avoiding their disadvantages. For a 100x, 1500-image stack of a difficult subject that Zerene Stacker seems to miraculously assemble out of a cloud of blur—yes, try this, especially for a low-key image where the darker sections have increased potential to accumulate noise. But for a subject like yours—high key, capable of being shot with a shallow stack—no. Zerene Stacker can probably handle this situation in its sleep.
I was avoiding doing the PMax slabs because I had read that PMax introduces color variations as well as noise and contrast issues.
While technically true, I think too much worry exists about this PMax trait—which I hesitate to call an “issue.” I use PMax and DMap about equally, and for situations where PMax works better, correcting color and contrast takes me less than a minute in Photoshop, with a curves layer. Your D800 sensor should produce very little noise, at base ISO, so even if it accumulates, I think you’ll usually find it not to be a problem. And if it is, you can always perform noise reduction on the input images or the stacked image.

Best,

--Chris

photomac
Posts: 8
Joined: Sat May 03, 2014 10:24 am

Post by photomac »

Chris, thanks again for your helpful answers. I didn't realize you were the creator of SlabberJockey! Unfortunately I am Mac dependent so I won't be able to try it out. I will try Bugslabber in the future if the situation calls for it.

I suppose that brings us to a central issue of my stacking technique. It appears I've been taking far too many photos which is an unnecessary drain on resources. I'll have to go back through the photos and try to figure out how much DOF I was obtaining. If I do a simple calculation on DOFMaster though, f8 and 20 inches distance from camera to subject gives me .8" of DOF. That would require a few more frames than you suggested, but still far, far short of the 100+ that I shot. I certainly agree there is no need to take extra unnecessary shots every time I attempt a stack- I'll study up on how to calculate more accurate stacking increments.

I do wonder about the potential of light spraying into the lens given my setup. Do you mind pointing out to me any examples of veiling flare you noticed in my examples? I would just like to be able to spot it for myself in the future so I can determine whether I need to tweak my setup. I like the 'glowy' effect I get with light behind the subject, but I'm wondering if that is offset by flaring issues.

The good news in this discussion is that shooting in preparation for a stack will be much less time intensive going forward. My 60mm does not have the generous focus throw of the 100mm so I'm going to try using some focusing software as Rik suggested. I know the Nikon version has a free trial, so i'll see how that works. Thanks again for all the help!

Chris S.
Site Admin
Posts: 4042
Joined: Sun Apr 05, 2009 9:55 pm
Location: Ohio, USA

Post by Chris S. »

Photomac,
photomac wrote:If I do a simple calculation on DOFMaster though, f8 and 20 inches distance from camera to subject gives me .8" of DOF. That would require a few more frames than you suggested. . . .
I wouldn’t worry that two different DOF calculation tools gave different numbers, as they probably use different formulae and assumptions, and the question of "optimal increments" is very much an open one at present. The source Rik referenced in this thread (and indeed, created) is excellent, but you need not worry about such differences right now. A DOF calculation tool just gets you into reasonable territory; after that, personal experimentation is the best way to determine what focus increments work for you. And as we agree, "reasonable territory" under any calculation system involves many fewer frames than originally used here.
I do wonder about the potential of light spraying into the lens given my setup. Do you mind pointing out to me any examples of veiling flare you noticed in my examples? I would just like to be able to spot it for myself in the future so I can determine whether I need to tweak my setup. I like the 'glowy' effect I get with light behind the subject, but I'm wondering if that is offset by flaring issues.
Pointing out examples in your image is difficult for me, as the whole frame--not just portions of it--says “flare” to my eye. What I see reminds me of the lack of contrast I see in my own images if light is spilling onto the front of my lens.

A good test for figuring out whether this is actually happening, and if so, giving your eyes practice in seeing it, would to take a few test images with no light at all on the background, nor any shining on your lens. Does your subject appear with far more contrast? I suspect it would. If so, your job is to obtain your white background without losing subject contrast. But you say you like the glowy effect—and indeed, some photographers do intentionally seek out flare. I’m just not one of them. I do, however, often like the glow of backlight shining through plant structures.

BTW--if you're not already considering this: I'd expect trouble if my white background were completely blown out: 255, 255, 255 in RGB terms. This might, of course, mean that the background is actually much brighter, and the sensor can't report higher values. Without the floral spray present, I'd want to place my background at no more than 250, 250, 250--and more comfortably, 240, 240, 240. Many photographers disagree--and their example shots generally look wrong to me, especially at the edges of the white-backgrounded subject. Remember that your floral spray has structures at its edges--hairs and other things--that will bend some of the light from the background around and in front of the spray's edges. If your background is hopelessly burnt out--even if you don't mind this--you probably don't want these subject areas to be burnt out as well.
The good news in this discussion is that shooting in preparation for a stack will be much less time intensive going forward.
Yes indeed! And you might even try running test shots at some small aperture such as f/32. Sure, you’d get diffraction blur; but you would also get most of the subject in focus in a single exposure, which would let you test a bunch of lighting regimes. Then, once that’s nailed down, you could open up, turn your speedlights down to compensate, and run your stack.

By the way, with this floral spray, subject motion might be a problem. I’ve found that unconstrained plant parts, even in studio conditions, can move around enough to ruin stacks. Contributing culprits include: Slight air movement in the room; drying out of the plant specimen, which can cause curling and other changes: and even movement induced by the flash itself.

Cheers,

--Chris

--edited typos
Last edited by Chris S. on Sun Jul 27, 2014 11:45 pm, edited 2 times in total.

photomac
Posts: 8
Joined: Sat May 03, 2014 10:24 am

Post by photomac »

Chris and Rik, thank you for your great replies! There are a lot of tips here that will really help advance my stack work. I'm looking forward to putting your advice into practice

Post Reply Previous topicNext topic