Beginner's macro stack

Just bought that first macro lens? Post here to get helpful feedback and answers to any questions you might have.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

ChrisR
Site Admin
Posts: 8669
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Beginner's macro stack

Post by ChrisR »

We see a lot of fine examples of stacking on the forum, with fine equipment and technique.
A friend is inclined to give up after a couple of results, :( dismal in comparison. Perhaps some examples illustrative of intermediate steps would be useful. I'd like to learn by mistakes other people make, I'll be dead before I've had a chance to make them all myself. (Did I see that on a cap badge?). I'd also like to be told what I'm doing wrong!

I have relatively basic equipment so far; a digital SLR, tubes, bellows, enlarging lens(es) and reversing ring, and a slide copying attachment which fixes to the bellows. Microscope lenses will have to wait, other than for flat subjects.

I've had some successes, but a lot of "opportunities for improvement".
I thought I'd post one result showing potential, with shortcomings.

Subject is impaled on a darning needle which is stuck into a cork, taped to a bit of bent tin hanging off the slide holder. Illumination is pop-up flash only, with a milk-bottle diffuser.

The only way I can focus practically, is to move the rear of the bellows. I can't really measure the steps so have to make them small, and hope. Moving the lens end would need impossibly small knob movements.
Doing that introduces the first snag - a big change in magnification from one end to the other and a crop (rear antenna) I didn't intend. Taken on 24 x 36, cropped from 4200 wide to about 2800 before stacking.
Mag is 4 ish, 50mm enlarging lens reversed at f5.6. Dim light so ISO2000.
Lots of frames. Only a few seconds each, so I didn't care. I was watching Wimbledon.

Image
The top two are the beginning and end shots, lower are Zerene Stacker's Pmax and Dmap output files.

The Pmax file is untouched but bigger here
First of course, the insect is filthy! I don't know how to clean them, other than a blower or "a hair from a paintbrush" Rik used. I admit I didn't look, first, I should have done. With so little DOF I didn't notice.

The change in mag has done strange things to the rear antenna, because it's OOF. The stack didn't go that deep because I ran out of bellows movement . OOPS!
For the first time (ever) I can see dirt on the sensor; a squiggle top right, a hair lower left and the piece of chewing gum, lower centre. The chewing gum shows as a line because of the zooming, and the ends of the line happen to mark where the lower end of the rear antenna should be. Ie it shows much too big. (I suppose I could have cloned from an unaligned frame in PS).
All the dirt is easy to retouch out in ZS.

The front antenna moved as it was being shot, so there's a bit of double imaging between 3rd and 4th segments. Luckily it moved between 2 frames as it was in focus, so again, easy to retouch in ZS.

The white dust, and other specular highlights, show a lot of smearing/glowing. I think that's a combination of the mag change, and what you get when you stack anyway. ?? ZS retouching helps, but Photoshop was necessary to remove the bright blobs.

In the areas where the was a lot OOF in "front" such as the parts which would have been behind the front antenna, the image is rather fogged. I don't know of a way to remove that. It's a pity because there is detail in the eye, etc.
Similarly, in the furry bits where only some hairs are sharp, there's a lot of mush. It's pretty ugly. Apart from getting the lot in focus, I don't know how to deal with it.
A question here - as I probably used too many frames, I have more OOF ones doing the fogging. On the basis, possibly unsound in stacking, that light + dark = light, has the number of frames made things worse??
With hindsight, I should have removed a tube and progressed, perhaps with a shot or two at small aperture so the OOF bits would have been less OOF. On the other hand the mag would have shifted more and I'd have a smaller wasp. ??

I think the lighting is reasonable - perhaps a reflector underneath would have helped, and I quite like the pose, apart from the leg, bunched up post mortem, lower left. I like the depth, and I wouldn't have liked the rear antenna too sharp.
The background was a piece of black paper. It's "fogged" again, by all that OOF stuff, and doesn't look good.
Maybe it doesn't look very sharp? It sharpens up a lot with USM, but maybe there's movement.

Comments please. My skin is as thick as I am.
As a sort of aside, here's what came out after a few mins in Photoshop, not really the subject of the post.
Image

rjlittlefield
Site Admin
Posts: 23564
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Chris,

It looks to me like you're doing fine. The biggest problem is the one that everybody runs into -- you've spent so much time with the image that you're painfully aware of its every fault!

Taking your points from the top...

Subject holding seems fine. I'm not sure what "tin" is. Around here, that term usually means plated steel cut from a can, which material is pretty springy. In that case, some copper or solder might be easier to work with. (Wait, you're a plumber, you know this already. OK, but I'll leave it in just because it might help somebody else.)

Illumination is working fine. Specular reflections are under control, nothing is going pitch black and losing detail.

Moving the rear of the bellows is fine. Actually it's ideal from one standpoint. When the lens doesn't move, relative to the subject, then there's no change in perspective throughout the stack. That minimizes problems with halo, though for sure it does not make them go away!

The big change in magnification is not a problem for the software. As you've realized by now, when you're focusing by rear movement it's a good idea to frame and adjust starting magnification with the most foreground frame, since that's where the magnification will be the largest and thus the field of view will be smallest.

ISO 2000... High ISO is never a good idea if you can avoid it, but running off pop-up flash, maybe you can't avoid it. Looking at the larger image you posted out, I don't see any big problem with noise levels. (Small problems, maybe, but more about that later.)

The DMap image is typical of what depth map algorithms give with this kind of subject. Regions with no overlap and with abundant fine detail look good, regions that show only OOF subject look bad. There's not much to recommend DMap for this kind of subject.

Dirty specimen, yep, dirt happens. Even preparing specimens under a dissecting scope, I overlook stuff that becomes painfully obvious in a high resolution stack.

You write that "The change in mag has done strange things to the rear antenna". There's more to it than that. Actually what happened is that the alignment process decided that your camera must have rotated slightly as it shot the stack. The rotation is not much, only about 3.4 degrees from first frame to last. But when that rotation combines with making the rear frames larger to compensate for changing magnification, the overall effect misleads your eye into thinking that something else has happened.

To illustrate, I have taken your montage and replaced the lower right image with a copy of the upper right image (the backmost frame), just scaled, rotated, and cropped to line up with the composite at lower left. I think you'll agree that this provides a pretty good explanation for why the antenna ended up where it did.

Image

To prevent rotations like this from happening in the future, just un-check the box in ZS under Options > Preferences > Alignment > Rotation. This essentially tells ZS that your setup prevents physical rotations, so don't mess with that.

Dirt on the sensor, yes. Notice that the dust track at lower right is curved. That's further confirmation about the rotation.

Smearing/glowing is not due to magnification change, it's due to accumulation of brightness from slightly OOF frames. All the pyramid algorithms I've seen tend to do this. I haven't figured out how to prevent it and still have the algorithm do the right thing everywhere else.

By the way, in the large image I notice that some of the bright white dust flecks near the base and above the front antenna look like starbursts. I suspect that's an interaction with your lens diaphragm. Nothing important, just an interesting aside.

That mushy area of eye above-left of the front antenna puzzles me a little. I've seen something like this before, when the actual detail is low contrast and dark, but there are other frames where the same region is bright and somewhat noisier (in an absolute sense) just because it is brighter. I might be able to do something about that in software, but it's hard to tell. The usual process is I change an algorithm to improve results under condition A, and that part works fine but they get worse under condition B. Anyway, it's on the list to look at. From your standpoint, the best thing you can do is to minimize noise -- shoot lowest ISO possible.

Furry bits where only some of the hairs are sharp, yeah, I don't know what to either, except get the lot in focus.

Optimum number of frames is a tough question. It's certainly not good to have too few. Generally I recommend having about 20% overlap between the in-focus slabs of adjacent frames. There is some penalty in having more frames with smaller steps, but I've always been bothered by the increase in time more than any effect on quality.

Your idea is probably good, to have progressed and even shot some at a smaller aperture. Software would have compensated for the shift in magnification.

By the way, just as a reminder, what ZS does right now is to process the frames in order, from top to bottom in the list. It uses the first frame as a master, aligns the second to the first, the third to the second, and so on down the list. It's generally good to start with the smallest field at the top of the list, as you did here. Processing in the other order would give you big streaky borders, with a clear central portion that covers only the same field you have here, but confined to fewer pixels.

As to sharpness, I'd expect better from my Olympus 38 mm bellows lens at f/2.8 or f/4, but from an EL Nikkor 50/2.8 at f/5.6, I dunno. Not much you can do about it anyway except be sure you're using the best aperture for your lens, or shell out for a much pricier one.

The final result looks good. Nothing to be ashamed of in this work!

--Rik

ChrisR
Site Admin
Posts: 8669
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

I appreciate the thorough analysis Rik, lots of good stuff there.
my Olympus 38 mm bellows lens
Is that the Oly fit one, or the RMS?

The hi-tech diffuser is one of
these
The chopped-off neck of the bottle fits round a 39mm tube on the back of the reversed lens.

rjlittlefield
Site Admin
Posts: 23564
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

I appreciate the thorough analysis Rik, lots of good stuff there.

You're most welcome. Writing makes me think. Thinking is good...or so I've been told. :wink:
Is that the Oly fit one, or the RMS?
The Oly, HERE. I have no idea how the two compare.

--Rik

Cactusdave
Posts: 1631
Joined: Tue Jun 09, 2009 12:40 pm
Location: Bromley, Kent, UK

Post by Cactusdave »

Hi Just caught up with this post. I'm impressed with your 'couple of minutes in Phtoshop'. I'd be interested in your work flow to remove those dust spots on wasp and sensor so neatly and cleanly. It's something I struggle with. I have to say I'd be pretty pleased with the final result you achieved.
Leitz Ortholux 1, Zeiss standard, Nikon Diaphot inverted, Canon photographic gear

ChrisR
Site Admin
Posts: 8669
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

Hi CD.
Well it's the Clone tool realy.
Remember Zerene Stacker is good for cloning from other layers in the stack to the same position, so that gets rid of sensor dirt, and some of the glare around the bright spots in the Pmax output.

I've been using PS for a while and though by no means a master, I'm several times better than when I'd only used it for a few days, say. If you see someone work with it who does it for a living, the speed is, without exaggeration, awesome.
Photoshop is sometimes like using a Combine Harvester when all you want is to mow the lawn. Several ways to do anything, and I find I often use several knobs at once where I used to rely on just one cos I didn't know what the others did.
So You might eg best use the pen tool to select, then feather it, twist stretch and rotate it, then clone from that on a separate layer in Darken mode. It's just a few clicks.

Experiment with Darken and Lighten in cloning, if you're not familiar.
Eg if you've got a dark hair on a light background, you can clone it to a midtone target in Darken and only copy the dark hair, not its light background. If you take the brush size in ZS too small, to just clone the hair, it doesn't quite work as you might expect - it becomes like having a "soft" or 50% transparent brush in PS.
Sometimes it's easier to use the pipette sample thing in PS to get a colour, then simply paint the detail in. Again you can use L and D. A pen tablet is good for that.

I didn't intend to do more than a quick job on retouching, but it's like doing it on a print, as long as the eye isn't drawn to the area, imperfections aren't noticed.

Now I'm going to look for the actual pixels and we'll probably see it isn't that good...
OK I found it, and spent a lot longer fiddling about trying to get this to work than in the original retouching.. :roll: As I expected, there are lots of errors and artefacts which shouldn't be there. I just did it so I could look at the wasp without seeing all the pony in the way.
:shock:
error - file too big - ..........
it's
666kB

rjlittlefield
Site Admin
Posts: 23564
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Yeah, those animated GIF's don't pack very well. Too bad there's no such thing as an animated JPEG.

Nice demo. You're right about the psychology. My first impression is vivid: dirty/clean/dirty/clean. If I study the "clean" image, aided by having the flashing pair, then I can see lots of artifacts in the cleaned up areas. But if I didn't have the flashing pair, and I didn't take the time to study, I would never notice.

--Rik

Cactusdave
Posts: 1631
Joined: Tue Jun 09, 2009 12:40 pm
Location: Bromley, Kent, UK

Post by Cactusdave »

Thanks very much for that comprehensive reply. Please don't be disappointed, but I felt my head starting to swim after the second line. :? The thing is Photoshop is a big complicated beast and I don't make my living from it. I thought I'd arrived when I learnt had to process a RAW file. :D I suppose I need to sit down and use it a bit more, maybe work through a good tutorial. Your results are so impressive to a non-acolyte and so casualy described that I feel like a kid looking into a sweet shop window with no money again.
Leitz Ortholux 1, Zeiss standard, Nikon Diaphot inverted, Canon photographic gear

ChrisR
Site Admin
Posts: 8669
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

I'm sorry to tell you that one tutorial is unlikely to do it :( :lol:
I was ill when I got CS2 so was able to sit through several days of training videos on CD. At that point I felt like someone who'd had a stick-shift gearbox and the car pedals explained, and had the task of driving across London. Definitely the wrong side of 14 years old to take it all in.

I'd suggest some tutorials on retouching for portraiture - there are masses on the net. And a book which covers everything for photographers, (rather than designers or art originators) such as by Martin Evening. You can often see s/h editions for older versions of Photoshop very cheap, and most of the basics haven't changed. (thank goodness!)

ChrisR
Site Admin
Posts: 8669
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

Had to have a try with Rik's most excellentset adventure.
I'd had a go at producing 3D pairs from a single stack of 5 frames in Photoshop, by shifting the frames manually - dismal results, so I'm glad to see Rik is on it!

I reran the wasp stack, (with rotation ability turned off this time) with settings thus:
Image
Perhaps the range is too wide?
It took overnight to run. 30 seconds per frame, x 147 images per stack, x 5 "views" (we need a word for these!) or was it two runs per view, as they're listed twice in the output window at the lower left of the ZS screen.

I haven't retouched anything. (Partly because I'm not sure how to. For each of the 5 output files, is there a separate set of 147 aligned source images ??)

I thnk about the best stereo pair I could make was this one, though I don't remember which frames I used. I think it was 2 and 3.
The Pair is sharpened, the others aren't.
Image

Here are the 5 images going from image 0 to 4...
Note the tell-tale black blob on the sensor as it tracks one way then the other, centre bottom.
Image
Image
Image
Image
Image

Perhaps I should have posted them 800 wide? Yup, but it's 1:30am - time for bed!

I have few comments only, I'll leave that to Rik.
The stack stopped part way through the subject, so the back portion would be dead flat.
The files (all Pmax) seem a lot softer/ lower contrast, than the original one, discussed above.
There's a lot of smearing of detail, which thinking about it I suppose we should expect? If there are too many frames, the same detail will be sharp in several of them, but be moved sideways in the 3D stack, so a dot appears as a line?
Much retouching would be needed, see how that front anntenna is transparent particularly in the first two , and last images.

I couldn't work out how to do the rocking image thing!

rjlittlefield
Site Admin
Posts: 23564
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

ChrisR wrote:Perhaps the range is too wide?
-5 and +5 are pretty wide, but I just finished doing a stack where they worked fine.

In any case, the five views would have been -5, -2.5, 0, +2.5, and +5, so the center three should be in good range.
It took overnight to run. 30 seconds per frame, x 147 images per stack, x 5 "views" (we need a word for these!) or was it two runs per view, as they're listed twice in the output window at the lower left of the ZS screen.
Let's use "view" for now.

There should be only one run per view, and each view should only be listed once. Did you perhaps use the new "Both" method, instead of just "PMax"?

30 seconds per frame is huge. You must be working with big images on a slow machine. Did you use Options > Preferences > Preprocessing to pre-size your images to 50% or smaller, as suggested by the documentation? For reference, my Intel Core 2 Q6600 @ 2.4 GHz does 6 megapixel images at 100% in 3 seconds per frame after the first pass. (The first pass is slower because it has to figure out the alignment too.) For stereo, I crank it down to 50% so it goes even faster, a little over 1 second per frame.
I haven't retouched anything. (Partly because I'm not sure how to. For each of the 5 output files, is there a separate set of 147 aligned source images ??)
Retouching doesn't work on shifted source images because the retouching tool doesn't know about the shifts. This is mentioned in the "Limitations" part of the documentation.

Transformed images are always generated on the fly as needed. At various times during processing, there were 5*147 different images in memory, but they never make it to disk.
I think about the best stereo pair I could make was this one, though I don't remember which frames I used. I think it was 2 and 3.
Well, um, "good start!" Actually, it looks pretty bad, and there's something weird going on.

The clearest indication of weirdness is that the center image, with shift=0, should be identical with the single image you got before (discounting the rotation tweak). But it isn't even close. Take a look at the front leg. In your single image at the top of the thread, the leg is cleanly yellow with no background showing through. In the center image here, it's overlaid by what look like big streaky hairs. Even the streaky hairs alone are odd, since the single image doesn't show anything like them in that area.

My first thought was that when you turned off Rotation you also turned off Scale, so what we're seeing here is the overlay of a bunch of images of different sizes. But on thinking more carefully, that exact cause doesn't make sense given the relative sizes of things and the trail of that dust spot.

But still, it looks like something weird is going on with scale. The streaks look radial, which is typical of scaling issues. These last images have a wider FOV than the first frame you showed -- notice that the first one cuts off in the middle of the front leg, while this last set includes all of the front leg, and some of the middle leg, and...hey, what gives??? These images include the knee of the middle leg and what looks like part of the wing, while the widest shot in the first set cuts off in front of that. This is either a different stack, or you didn't show us the whole frame the first time.

Now that I'm thinking about the full frame, I'm wondering about that bright/dark border on the left side. Where is that thing coming from? Is it really part of the scene, or did some unrelated frame get slipped into the stack, or is that part of the optics -- a light leak or stray reflection? If it's a light leak or stray reflection, that'll wreak havoc on the alignment.

I don't think I can figure this out without seeing the whole stack. Hopefully I've given you enough clues that you can figure it out yourself.
Note the tell-tale black blob on the sensor as it tracks one way then the other, centre bottom.
Yep, that's typical behavior for sensor dust, and a good indication of how the images are ordered.
The stack stopped part way through the subject, so the back portion would be dead flat.
True -- thanks for pointing that out.
The files (all Pmax) seem a lot softer/ lower contrast, than the original one, discussed above.
There's a lot of smearing of detail, which thinking about it I suppose we should expect? If there are too many frames, the same detail will be sharp in several of them, but be moved sideways in the 3D stack, so a dot appears as a line?
Yes, no, and sort of.

"Yes", there's a lot of smearing, but as mentioned above, I suspect that's caused by some scaling problem.

"No", we shouldn't expect to see nearly this much smearing, and any smearing we see should be horizontal only, never radial. In my tests, +-3% is usually clean and even +-4% doesn't look too bad.

"Sort of", the number of frames in the stack doesn't really matter much. What's critical is the DOF per frame versus the total DOF of the stack. If your focus step is smaller than it needs to be, then you get more frames but less shift per frame so things pretty much cancel out. The difference is only that using a finer focus step will give you something that looks like a smooth continuous smear, where using a coarser focus step may give you something that looks like echos.

Maybe some numbers will help. The shifts are symmetric around stack center. That is, the middle frame of the stack never shifts, while lower number frames shift progressively to half the total amount one way and higher number frames shift to half the total amount the other way. Suppose your maximum shift is 5% (0.05), you're going to make a 400-pixel image (800 pixels in the stereo pair), and you have 150 images in the stack. Then as you work your way through the stack, the shift is only 0.05*400/150 = 0.133 pixels per frame. Also suppose (why not?) you have so much DOF that each prominent feature dominates in 15 shifted frames (1/10 of the DOF of the whole stack). Then each such feature will get smeared out into a bar that is 15 * 0.133 = 2 pixels wide. That's a pretty big smear -- bigger than I'd be willing to put up with. But if you cut the shift by half (to 2.5%), and open up the aperture to cut the DOF per frame by half (to 1/20 of the stack), then the smear drops to 0.5 pixels, which looks pretty good.
Much retouching would be needed, see how that front anntenna is transparent particularly in the first two , and last images.
Bright background, dark foreground, and deep overlap is a bad combination, no doubt. But even though retouching doesn't know about the shifts, it still should work OK to retouch a shifted full-stack result with a shifted partial-stack result. So I'm thinking this retouch won't be too painful even with the current tools.
I couldn't work out how to do the rocking image thing!
That one takes a little practice. Search on something like make animated GIF.

--Rik

ChrisR
Site Admin
Posts: 8669
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

Last thing first - I can do them (there's one in this thread!) but I had the idea it was part of ZS..
Quote:
I couldn't work out how to do the rocking image thing!

That one takes a little practice. Search on something like make animated GIF.
See what you mean about the scaling. Sorry, I forgot that I must have cropped all the frames before I did the stack up top of the thread. I have a folder called "waspcrops" with a lot of pictures in it :oops: :oops: .

The white thing showing on the left of frame is the darning needle with which I impaled the victim. (It's really hard to catch them like that - I must get a net....)

I'll run it again when I can leave the PC for a while.

rjlittlefield
Site Admin
Posts: 23564
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

ChrisR wrote:Last thing first - I can do them (there's one in this thread!) but I had the idea it was part of ZS..
Nope, not yet.

The initial documentation was unclear on the need to use external tools. I had added some clarification yesterday, but probably after you had already read it.

--Rik

ChrisR
Site Admin
Posts: 8669
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

Upped the ram from 1GB to 3GB, using source images 1000 wide, and it's now screaming along at 13 seconds per frame. 2.66 GHz Pentium 4, definitely no malware running. Two and a half hours, then, to go.

I tried altering the priority of "javaw.exe" in Task Manager, at Rik's suggestion. It was "Normal". Raising or lowering came no difference to the speed of ZS while the PC's doing nothing else, but putting it to "Below Normal" makes a massive difference to the usability of the machine for other tasks while a stack's running. Would recommend anyone with a slow machine tries the same.

rjlittlefield
Site Admin
Posts: 23564
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Thanks for the report about priority, sounds useful.

The relatively slight drop in time even with drastic presizing suggests that most of your time must be going into just reading the images. In that case, you could make a big improvement by running your images through Photoshop to actually make them smaller. ZS does its "presizing" on the fly, every time the image is read but before it is processed any further. Images get read once for each view using PMax.

Just for my calibration, how big are your source images (pixel counts and file sizes)?

As a crosscheck, I just now tried running my recent lava rock stack on an old laptop, 1.3 GHz Pentium M with 1 GB physical RAM (default 760 MB for ZS). Source images are 3072x2048 pixels, jpegs, average file size 0.85 MB. Presized to 1024 pixels wide (33.3333%), the first pass of PMax ran at 7 seconds per frame including alignment, subsequent passes 4.2 seconds per frame.

The other thing that would help for speed is to make a subset of images that is not oversampled in depth. It sounds like you're using 147 frames to cover one antenna and not quite all of the head. Just offhand, I'd expect something like 50 frames to be adequate even for a high resolution version. For 400-pixel stereo, something like 30 would likely be just fine.

In general, I recommend to start small and work up, when learning a new tool. The problem with starting big is that test runs take so long that the learning curve gets stretched way out -- slow and frustrating.

--Rik

Post Reply Previous topicNext topic