Let's begin at the end, with a fairly nice result:
Despite its apparent simplicity, this stack is really very difficult to handle well. Here's the area that Steve was wrestling with:
The basic problem here is that we have a brightly illuminated foreground structure seen against dark OOF background, with even darker focused background, the black feather. As the bright foreground goes out of focus, its brlght blur spreads across the background. This has two ill effects. First, the bright blur obscures whatever is in the background that we'd like to see. Second, because the edge of the spreading blur is fairly well defined, it appears to the stacking software to be "interesting" detail that's worthy of being preserved, and this tends to cause halos.
I ran the stack through a variety of fully automatic methods: Helicon Focus Methods A and B [*], Zerene Stacker DMap and PMax, and Photoshop CS5 with and without seamless transitions. I'll save the Photoshop results for later. Here's what Helicon and Zerene did:
As you can see, all of the automatic methods produce halos of one sort or another. In Helicon, there aren't any controls to specifically address those, so I'd probably attack with either retouching inside HF (tedious, since the affected region spans 14 source images) or by masking in Photoshop after saving out of Helicon. Zerene provides the DMap contrast selection slider, which is not completely effective when applied to a DMap of all the frames, but works quite well when applied to a selection of the 14 frames that show the hook and tail feathers in focus. Zerene's PMax works quite badly in this area, showing every one of PMax's classic artifacts --contrast increase, inversion halos, and introduction of starlike flare patterns -- plus a sort of false-detail artifact that was new to my list.
I produced the final result shown at top of post by using Zerene retouching to combine several sources. The bright hook and tail feathers are mostly from a DMap of 14 frames selected and slider adjusted to give a clean rendition of that area. Most of the remaining area is from a PMax output, which did the best job with those heavily overlapping hackles. The vertical pin that holds the fly, and one bit of the hook, were most easily retouched from two individual source images.
The remaining problem was that troublesome background feather that Steve asked about. I'm conflicted about the best way to handle that, so I'm showing you two approaches. The big image at the beginning is done with Zerene retouching, using a small brush to stroke over each bit of feather, and reaching as far into the bright blur as I could go without getting artifacts. The result looks pretty good at this scale, but when pixel-peeping I'm bothered by the remaining gap where there just isn't any visible detail to paint. I guess if I were so inclined, it wouldn't be hard to just clone in a reasonable facsimile using Photoshop, but I'm generally a "selectionist" rather than a painter. So the other option I'm showing here is to just get rid of that one feather altogether, using the simple expedient of not restoring it into the DMap'd area where it was naturally gone very OOF anyway. I think any of these approaches makes fine sense, and there are probably others that haven't occurred to me.
Ah yes, one final thing...I promised to show you the Photoshop result. I have no idea why, but it botches the alignment, and from there on the situation is hopeless.
I hope this is helpful. Many thanks to Steve for raising the issue to begin with, then letting me use his stack for this discussion.
--Rik
[*] This study was made in Feb 2012, before Helicon Focus added Method C. The HF Method C result should be similar to Zerene Stacker's PMax. (Comment added 6/22/2016.)
