www.photomacrography.net :: View topic - Sub-stacks
www.photomacrography.net Forum Index
An online community dedicated to the practices of photomacrography, close-up and macro photography, and photomicrography.
Photomacrography Front Page Amateurmicrography Front Page
Old Forums/Galleries
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
Sub-stacks
Goto page 1, 2, 3  Next
 
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Macro and Micro Technique and Technical Discussions
View previous topic :: View next topic  
Author Message
ChrisR
Site Admin


Joined: 14 Mar 2009
Posts: 7092
Location: Near London, UK

PostPosted: Tue Nov 08, 2011 1:26 pm    Post subject: Sub-stacks Reply with quote

Stacking only part ( say 10 images) of a whole (say 100) stack of images can be useful.
I haven't done it an awful lot because it takes (me) so long. I'd like someone else to tell us a better method than my ad-hoc approach, to deliver more potential.

As a starter, this was sharp end of a 15mm (?) ground beetle. ( It has been poted before). Viewed from the top the depth to width ratio is qute high, so there's a lot of images in the stack even for a low magnification. This situation I find leads me to hazy indistinct images, for a bunch of reasons I only partly understand. A lot of it has to do with lighting, I'm sure. At higher magnifications, everything gets harder of course.
The magnification here, is possibly about 4-5x onto most of a 24 x36 sensor. Lens not recorded, it was a year or more ago.
First is a quick orientation pic of the beetle (about 8 images in that stack at f/22, reversed kit lens),
then a Pmax output of the whole closer stack (238 images).
There was plenty wrong, as you can see. Blue, dust...
One of the sub stacks' result is shown, which came from just the (single remaining) antenna, which I think was 80 frames by itself. NB the antenna is "cleaned" of its background, so it can easily be trimmed or masked and dropped onto the remainder of the beetle.

The last image is the result of playing for what felt like long enough. There remain things ( eg artefacts) which would be altered if it were important.
There were about 5 layers in Photoshop, though a couple could have originated from more than one substack.
what I got to when I stopped playing, with 5 layers in Photoshop.
I did try to "work on" the whole image in Photoshop, but one single whole pmax wasn't good enough as a start.

I expect some of you are doing this better, and routinely, and I'd like to hear how Smile







Back to top
View user's profile Send private message
Chris S.
Site Admin


Joined: 05 Apr 2009
Posts: 2757
Location: Ohio, USA

PostPosted: Tue Nov 08, 2011 3:45 pm    Post subject: Reply with quote

Chris, I'm doing this a lot, though I call the sub-stacks "slabs." Not sure who coined this term--perhaps Rik? Anyway, I am finding the approach enormously useful, and suspect that in time, slabbing may be the rule, rather than the exception, in my workflow.

Based on some very useful guidance from Rik, I can do this pretty quickly, using an xml file that I call from Zerene Stacker's batch dialog. For different stacks, I edit the file. I suspect that Rik is already typing guidance, and will do a far better job of it than I would, but I'll email you the file I use, which you can edit for your purposes.

--Chris
Back to top
View user's profile Send private message Send e-mail
DQE



Joined: 08 Jul 2008
Posts: 1653
Location: near Portland, Maine, USA

PostPosted: Tue Nov 08, 2011 4:29 pm    Post subject: Reply with quote

The final photo is to me a very unusual view of this type of creature, and quite interesting.

----------------------

Thanks for the insights into your slab-based workflow. Intuitively, it makes sense to me, although I have little stacking experience. It just seems correct that breaking a project down into sections would be more manageable and perhaps even more understandable and better optimized than trying to do everything in one grand unified stack.

Would it sometimes make sense to adjust the lighting between slabs? I was thinking here that one could provide lighting to see inside the creature's mouth that was only active while sharply imaging the interior. When mostly imaging the exterior, perhaps lighting that optimally but selectively would work.

Just some thoughts. Yet I can also imagine many problems trying to merge images from slabs that were illuminated quite differently. No doubt variable lighting within a stack would have to be carefully and systematically developed, to have much credible viability...
_________________
-Phil

"Diffraction never sleeps"
Back to top
View user's profile Send private message
elf



Joined: 18 Nov 2007
Posts: 1312

PostPosted: Tue Nov 08, 2011 9:03 pm    Post subject: Reply with quote

http://www.photomacrography.net/forum/viewtopic.php?t=14680
Back to top
View user's profile Send private message
ChrisR
Site Admin


Joined: 14 Mar 2009
Posts: 7092
Location: Near London, UK

PostPosted: Tue Nov 08, 2011 9:41 pm    Post subject: Reply with quote

I imagine that's a front end for batch scripts?

I can't get it to open - it shuts immediately.

The difficulties I find with substacks are more about processing the curves etc in each part and then reintegrating the subs together.
Back to top
View user's profile Send private message
elf



Joined: 18 Nov 2007
Posts: 1312

PostPosted: Tue Nov 08, 2011 10:19 pm    Post subject: Reply with quote

ChrisR wrote:
I imagine that's a front end for batch scripts?

I can't get it to open - it shuts immediately.

The difficulties I find with substacks are more about processing the curves etc in each part and then reintegrating the subs together.


Yes, it just creates the batch scripts.

Are you running it from the commandline? You also need to provide the initial parameters. I'd make a pretty front end for it, but I think Rik will probably do that inside of Zerene Stacker (hint, hint).
Back to top
View user's profile Send private message
Chris S.
Site Admin


Joined: 05 Apr 2009
Posts: 2757
Location: Ohio, USA

PostPosted: Tue Nov 08, 2011 10:28 pm    Post subject: Reply with quote

By the way, Chris (R.)—very nice image! Sorry for being so breathless about slabbing that I didn't comment on a superb photograph.

I wouldn't have the patience to routinely combine slabs using Photoshop layer masks. While I often use Photoshop layer masks, Zerene Stacker can stack slabs very well, so I let it do most of the heavy lifting.

I’ve been doing slabs one way or another for some time--it's been cumbersome but often worthwhile. As you demonstrated above, having a slab in which an element is delineated cleanly, but all else is a creamy blur, can help a lot when retouching. And slabs can reduce the interference of things in front of other things.

But a large step forward in my slabbing came a couple of weeks ago, when I asked Rik for advice about a stack I was finding difficult. It was, for me, a shallow stack of 259 frames. But by necessity, the individual frames were dark, fuzzy, and confusing, and my naked eye could barely make heads or tails of them. It was not surprising that the resulting stacks had major artifacts and were hard to retouch. Rik suggested that I make PMax slabs of about 20 images, overlapping each slab with the prior slab by ten frames. Then, he suggested, I should stack the resulting slabs using DMap. And he gave me the tips I needed to create an xml file that could be loaded into the Zerene Stacker batch dialog to do this automatically. I found it tedious to create the first xml file, but thereafter have just edited it for each new job.

Wow, has this approach worked. The difficult stack turned out shockingly better right out of the box. But also notable was how much easier other things were. I could quickly test various DMap settings—which unsurprisingly took an order of magnitude less time to run on 25 slabs than on 259 frames. Since I could test more quickly, I did more testing, and found better combinations—some of which I’ll combine using Photoshop layer masks (right now I’m mostly shooting—will leave the finer points of post-processing until the snow flies). And Zerene Stacker’s retouching tools became much easier to use for this very shallow DOF stack (100x). When a feature is spread across 20 frames, it can be hard to retouch in. When it is contained within a single slab, it’s easy.

So for every stack I’ve shot since (about three per day, the shallowest being 94 images), I’ve used this approach. It’s early to say, but I strongly suspect that slabbing may form a key portion of my workflow for anything other than very shallow, low-magnification, stacks. Definitions of “shallow” and “low-magnification” differ from one photomacrographer to another. The lowest magnification I’ve shot during this period was a 5x stack of 461 images. For this stack, I tried both the slabbing regime described here and a straight (no slabbing) stack. My sense was that Zerene Stacker could handle either approach very well. But the human element—me—found the slabbing approach much easier. Retouching was orders of magnitude easier with the slabs—no confusion about which elements are in front and which behind, and no endless searching for which frame to retouch in. Glorious! And with the slabs, I quickly tried several sets of DMap settings, and (as the tutorials say) found that certain settings worked better for various elements of the image. I’ll retouch the best elements together later. Without the slab approach, trying various DMap settings would have been more onerous, and I’d probably have skipped it.

Sorry for the long post. But I suspect this is paradigm-level issue in photomacrography. A frontier. Thanks, Chris R., for bringing it up.

Ed (“elf”) was generous in sharing his superb generator of xml files for Zerene Stacker batch jobs. His work deserves much more attention than it has gotten. I looked—hard—at it before generating my own xml file, but decided that I needed the learning experience of writing such an xml file myself. And I wasn’t sure, on initial inspection, if Ed’s approach would let me generate the overlapping DMap images that I wanted. (It was partly because of my confusion that I decided I needed the learning experience.) But my sense is that Ed’s application either can, or could with minor modifications, generate an xml file such as I’m using, much more easily than I'm doing it. Ed hasn’t gotten his deserved share of kudos for this contribution, and I think it's because most of us (myself included) are not sufficiently advanced to appreciate what he gave us.

And maybe Rik is cooking up enhancements to facilitate slabbing in Zerene Stacker? While I have no inside information, does anybody care to bet against me? Very Happy Anyway, I hope Rik is, because slabbing seems to take focus stacking to a new level.

ChrisR wrote:
I imagine that's a front end for batch scripts?

I can't get it to open - it shuts immediately.

The difficulties I find with substacks are more about processing the curves etc in each part and then reintegrating the subs together.

Chris, I can open Ed’s script easily—issue on your machine?

Yes, Ed’s script generates a very nice xml file that can be loaded via the Zerene Stacker batch dialog.

Processing “curves, etc.”—this seems Photoshop specific. Any reason not to stack your slabs with ZS DMap? If you prefer to do it in Photoshop, what’s the problem in processing curves? I use curves layers daily, so this seems like the easy bit. It’s the endless painting in with layers that would kill me.

Cheers,

--Chris (S.)

(Why are so many of us at PMN named "Chris"?)
Back to top
View user's profile Send private message Send e-mail
ChrisR
Site Admin


Joined: 14 Mar 2009
Posts: 7092
Location: Near London, UK

PostPosted: Wed Nov 09, 2011 7:03 am    Post subject: Reply with quote

Quote:
Chris, I can open Ed’s script easily—issue on your machine?

50% chance of that I expect.
And 50% chance I'm missing something out.
And 30% of something else...

Edit - redownloaded, I can get the help screen.
After lots of guesses about what it means, the program appears to run, but Zerene "doesn't recognise" the .xml file as a batch script.
Hand-holding would help here, with examples. I can't type a 40+ character path name in without an invisible typo, which means doing it all again, and I don't pre-know things like whether to include the < > symbols!
I'll get back to it later.



Quote:
Any reason not to stack your slabs with ZS DMap? It’s the endless painting in with layers that would kill me.

This is an example of where I was looking for guidance Smile so the prompting-post is working! I found that the stacker would take whichever slab it thought had more detail and made that layer "show" even it it happened to be behind something less detailed.
I've always felt that ZS needs a notion of foreground being more important that background, though with some (reverse perspective)lenses it would have to be told which was which.

Not enough overlap between my slabs perhaps?? I used a few frames, from memory of that beetle.

Yes the photshop editing part is easy, usually with cloning and other fakery!
Masking isn't so hard - you can use the auto-select tools, with a small edge-feather, which usually get it about right.

Also the background in this was messy so I replaced it completely
Back to top
View user's profile Send private message
elf



Joined: 18 Nov 2007
Posts: 1312

PostPosted: Wed Nov 09, 2011 10:17 am    Post subject: Reply with quote

Copy/Paste is your friend Smile When you have long paths to enter, navigate to them in Windows Explorer, then copy and paste them into the commandline. I usually create long commandlines by copying the template into notepad, then edit in notepad until it's correct. This makes it easy to retry until it's correct. <something> means replace everything including the <> with the text for 'something'.
Back to top
View user's profile Send private message
ChrisR
Site Admin


Joined: 14 Mar 2009
Posts: 7092
Location: Near London, UK

PostPosted: Wed Nov 09, 2011 10:23 am    Post subject: Reply with quote

Believe me I tried!
Ctrl-X-C-V don't work in the Command window. It's like going back to DOS !

AH but I just tried right-click, something I don't normally do, and that does work Smile
Back to top
View user's profile Send private message
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 17991
Location: Richland, Washington State, USA

PostPosted: Wed Nov 09, 2011 2:15 pm    Post subject: Reply with quote

Chris S. wrote:
Chris, I'm doing this a lot, though I call the sub-stacks "slabs." Not sure who coined this term--perhaps Rik? Anyway, I am finding the approach enormously useful, and suspect that in time, slabbing may be the rule, rather than the exception, in my workflow.

As far as I can trace it, the term "slab" as used here started in a post of mine on Apr 21, 2008. That was in the days of TuFuse and CombineZ, but the concepts were the same.

In terms of Zerene Stacker, the idea is to exploit the complementary tradeoffs of PMax and DMap. PMax is good at handling complicated structure with lots of overlaps, but it accumulates noise, alters color and contrast, and those problems get worse with deeper stacks. DMap is not so good with overlaps, but it doesn't have those other problems. So the concept is to use PMax on shallow sub-stacks (slabs) to produce partial outputs, then use DMap to combine the partial outputs. With proper choice of slab depth and DMap parameters, and maybe a bit of luck, almost all of the final pixels can be generated automatically, and where manual retouching is required that will go a lot faster.

In the long run, ZS will get some controls that make this easy to do. But for now the workflow has to be built up from simpler pieces.

A workflow that is tedious but simple to understand goes like this:
  • Launch ZS
  • Load all source images
  • Stack > Align All Frames
  • Select images 1-20
  • Stack Selected (PMax)
  • Select images 11-30
  • Stack Selected (PMax)
  • repeat previous two steps for all other slabs.
  • Select all output images
  • File > Save Output Image(s) to some scratch directory.
  • Exit ZS
  • Re-launch ZS
  • Load the images you just saved
  • Uncheck everything at Options > Preferences > Alignment
  • Stack > Align & Stack All (DMap).

Quote:
Based on some very useful guidance from Rik, I can do this pretty quickly, using an xml file that I call from Zerene Stacker's batch dialog. For different stacks, I edit the file.

The idea here is avoid having to interactively select, stack selected, select, stack selected, select, stack selected..., by creating a batch file to do those steps unattended.

The GUI is way too tedious to generate such a batch job. However, it's a good way to see what the batch job should look like, so you can build one programmatically. What I suggested to Chris S. was to open his project, select a few source files, open the batch dialog, do an Add Current Project, Add a task to Stack Selected (PMax), close the batch dialog, select some other files, open the batch dialog again, click to select the existing batch, and add another Stack Selected (PMax). Then Save Batch, close the dialog, and in the main menu do a Batch > Save Batch Script. What that produces is a ZereneBatch.xml file that you can read to figure out the pattern. There'll be a lot of repeated information in the OutputImageDispositionCode and Preferences sections. The critical variation is in parts like these:

Code:
        <Tasks length="2">
          <Task>
            <OutputImageDispositionCode ... />
            <Preferences>
            ...
            </Preferences>
            <TaskIndicatorCode value="4" />
            <SelectedInputIndices length="6">
              <SelectedInputIndex value="17" />
              <SelectedInputIndex value="18" />
              <SelectedInputIndex value="19" />
              <SelectedInputIndex value="20" />
              <SelectedInputIndex value="21" />
              <SelectedInputIndex value="22" />
            </SelectedInputIndices>
          </Task>
          <Task>
            <OutputImageDispositionCode ... />
            <Preferences>
            ...
            </Preferences>
            <TaskIndicatorCode value="4" />
            <SelectedInputIndices length="6">
              <SelectedInputIndex value="20" />
              <SelectedInputIndex value="21" />
              <SelectedInputIndex value="22" />
              <SelectedInputIndex value="23" />
              <SelectedInputIndex value="24" />
              <SelectedInputIndex value="25" />
            </SelectedInputIndices>
          </Task>
        </Tasks>

Hopefully this is sort of self-explanatory. As shown here there's a vector of tasks, length 2; each task is TaskIndicatorCode 4, which is Stack Selected (PMax); and each task operates on a vector of SelectedInputIndex values, in this case 6 files per task, with an overlap of 3. Index values start at 0, by the way.

With proper choice of output image disposition and naming template, this shortens the workflow to:
  • Edit batch script
  • Launch ZS
  • Load all source images
  • Load batch script
  • Run batch script
  • Exit ZS
  • Re-launch ZS
  • Load the images created by the previous run
  • Uncheck everything at Options > Preferences > Alignment
  • Stack > Align & Stack All (DMap).

I'm sorry this is still so complicated, but it'll be a while before the process gets simpler.

--Rik
Back to top
View user's profile Send private message Visit poster's website
AndrewC



Joined: 14 Feb 2008
Posts: 1436
Location: Belgium

PostPosted: Thu Nov 10, 2011 2:37 am    Post subject: Reply with quote

Curious about your choice of sub-stack overlaps, a 50% overlap seems a lot (1-20, 10-30, 20-40, etc). When I do this (and I think of it as "chunking" or "slice'n'dicing" not "slabbing") I tend to only go for about a 25% overlap (1-20, 15-35, 30-50, etc) but I've never done any actual comparisons. It what I picked the first time ! Also, I've never tried combining P and D-max that way, I use/used it as way of creating partial stacks for retouching so something else to try now the nights are getting longer.
_________________
rgds, Andrew

"Is that an accurate dictionary ? Charlie Eppes
Back to top
View user's profile Send private message Visit poster's website
Harold Gough



Joined: 09 Mar 2008
Posts: 5787
Location: Reading, Berkshire, England

PostPosted: Thu Nov 10, 2011 4:07 am    Post subject: Reply with quote

Slab away, if it gives us more such excellent final images.

Harold
_________________
My images are a medium for sharing some of my experiences: they are not me.
Back to top
View user's profile Send private message
descall



Joined: 01 Oct 2011
Posts: 156
Location: United Kingdom

PostPosted: Sat Jun 02, 2012 12:48 pm    Post subject: Reply with quote

I wonder if there has been any further development with simplifying the slab stacking approach in ZS?

Best wishes,
Des
Back to top
View user's profile Send private message
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 17991
Location: Richland, Washington State, USA

PostPosted: Sat Jun 02, 2012 1:06 pm    Post subject: Reply with quote

descall wrote:
I wonder if there has been any further development with simplifying the slab stacking approach in ZS?

In ZS proper, not yet. But photomacrography member elf has written a program to facilitate the process by creating a batch script that generates the substacks. See Automating Partial stacks in Zerene Stacker for more details.

--Rik
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Macro and Micro Technique and Technical Discussions All times are GMT - 7 Hours
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group