Batch processing multiple stacks located in the same folder using Zerene Stacker

[Quinn]

I am trying to figure out the best way to automate processing multiple stacks in Zerene Stacker when all the files are located in the same folder.

I can do this pretty simply in Helicon Focus 8 in their batch utility using the split stack option, but I think Zerene has an edge in output quality using the pyramid method, which seems to work best for the amount of detail I need, so I would prefer to find a workflow using Zerene.

My process:
I am doing a stack and rotate using a Stack Shot to create videos using each stack as a single frame. Shooting tethered all the photos end up in the same folder. I then stack each frame and use those to create a video.

Solutions I've thought about:
I have found folder splitting utilities that can split everything into separate folders by number of files but would much prefer a more streamlined workflow and more photo specific splitting options (ex: time based using exif data).

I'm thinking there is a way to write a batch script for Zerene to do this but haven't been able to figure out exactly how from reading all the documentation. I'm not looking to utilize slabbing (tried that with 0 overlap but it tries to align all and each stack needs a slightly different alignment due to the rotation so it's a no go). I would like to avoid splitting the files into separate folders using a separate program if possible. I do have some amateur level programming knowledge so if I have to learn more Java and write a batch splitting utility that plugs into Zerene myself I am open to doing that, just trying to see if there is already a solution that I have missed, or if anyone else has better ideas about it than I.

I'm hoping Rik has already thought of this and I just haven't found his solution. I'm also thinking Jay McClellen might have some insight as I am fascinated by the setup he has built to automate large parts of this process. I wanted to post this rather than ask them privately so that others can benefit from whatever knowledge comes of this inquiry too.

Thanks for your help!

[rjlittlefield]

I'm hoping Rik has already thought of this and I just haven't found his solution.

No such luck!

Yes, I have thought about this. But my standard recommendation is one that you've listed and said you'd like better.

To be more explicit...

My standard recommendation is to use any one of the many third-party "folder splitter" programs to rearrange the files into a collection of folders that each contain N files.

In that workflow, splitting of the files is only one operation, and setting up to process all the resulting folders is another one operation, independent of the number of folders, using the method outlined at https://www.zerenesystems.com/cms/stack ... processing .

When that method does not work, things start getting complicated. You mentioned time based, using Exif data. There are only a few folder splitting programs that can use EXIF data, with no consistency between Mac & Windows, and a tendency to become obsolete especially on Mac. I have not re-searched (as in "searched again") to see what is available today.

The most general approach to folder splitting is to use a purpose-built script, typically written in Linux bash or macOS zsh or Windows PowerShell. Lots of examples turn up in google search. But in my experience these tend to be fiddly and of limited application. Some people are comfortable writing or modifying such scripts, and then all I hear back is "worked fine, thanks". On the other hand, a lot of people find scripts too painful to be practical, if they can use them at all.

As for Zerene Stacker's built-in batch capability, there is currently no way to target anything other than (A) entire folders of source images or (B) pre-existing projects that already specify source images. To process groups of source images that are all packed into one folder, one approach would be to programmatically create a project for each set of images, then batch process all those projects. In that approach, each project would consist of a folder that contained a .zsj file listing all the source images but no preference settings. That way all the preference settings would be inherited from settings in the batch setup. https://www.zerenesystems.com/cms/stacker/docs/batchapi contains some relevant documentation, but nothing that exactly matches what you want to do.

You mentioned "write a batch splitting utility that plugs into Zerene". You cannot do that yourself, because Zerene Stacker has no mechanism for user-provided plugins. Whatever you wrote would have to run alongside Zerene Stacker, similar to how third-party slabbing utilities used to work. Those utilities worked fine when they worked at all, but they turned out to be unstable over time because of operating system evolution, especially on Mac. It was the goal of having a stable facility that eventually motivated me to add slabbing directly inside Zerene Stacker.

Something like that situation seems to be developing with folder splitting and/or other ways of processing datasets like you have.

So, I am now considering what a good practical facility should look like, if I were to add one into Zerene Stacker.

I would be interested to hear more from you and other people about what data organizations you have and what you'd like to do with them.

--Rik

[Bob-O-Rama]

If the objective is to keep from having two copies, use hard links under Windows ( or either hard or soft links under Linux and BSD / MacOS ) then you can keep your source materials in a folder, but create a folder tree for batch processing. The files are not duplicated, only the file system metadata. I wrote my own cheezy script to do this.

[mkbn]

If the objective is to keep from having two copies, use hard links under Windows ( or either hard or soft links under Linux and BSD / MacOS ) then you can keep your source materials in a folder, but create a folder tree for batch processing. The files are not duplicated, only the file system metadata. I wrote my own cheezy script to do this.

I like the idea of using symlinks, I use Linux primarily, but also have to use windows regularly!

I'm glad someone asked the question and to hear some elaboration on it.

One thing I've found myself doing is importing my files into Darktable, then turning on focus peaking / highlighting for the lightroom gallery tab. I then take a quick mental note of where a stack starts and ends, and tab over to the stacking program to load a stack.

I can then scroll through my gallery and fairly quickly see approximately when the in focus part starts at the beginning of a series of images, then ends, and makes it a little easier to filter out the frames where I was checking exposure/ modifying flash power / changing framing.

Something I thought of the other day was a script that would look through a folder and search for series of images, using focus highlighting similar to Darktable, to automatically find and select images with consecutive focus. It's the kind of thing a crafty person that loves programming could probably do to pre sort images. I am not that person, I can be crafty sometimes but programming I do not always enjoy when things get complex. But I also didn't thinking about general folder splitting programs based on times or soft links to help with sortation until this thread!

[pbraub]

So, I am now considering what a good practical facility should look like, if I were to add one into Zerene Stacker.

I would be interested to hear more from you and other people about what data organizations you have and what you'd like to do with them.

I really would love to see a batch facility in Zerene.
My wish would be to go from 0) all images in one folder 1) split by time delta 2) apply a number of methods/parameters of my choosing 3) save to folder.

Due to some constraints I mainly go for short macro photo walks when out with my kid for nap time. Shoot with in camera focus brackets (OM-1) and do editing in the late evening. So good having a quick way to get "good enough" results is my way to go. I don't have time to retouch and tweak for every single stack (at the moment).

So in basic something that runs from the files on sd card (or copied to a drive) as-is and give me a bunch of images ready to evaluate (and select in which to put more effort).

Processing time is not the major issue. There is always something to tidy up while stacking runs.

Peter