Zerene: batch problem

A forum to ask questions, post setups, and generally discuss anything having to do with photomacrography and photomicroscopy.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

iconoclastica
Posts: 486
Joined: Sat Jun 25, 2016 12:34 pm
Location: Wageningen, Gelderland

Zerene: batch problem

Post by iconoclastica »

Last night I had a rather long batch running. Not surprising, I ran out of disk space just before two o'clock.
When today I prepared to restart with the remaining tasks, I noticed that 15 project files had been reset to 0 bytes, so lost their content. In 5 cases there was a .zsj.backup file with (aproximately) the same time stamp as the project file. I found one where the .zsj.bkp file had zero bytes but the .zsj file survived.

The output was written to another disk. The same disk as where the Windows' temp filder resides. The full temp-folder in itself is a source of problems.

I think that the batch processor would better wait for user intervention in case of space problems. Now it seems to continue with the next batch in the list.

Version 1.04 Build T2021-05-28-1930
--- felix filicis ---

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Re: Zerene: batch problem

Post by rjlittlefield »

Speaking as the author of Zerene Stacker, I agree completely. The handling of errors in batch mode is not good.

In general, the handling for errors in ZS is basically "call human and keep making progress if possible".

Unfortunately, for some classes of errors ZS makes the mistake of acting like progress is possible when really it is not.
iconoclastica wrote:
Sat Nov 26, 2022 3:57 am
... would better wait for user intervention in case of space problems.
That sounds like the sort of precise diagnosis and recovery that we might hope for in a disk backup program: wait for more space to be made available, then re-do whatever operation failed due to the shortage, and continue seamlessly from there.

I'm pretty sure that level of elegance is more than I can manage. There are at least dozens of places inside ZS where I/O might fail due to space issues, and no existing code structure that would facilitate retry and continue.

But probably I can do something like adding an option to immediately pause operations after "call human". I've added that to the infamous list of requested features.

--Rik

iconoclastica
Posts: 486
Joined: Sat Jun 25, 2016 12:34 pm
Location: Wageningen, Gelderland

Re: Zerene: batch problem

Post by iconoclastica »

It's not a complaint, just an observation :)
Every user is a tester. On the whole, the batch executor did an excellent job, doing overnight what would have taken me over a week of boring work to do manually.
--- felix filicis ---

Macro_Cosmos
Posts: 1511
Joined: Mon Jan 15, 2018 9:23 pm
Contact:

Re: Zerene: batch problem

Post by Macro_Cosmos »

I have run into this too. I plan to fix it by having C raid with another disk, in the future. I love the batch function, it is so useful for diatom stacks.

iconoclastica
Posts: 486
Joined: Sat Jun 25, 2016 12:34 pm
Location: Wageningen, Gelderland

Re: Zerene: batch problem

Post by iconoclastica »

It can be easily fixed by running a script against the (compound) batch xml that makes an extra copy of every project file involved.
--- felix filicis ---

Post Reply Previous topicNext topic