How to increase the performance of the Zerene

[WojTek]

Hello everybody,
Does anybody know how to increase the performance of the Zerene?
Usually I increase the task-priority (e.g. set the priority of the javaw.exe under windows to the real- time or under unix the nice to -20) but maybe should I modify the garbage collection of the VM ?
BR, ADi

[rjlittlefield]

Quoting from one of the FAQs, https://zerenesystems.com/cms/stacker/d ... ess_faster :

Are there settings to make stacks process faster?

Yes, there are several. First, turn off functionality you don't care about. This applies especially to “Cache unaligned screen images” and “Cache aligned screen images”, found at Options > Preferences > Caching & Undo. If you seldom or never want to detect movement by playing a stack as if it were a filmstrip, then remove the checkmark on “Cache unaligned screen images”. If you don't plan on retouching from source or you're prepared for frame selection to be sluggish, then remove the checkmark on “Cache aligned screen images”. Un-checking Options > Preferences > Alignment > Rotation and Brightness can help too, if your setup does not allow rotation and has stable illumination. On Mac, try un-checking the External TIFF Reader at Options > Preferences > Preprocessing; if you still get correct results, you'll get them a little faster. Also, note that there's an option at Preferences > Multiprocessing, to “Overlap I/O with computation if possible”. If you're using trial, or a Personal or Student Edition license, then that option is turned off by default. You can turn it on by explicitly selecting it and answering “Yes” to the question about “start trial?” For other options and details, see the discussion at http://www.photomacrography.net/forum/v ... 699#141699.

Adjusting priority and garbage collection settings probably won't make much difference. The big improvements will come from turning off functionality that you don't need, and from selecting the option to "Overlap I/O and computation", combined with enough memory allocation to allow that. If you have multiple stacks to process, and enough memory to do it efficiently, then some improvement can be made by running two or more stacks at the same time. This is discussed in more detail at the link given above.

--Rik

[ChrisR]

Adi
Approximately what are your current per-image stacking time, and size?
I mean numbers something like 5 seconds, 20Mp, Align + Pmax?

[WojTek]

Hello Rik, hello Chris,
Do I really need the feature: “Overlap I/O with computation if possible” ?
( my system: OS 64 Bit, i7, RAM 64GB and SSD-disks )
BR, ADi

[rjlittlefield]

Do I really need the feature: “Overlap I/O with computation if possible” ?

"Need" is a strong word. You'll have to decide whether the speed improvement is enough to justify the cost of Prosumer versus Personal.

Here are some times as described at http://www.photomacrography.net/forum/v ... 943#142943 (Mar 08, 2014),

Here is a series of numbers from tests that I ran just now on my new 6-core machine. This is for PMax'ing 200 frames of 12 megapixels, all alignment options turned off, all other parameters at default except as noted.

1. Current full release T201402072140, 3:40 (min:sec)
2. Turned off caching, 3:30
3. New beta T201403071805_beta, 2:38
4. Turned on I/O overlap, 1:57
5. Throttled display to once per 15 seconds, 1:32 (min:sec)

As always, your mileage will vary. Many of the mods were motivated by this application and my new 6-core hyperthreaded machine, so they won't have as much effect in other situations.

In a different test just now, I ran Align & Stack All (PMax) with screen preview caching turned on, 14 frames at 18 megapixels. With I/O overlap not selected, the time was 44.95 seconds; with I/O overlap selected, it was 31.02 seconds, about 30% less time.

That said, with I/O overlap not selected but with screen preview caching turned off, the time was only 29.34 seconds, even faster. Then turning I/O overlap back on, still with screen preview caching turned off, the time dropped only a little more, to 28.52 seconds.

So, in this second set of tests the main effect of I/O overlap is to get that 30% improvement when screen caching is turned on. With screen caching turned off, I/O overlap made only a minor difference.

The effect of I/O overlap can vary significantly depending on file format (for example JPEG versus 16-bit TIFF) and on disk speeds.

I suggest to try putting a checkmark on Options > Preferences > Multiprocessing > "Overlap I/O with computation if possible", and see what effect it has on your workflow. (If you have previously tried that feature, so that the feature's 30-day free trial has expired, then send email to support@zerenesystems.com to get an extended trial.)

Be sure to set the memory allocation to around 200 megabytes per each 1 megapixel, so that the program has enough memory to hold the extra images on which to do I/O, while also computing with others.

You can confirm that enough memory has been provided by looking at the log info exposed by Options > View Console Log. When things are working properly, there will be one message that says "Overlapped I/O Permitted = true" (as opposed to "Overlapped I/O Permitted = false") ,and there will be multiple messages about "Starting preload" and "Finished preload". If you have screen preview image caching turned on, then there will also be multiple messages about "Asynchronously writing preview file" (as opposed to "Synchronously writing preview file".)

There is no string search capability built into Zerene Stacker, so the simplest way to look for these messages is to click once into the console log window, then control-A control-C (on Mac, cmd-A cmd-C) to copy the whole console log to clipboard, then paste into your favorite text editor so that you can use its search capability.

--Rik

[WojTek]

Hello Rik,
OK, I will test it.

BTW, my biggest problem is the performance of the DPP4:
http://www.photomacrography.net/forum/v ... 814#223814

BR, ADi

[WojTek]

Hello Rik,
I have just tested the performance of the Zerene with my workflow (usually the second step) on my old desktop with 32GB RAM.
For this test I have taken the 15 tiff-files (output of the slab with PMax) and processed with "Align & Stack All (PMax)"
Zerene has been restarted for each test. Duration times come from the log-files.


Caching-settings:


Memory-settings:


IO-settings ( test 2 and 4 )


Results:


How to explain this behavior?


BR, ADi

[rjlittlefield]

It looks to me like turning on I/O overlap results in around 45% improvement, while increasing the priority makes less than 9% improvement.

This seems completely consistent with numbers that I posted earlier, except that you're seeing even more improvement from I/O overlap than I did.

Probably that aspect is because you're testing with TIFF images, which are typically slower to read than JPEGs.

Is there some other part of your results that requires explanation?

BTW, there's a new beta, just posted out a couple of hours ago, that should fix the problem of fields too small to show the full numbers in Options > Preferences > Memory Usage.

--Rik

[WojTek]

Hello Rik,
I have just activated the pro-features, which I have never used before.
So, I will have fun for the next 30 days :-)
BTW, I have expected a little bit bigger difference between the normal and real time.
BR, ADi

[rjlittlefield]

Setting a high priority such as realtime just tells the computer "if two different tasks are both ready to run, do this one first".

But typically when a computer is processing images, especially when running a benchmark for time, it does not have much else to do. It's just going to process those images as fast as it can.

To be honest, what surprises me is that changing the priority made even as much difference as your numbers show. Your computer must have had something else to do.

--Rik

[WojTek]

Hello Rik,

”when running a benchmark for time”

Ok, in this case maybe not but usually the computer is being used.

BTW, over 90% of the time of my workflow takes the Canon DPP4 for the conversion of the CR2 into the TIFF 16 bit

BR, ADi