Zerene Stacker - input appreciated with handheld stacking

[DQE]

My recently deceased PC was a 4-core, overclocked gaming PC tower with some extra I/O ports, including Firewire 800 and two eSATA (external SATA) ports. I'm planning to replace it, probably with a "workstation" model by HP or perhaps Apple's Mac Pro within a few weeks.

I found that my now dead PC was enjoyably fast when reading my RAW still photos by virtue of using a combination of Firewire for my CF card reader (Firewire 800 with the best-rated CF card reader bumped the I/O rate from 20 Mbytes per second via USB2 up to (I think - memory and my age don't mix) 60 Mbytes per second. Similarly, eSATA runs several times faster than USB2 for backups to external hard drives. I figure if I keep my working digital photo sets on a SSD, it should be faster to get Photoshop started with the day's images. I do hope to expand into more stacking this year, health permitting.

For my personal needs, I therefore believe that speeding up the relatively slow step of reading in say several hundred Canon 5DII raw files per batch, faster I/O would be at least somewhat helpful. Also, consider that the very newest (and most expensive) CF cards run quite a bit faster to transfer files to one's PC than typical previous generation CF cards. Otherwise, this step alone may consume up to tens of minutes, considering that I do this via Photoshop CS5 Bridge and that it simultaneously makes a copy of each raw image to a dedicated external hard drive. I've noticed that CS5 runs all 4 cores full tilt during this process, unlike some other things in Photoshop. I've got to reread the recent reviews of the downloading efficiency of the leading photo software packages. I think some say Bibble is faster, but I am very used to Photoshop's Adobe Camera Raw (ACR) and Bridge quirks, especially with respect to color balance.

I figure that if I follow the current trends and select a 64-bit OS such as Windows 7, and acquire at least 12-16GB of RAM, I should seldom have to use the page or swap files for Photoshop. Everything should mostly be in RAM once it's loaded, and it should be able to stay there.

Yet I still fret about the issue of reading in up to a large number of slices, and the associated time needed to do the I/O. As usual, the answer to this worry must be "it depends". (grins)

I was a little surprised to read in Rik's post that Zerene doesn't benefit much from using an SSD since I would think that it would need considerable elapsed time to read in what may be a large number of slices. It must be the case that the time needed to process the slices in the CPU is long enough so that time for I/O doesn't slow down the CPU processing much. In other words, Zerene must be CPU-bound rather than I/O-bound under normal circumstances. Just a guess - I have no data or other direct evidence. Would this still hold for a CPU at the higher end of specs, or an overclocked multi-core CPU?

My understanding is that if the CPUs are kept usefully busy during a process and not waiting on I/O, there isn't much one can do other than get a faster CPU. Someone once said that the definition of a supercomputer is that it's I/O-bound!

In any event, I've read reports of PC boot times measuring as little as about 10 seconds when one has an SSD, but again I don't have my own data to cite. This wouldn't help too much since I don't need to reboot too often as I use my PC in retirement. Usually, I just turn it on in the morning and turn it off at bedtime.
------------------------

One of the many stressful parts of selecting a new PC for a wide range of photo processing work is that the only major vendor retail PCs that accommodate more than 2 hard drives are so-called workstations. For various reasons these PCs cost MUCH more than the same CPU in smaller tower case. To make matters worse, if one were to buy a workstation with a large array of factory-installed RAM and 2TB hard drives, one could easily spend $6-8K. Video cards for workstations can easily cost $1-2K US dollars. This is of course absurd. If I go the workstation route, I would buy a bare-bones workstation and add RAM and hard drives from Newegg or somewhere. This partial DIY approach would save over $2K, more if one adds an independently acquired consumer video card (such as the Nvidia 580 or the ATI 5870.

Another stressful issue is PC vendor selection re repairs. HP, Dell, and others implement PC repairs using local contract repair people (temporary employees of an agency, not employees of the PC company). They also use minimally-English-speaking telephone support, which drives me bonkers. AFAIK, only Apple now provides high-quality post-sale service, and they also have a local Apple store for in-person repairs and assistance in my town.

Thus, one of my lines of thought is to buy a basic Apple Mac Pro workstation, add independently acquired (and much less costly) RAM, hard drives, an SSD, eSATA ports, and as a fallback position plan to install Windows 7 if I can't easily convert and adjust to the Mac OS and Apple's restrictive policies. I really don't like being restricted by signing up for Apple's "gated community" approach to computing, though. Yet their high quality service and support is very compelling for such an expensive and repair-prone item as a PC workstation.

If only HP had stayed the course of providing a high level of service and support...

I would greatly appreciate comments on configuring and acquiring a new PC, at least somewhat optimized for various Photoshop tasks as well as stacking.

[SONYNUT]

maxed...ended up slower than 1 at a time anyway...

I've run 3 at once so I guess im good...

Hhmm... Perhaps you're running 32-bit Windows? In that case, it maxes at 1.6 GB per process, no matter how much physical memory you have. If it's 64-bit Windows, then I'd definitely be interested to know what Task Manager says for memory consumption when all three are running.

--Rik

[rjlittlefield]

I was a little surprised to read in Rik's post that Zerene doesn't benefit much from using an SSD since I would think that it would need considerable elapsed time to read in what may be a large number of slices. It must be the case that the time needed to process the slices in the CPU is long enough so that time for I/O doesn't slow down the CPU processing much. In other words, Zerene must be CPU-bound rather than I/O-bound under normal circumstances. Just a guess - I have no data or other direct evidence. Would this still hold for a CPU at the higher end of specs, or an overclocked multi-core CPU?

Yes, CPU-bound in all the cases I've seen. But I'm sure "it depends".

To illustrate the case in front of me right now, I'm running Align & Stack All (PMax) with 10 megapixel images from 30 MB TIFF files (8-bit RGB), on a Core 2 quad core Q6600 at 2.40 GHz. Stacking 21 frames takes 125 seconds wall clock. Total I/O is under 900 MB, or roughly 7.5 MB/second average. These ratios are determined by what all is being done, of course. Turn off all the alignment options and it runs faster, about 106 seconds. Reprocess an already aligned stack (as in generating stereo/rocking) and it's down to 88 seconds. Moving up to 16-bit RGB would close to double the I/O requirement. Put in a faster and overclocked processor, the compute would drop more, though I don't have a number for that.

I suppose performance can get dominated by I/O under some conditions; I just don't know where that boundary would be.

I'd be interested to hear what sort of performance ratios are seen by folks with bigger/faster machines than I have. What's the highest MB/sec number you've seen while stacking?

I would greatly appreciate comments on configuring and acquiring a new PC, at least somewhat optimized for various Photoshop tasks as well as stacking.

Might be some useful information and/or contacts here:
http://www.photomacrography.net/forum/v ... php?t=9435

--Rik

[rjlittlefield]

maxed...ended up slower than 1 at a time anyway...

I've run 3 at once so I guess im good...

Hhmm... Perhaps you're running 32-bit Windows? In that case, it maxes at 1.6 GB per process, no matter how much physical memory you have. If it's 64-bit Windows, then I'd definitely be interested to know what Task Manager says for memory consumption when all three are running.

OK, thanks for the info. This sounds like what happens when memory gets oversubscribed and processes start forcing each other to page. If you can manually crank down the allocation enough that two or more processes stay in memory, they should complete faster by running together than one at a time. On the other hand, the first one will take longer to get done. Whether that's better or worse depends on what you care about.

--Rik