Image comparison of ZS and PS CS5 on deep high mag stacks

A forum to ask questions, post setups, and generally discuss anything having to do with photomacrography and photomicroscopy.

Moderators: rjlittlefield, ChrisR, Chris S., Pau

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Image comparison of ZS and PS CS5 on deep high mag stacks

Post by rjlittlefield »

I was recently prompted by an offline email conversation to revisit the tradeoffs of Photoshop CS5 as a stacking tool. Normally I wouldn't run Photoshop on a deep high magnification stack, but for completeness I thought I should retest to see how it's coming along. So, I'm doing a series of runs using some of my standard test stacks.

Here's the first one:

Image

The stack is 167 frames at 3072x2048, shot through a 10X microscope objective.

The times shown here are wall-clock minutes on Windows 7 64-bit, 4 GB RAM, 2.4 GHz Q6600 Intel Core 2 Quad cpu. Photoshop auto-align options were collage (allow rotation, scale, and translation), no vignette removal, no geometric distortion; auto-blend options were stack images, seamless tones and colors.

I'll post out more results as they come in, though I have to admit that this first one does not encourage spending a lot of time on this.

--Rik

elf
Posts: 1416
Joined: Sun Nov 18, 2007 12:10 pm

Post by elf »

I'm impressed that you even were able to stack that many images in PS. My system fails around 35 :)

Chris S.
Site Admin
Posts: 4042
Joined: Sun Apr 05, 2009 9:55 pm
Location: Ohio, USA

Post by Chris S. »

Photoshop seems to have messed up not only the hard stuff I'd have expected it to struggle with, also some things that would would be easy, such as bristles on a plain background.

ChrisR
Site Admin
Posts: 8668
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

If it was I who reprompted the subject, I was remembering, or misremembering:
1) A guy on a website did a test of some of Zerene, HF , PS and CZM(?) for landscapes. He did manage to make a bit of a mess with Zerene.
Rik's reworking of his files produced a result which was declared to be not as good as the PS output, and needing a lot more reworking.
Being landscapes there weren't many files of course.

2) Somewhere, sometime, in the forum, I thought Rik said that PS might be better for a particular type of subject. I believe it was more landscape than close-up. I can't find it - searching on the likely terms produces rather a lot of hits.

If I find links to either, I'll put them in!

johan
Posts: 1005
Joined: Tue Sep 06, 2011 7:39 am
Contact:

Post by johan »

By coincidence, came across this comparison. Author is a poster here
My extreme-macro.co.uk site, a learning site. Your comments and input there would be gratefully appreciated.

stevekale
Posts: 172
Joined: Wed May 11, 2011 2:40 pm
Location: London, UK

Post by stevekale »

How he can rate the CS5 version of the last image a the best is beyond me. Did he even look closely at the results?

ChrisR
Site Admin
Posts: 8668
Joined: Sat Mar 14, 2009 3:58 am
Location: Near London, UK

Post by ChrisR »

Yes Johan, that's the one.
I think some of those pictures are bad enough that they're misleading. The only way I can imagine Pmax producing some of the results shown would be to use poor Jpeg compression then oversharpen, and stack the results! I don't see an explanation of what parameters were used or altered to try to improve things.

The Dmap efforts too. If you're used to a Ford with an auto gearbox then go try a Ferrari, you might well complain that it revs rather a lot and doesn't get to a very high speed. Unless you read about the stick-shift gear change, and find you're still in first gear.
I suppose if you need a one-stick solution and don't worry about your details, then maybe PS is for you.

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

elf wrote:I'm impressed that you even were able to stack that many images in PS. My system fails around 35 :)
I don't know what's up with that. My system doesn't seem to have a hard limit; it just starts doing lots of I/O. I think Task Manager showed around 40 GB read/write on this one.
Chris S. wrote:Photoshop seems to have messed up not only the hard stuff I'd have expected it to struggle with, also some things that would would be easy, such as bristles on a plain background.
Yes, that is interesting. I didn't expect it either, but having seen it, I think I know what's happening. Photoshop seems to be using a feature-points method of alignment, which is typical for panorama registration. That works pretty well also for low magnification stacks where every frame shows basically the same recognizable stuff, albeit focused differently. But in these high mag stacks, most of the frame is full of featureless blur. Often the only thing that's in focus are the shafts of bristles. Those doesn't have recognizable features on two axes, only one. As a result, the feature-finder passes by those areas and instead locks onto strong features that are located elsewhere in the image. If the feature points that are found happen to be badly located, for example clustered in an area far away from the bristle, then slight errors in registering the feature points can cause large errors in registering the bristles. In other words, I suspect that Photoshop didn't even consider the bristles because they didn't meet its criteria for being important.
johan wrote:By coincidence, came across this comparison. Author is a poster here
Ah yes, that comparison. I'm familiar with that work. Actually I know a fair amount about it because the author and I conversed for a while by email, and at my request he even sent me some of the stacks to look at myself.
stevekale wrote:How he can rate the CS5 version of the last image a the best is beyond me. Did he even look closely at the results?
Well, different people have different criteria. I think he looked very carefully at the results. Certainly I did, and I agreed that the CS5 results were definitely better than ZS.

However, they were all pretty bad, by my standards. Photoshop was doing its usual trick of missing detail, while ZS PMax was doing its usual tricks of increasing noise and giving echos and halos everywhere there was motion or hard-edged OOF blurs. Initially he had only tried PMax. I suggested trying DMap with certain parameters, but the results were awful.

That was when I asked if he could send me some stacks to look at.

When I received the DVD, the basic problem was obvious. Rather than normal sequential stacks, he had shot unordered "heaps" in which the focus bounced around pretty much randomly. For example in his 5-frame "stack", the usual front-to-back ordering was 4, 1, 2, 5, 3 in his shooting order. In other words he shot pretty close, slightly farther back, then all the way back, then all the way forward, then almost all the way back again.

If you know how DMap works, you realize that random input practically guarantees awful results. But if you don't, then it's easy to reach other conclusions, as that author did.
ChrisR wrote:I suppose if you need a one-stick solution and don't worry about your details, then maybe PS is for you.
That's a vivid analogy, and pretty accurate, I think.

The great thing about PS is that it doesn't have any controls to learn. Once you know which buttons to click, just click 'em and the result pops out.

The bad thing about PS is that it doesn't have any controls in any case, so if what it does is not what you want, you're pushed all the way back to manual retouching. That's still not such a bad thing if you're dealing with a handful of images, even though it becomes wildly impractical with some kinds of work we do here at photomacrography.net.
2) Somewhere, sometime, in the forum, I thought Rik said that PS might be better for a particular type of subject.
Probably, though I can't recall for sure either. What I've seen recently is that Photoshop does well on small stacks, especially if you're more sensitive to tonal quality than to loss of detail. In addition Photoshop is easier to drive than the specialty stackers, and of course its stacking comes at no additional cost and is better integrated with the rest of Photoshop than the other tools are. I think the issue is not so much the particular type of subject, but the overall situation.
ChrisR wrote:If it was I who reprompted the subject, I was remembering, or misremembering:
1) A guy on a website did a test of some of Zerene, HF , PS and CZM(?) for landscapes. He did manage to make a bit of a mess with Zerene.
Rik's reworking of his files produced a result which was declared to be not as good as the PS output, and needing a lot more reworking.
I think the test I saw was just ZS, HF, and PS, not CZ. The Zerene outputs did look pretty bad. Most of the reason is explained above: PMax did its usual handling of subject motion, hard-edged blurs, and tonal changes, and the random ordering kept DMap from doing anything useful at all.

The key thing I did to "rework" his files was to just sort them into sequential order by focus point and run them through DMap using recommended settings. I also turned off Brightness correction to prevent any changes to pixel values, which seemed to be really important to him. The complaint then became that while the results were much improved, they weren't really any better than Photoshop's. No disagreement there, especially if the criteria don't count missing detail as being a significant flaw. Different strokes for different folks, and all that.

I will admit to being puzzled by the sweeping conclusions and strident tone of the comparison, but that's a matter for another day and a different venue. Probably best to think of it as an editorial opinion piece and move on.

--Rik

Chris S.
Site Admin
Posts: 4042
Joined: Sun Apr 05, 2009 9:55 pm
Location: Ohio, USA

Post by Chris S. »

rjlittlefield wrote:I will admit to being puzzled by the sweeping conclusions and strident tone of the comparison, but that's a matter for another day and a different venue. Probably best to think of it as an editorial opinion piece and move on.
I was bothered by a number of elements in that fellow's tirade. Among them, that he referred to Zerene Stacker as a $300 program. Sure, it's $290 for professional use, but I'll bet the great majority of users are paying $40 (students) or $90 (non-pros). And Zerene Stacker is by no means crippled at these lower price points. A few of the "faster workflow" capabilities are available only under the pro license, but all of the "quality output" features are there even under the cheapest license.

His test seems to start with bad photography and a demand that some magic wand will correct it for him. But most stacking photomacrographers want to step through boundaries that even good photography has heretofore not able to cross. So the guy just doesn't get it. From his findings, all I can learn is that if someone wishes to make precisely the same mistakes he does, his findings might apply. Sheesh.

Photoshop stacking vs. Zerene Stacker stacking is like Yugo vs. Ferrari. But even if the two programs were equal, the support that comes with Zerene Stacker would make it an easy buy. On behalf of Fortune 500 companies, I've been involved in multi-million dollar software purchases. Did my clients ever get the kind of support Rik gives? No way, not ever. Not long ago, I made a suggestion to Rik for an "N'th frame" stacking feature, as a help in determining optimal movement increments. I've come to expect incredible support from Rik, but was still blown away when, that very day, he sent me a link to a beta version of Zerene Stacker he had altered to add this capability. That very day. And it worked great. Now, a few weeks later, that feature, among others, has been rolled out without fanfare, in an updated release at no cost to licensed Zerene Stacker users. Anybody think this would happen with Photoshop?
rjlittlefield wrote:Probably best to think of it as an editorial opinion piece and move on.
Too kind, Rik. As The Bard said: It is a tale told by an idiot, full of sound and fury, signifying nothing.

--Chris

yeatzee
Posts: 292
Joined: Sat Jan 29, 2011 3:13 am

Post by yeatzee »

Chris S. wrote: I was bothered by a number of elements in that fellow's tirade. Among them, that he referred to Zerene Stacker as a $300 program. Sure, it's $290 for professional use, but I'll bet the great majority of users are paying $40 (students) or $90 (non-pros).

--Chris
From another perspective, CS5 costs WAY more than zerene especially the student version. For someone who doesn't own photoshop and uses GIMP (a free program) for advanced editing, the cost is GREATLY in favor with zerene.

And thats completely sidestepping the fact that zerene is an absolutely amazing tool I couldn't live without.....

DQE
Posts: 1653
Joined: Tue Jul 08, 2008 1:33 pm
Location: near Portland, Maine, USA

Post by DQE »

I personally believe that most people of the caliber that would be significantly interested in a demanding specialty like focus stacking, would quickly judge that an irrational review is not worth paying much attention to. One has to constantly sort out disinformative reviews from informative ones in today's Internet-based society.

Yet newcomers might not be able to tell the difference, but even newbies could save themselves from such a mistake if they simply look around the various macro forums and note that the major stacking software in use doesn't usually use Photoshop except for manually constructed, low-image-count stacks (typically created from a handful of hand-held field macro photos, not CS5's automated stacking). Yet I notice that Lord V seems to use Zerene for many or perhaps most of his hand-held field stacks.

Perhaps people with experience in creating high-quality stacks (studio or hand-held) should reply more frequently, especially at other forums where disinformative reviews and opinions are sometimes promulgated without much rebuttal. I'd hate to see too many people begin to use an inadequate solution for stacking software without understanding what they are missing.

Unfortunately, on other forums such "discussions" too often degenerate into flame wars, etc. I suspect this is why many informed people don't always comment specifically and critically when strongly expressed but disinformative reviews and comments are posted (on other forums). Yet if informed opinion is not presented effectively and regularly, uninformed opinion may sway much opinion, at least for a while. Just consider current US politics! This concern is worrisome to me.

Fortunately, photomacrography.net is such a very valuable refuge, source of knowledge, and a haven for macro and micro photographic enthusiasts, due in no small part to sustained efforts by Rik and others. Thanks for everyone's ongoing efforts!

Perhaps a group of us should informally get together and draft a jointly signed reply at other macro forum(s) when a significant but disinformative review is posted? This is how journals handle such issues, with respect to published articles or reviews, etc. Just a thought.

In an ideal world, author(s) of inadequate reviews comparing CS5 and Zerene, etc, would read this thread and (peacefully) help everyone reach a better-informed state of knowledge. Such a goal does require that everyone be reasonably knowledgeable, in basic agreement as to how to credibly compare software and image quality, or at least educable and well-intentioned.

In any event, I found Rik's comparison testing to be very informative and very credible, but not surprising (which is a good thing in this context!).

I hope my two cents/two Euros are helpful.

Disclosure: I own both Zerene and Photoshop CS5.
-Phil

"Diffraction never sleeps"

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Resuming the original direction of this thread, here is the second stack. By the way, the stacks I'm testing here are the same ones that I used earlier for An image comparison of Zerene Stacker and Helicon Focus. I've reprocessed to make sure that I'm using the same input images for Photoshop CS5 and Zerene Stacker, and to collect timings.

Image

This one was a smaller stack, 80 frames at 3072x2048, so it didn't take as long to process.

Photoshop CS5 looks to have done pretty well with the registration on this one. There's some slight doubling of edges at a few places, but they don't pop out immediately.

The most obvious issue to my eye is lost detail in low contrast regions ("stacking mush"). Interestingly CS5 did better then Helicon in that regard.

Perhaps the most interesting aspect, certainly the one that surprises me the most, is that in this case it's Photoshop that changed the colors. The background should be uniform gray, but Photoshop changed it into a light cyan at one place and added an overall cyan cast over the rest. I'm not sure why that happened. This stack suffers from uneven exposure due to variable flash brightness, and I'm speculating that something about the red subject caused Photoshop's "seamless tones and colors" option to push the background in the opposite direction.

One more test and then I think this exercise will have consumed enough time. Tomorrow, maybe...

--Rik

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

OK, one more to finish out the set. Dandelion seeds, shot at 6.9X (full frame shown HERE).

Image

This was a fairly big stack, 121 frames at 4752x3168, so it took quite a while to process. The difference in processing times seems significant -- 6 hours and 52 minutes for PS CS5 versus 25 minutes for Zerene Stacker.

I'm not sure what causes the bands and blobs of blur in Photoshop's result. The band could be cropped off, though at the loss of some interesting detail I deliberately framed in when I shot it. Or maybe it could be handled by changing to one of the other alignment strategies, but at 6+ hours per experiment, I'm not inclined to investigate. In any case, the underside of that right-hand seed would be more troublesome to recover, since it's spread over about 40 frames. I think if I was working with Photoshop I'd just decide to give up and not worry about that part.

The DMap result here was done with Brightness adjustment turned on, which is default. In retrospect that seemed like a mistake, because this stack was shot with continuous illumination so the actual exposures should be uniform with no correction required.

I couldn't resist running that experiment while writing up this post. The conjecture turned out to be wrong -- in fact there was enough unevenness to produce visible contouring in the smooth background. So, bad decision, brightness adjustment really was the right thing to have done. I ran the test at 50% presizing and stack every 2nd frame; processing time for the test was 5 minutes and 15 seconds. Interesting experiment.

--Rik

rjlittlefield
Site Admin
Posts: 23561
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Admin note: Some readers of this thread will notice that it's now a lot shorter than it used to be. The old continuation was a distracting shift of style and content, so we have split off that portion to form a separate thread, which is now locked. Further substantive discussion is welcomed in this thread, following our forum's usual conventions.

--Rik

DQE
Posts: 1653
Joined: Tue Jul 08, 2008 1:33 pm
Location: near Portland, Maine, USA

Post by DQE »

Rik,

Thanks for reorganizing and simplifying this instructive and interesting thread.

I've been trying to think about how one might make benchmark comparisons of software packages for hand stacked field macro photos. I quickly became confused as I tried to reason my way through this.

Even if one restricted one's efforts to using stacks of people with much experience and demonstrated competence in creating hand-held stacks, and persuaded them to submit their stacks to another person to run them through a few stacking programs, there are still many hard-to-resolve issues, IMO.

The simplest item to deal with would be to require that all photos be arranged in focus order, of course, which is simple for the small number of slices used for hand stacks.

Is there is a practical way to attack this complex and ambiguous subject? Is it appropriate for a forum effort?

The current status quo is for each person to make his/her own tests, but I wonder if we would benefit by collectively analyzing and commenting on multiple photographers' "representative" benchmark stacks, and building on that experience. Yet it's likely that the ability to obtain useful images for a field stack vary considerably from one person to another.

One of the many complications is the inevitable and important role that hand retouching would play and this is at least somewhat subjective.

At least hand stacks wouldn't be nearly as impractical to pass around the internet using one of the file sharing services. Even so, we'd might have to avoid using raw or tiff files due to bandwidth and file size limits.

Just some weekend musings, hoping we can build on your very informative studio benchmark efforts...
-------------

Crescat scientia; vita excolatur (Latin)

Let knowledge grow from more to more; and so be human life enriched. [translated to English]

-- motto of the U of Chicago
-Phil

"Diffraction never sleeps"

Post Reply Previous topicNext topic