Zerene Stacker now has dust & hot-pixel removal

A forum to ask questions, post setups, and generally discuss anything having to do with photomacrography and photomicroscopy.

Moderators: Chris S., Pau, rjlittlefield, ChrisR

Luisifer
Posts: 84
Joined: Wed Sep 05, 2018 1:40 am
Location: Czech Republic
Contact:

Post by Luisifer »

Zigzag curve is applied only for popping up the dust spots (it is not recorded in action).

rjlittlefield
Site Admin
Posts: 20760
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Luisifer wrote:For those who are not familiar with "popping up" of the dust on photo, i use zigzag curve. It helps near to perfectly.
mawyatt wrote:Isn't the "zigzagged" caused by a hot pixel that is fixed in the sensor?
I understand Luisifer's "zigzag" to be the shape of an unusual non-monotonic levels adjustment, using what is called "Curves" in the English version of Photoshop.

The idea is to enhance local image contrast at almost all brightness levels, at the cost of brightness inversion over a total one-half the range.

--Rik

mawyatt
Posts: 2465
Joined: Thu Aug 22, 2013 6:54 pm
Location: Clearwater

Post by mawyatt »

Thanks Rik, was thinking this zigzag as result of hot pixel, not a post processing function.

Best,
Research is like a treasure hunt, you don't know where to look or what you'll find!
~Mike

rjlittlefield
Site Admin
Posts: 20760
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Luisifer wrote: i have action(s) in PS that retouche dust.... It would be ok but PS doesn't accelerate it so it runs very slow (for example one frame per two minutes). So in case of thousands frames it is neverending batch...
I understand Luisifer's approach to be selecting how to fix each dust spot separately, essentially by "clone this content to there".

That is the approach that I used several years ago, when I used Lightroom to fix the dust spots in this thread's first example, the weathered plastic table. In that workflow, Lightroom applied all the edits pretty quickly, a matter of some seconds per frame.

The approach of handling each spot individually is some sort of "gold standard", because it allows the photographer's understanding of the subject to decide the best replacement for each spot. But it is always labor intensive, because even if the dust is constant, the best replacement will change from one stack to the next.

An intermediate approach that may be better in some circumstances is to use Photoshop's "content aware fill" feature. In that case you would use a mask to form a selection of which pixels to fill, for the entire image, and then use a single Edit > Content-Aware Fill to do them all at one time. Wrapping the select-and-fill into an action, then using Automate > Batch... to apply it to the entire stack, sounds like it would be a lot faster than the action-of-many-edits approach that I think Luisifer is using.

It's a different question, and one I cannot answer, whether content-aware fill would give a good result in any particular case.

In general, I consider large dust spots to be a problem with no good recovery, because any recovery inevitably requires "making up" pixel values to replace the ones that were never captured. The only really good solution, as Beatsy points out, is to clean the sensor and keep it that way. That philosophy is one big reason why I never put a mask function into Zerene Stacker until now.

But hot pixels are a different matter. They're much smaller than dust, and they don't move around, and you cannot always get rid of them. What finally pushed me over the edge was that I really Really wanted to use a long exposure with continuous light, to investigate a technical question of continuous illumination versus flash, and the hot pixels with continuous illumination were driving me nuts.

So I finally bit the bullet, put in a mask, and then discovered to my pleasure that it worked better for dust than I expected it to. It's not perfect (see earlier paragraph!), but I expect it's good enough to keep quite a few people happy.

--Rik

Luisifer
Posts: 84
Joined: Wed Sep 05, 2018 1:40 am
Location: Czech Republic
Contact:

Post by Luisifer »

Ok, i tried some comparsion.

Only one stack (without substacks, wihout other retouching in ZS).

Without retouching in preprocess:
https://12in.cz/temp/zs_fly/wo_rm.jpg
Image

With retouching in preprocess over ZS:
https://12in.cz/temp/zs_fly/zs_rm.jpg
Image

With retouching in preprocess over PS:
https://12in.cz/temp/zs_fly/ps_rm.jpg
Image

rjlittlefield
Site Admin
Posts: 20760
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Luisifer wrote:Ok, i tried some comparsion.
This is weird.

I downloaded the images to be layers in Photoshop, and did a flash-to-compare.

The middle stack has completely different alignment of the foreground antennna(?) versus the eye and other bristles behind it -- both rotated and shifted.

That is not at all an effect that I would expect infilling dust spots to produce.

Does the stack go through some stage where nothing is in focus, or much of what is in focus is covered by the dust mask?

--Rik

Edited to add: If your rig prevents rotation mechanically, as most high mag setups do, then you should remove the checkmark on Rotate. Since this is high mag (10X or above?), typically you should remove the checkmark on Scale also. This is for the reasons discussed at https://www.photomacrography.net/forum/ ... 9878#79878 , that with high mag and shallow DOF, those "corrections" are more likely to introduce alignment problems than to solve them.
Last edited by rjlittlefield on Mon Mar 30, 2020 6:50 pm, edited 1 time in total.

TheLostVertex
Posts: 315
Joined: Thu Sep 22, 2011 9:55 am
Location: Florida

Post by TheLostVertex »

I have a new question: Does the infill operation only have to happen once if you are doing stereo/rocking images, or will the infill operation take place for each new image for the stereo/rocking set?

TheLostVertex
Posts: 315
Joined: Thu Sep 22, 2011 9:55 am
Location: Florida

Post by TheLostVertex »

I just ran a deep stack as a test, and the alignment is slightly different between the one with with and without the dust mask. It was a very deep stack(600 images) with a slight misalignment of the stacking axis, as seen by the streaks.

Image


Image

rjlittlefield
Site Admin
Posts: 20760
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

TheLostVertex wrote:I have a new question: Does the infill operation only have to happen once if you are doing stereo/rocking images, or will the infill operation take place for each new image for the stereo/rocking set?
At present it will happen every time a source frame is used. That is, there's no caching of the infilled images, just like there's no caching of aligned images except for low quality screen preview. The whole issue of caching transformed versions, both in memory and on disk, is one that I need to address in this new era of large RAM and fast I/O like SSD's and RAID's.
I just ran a deep stack as a test, and the alignment is slightly different between the one with with and without the dust mask.
With deep stacks, I am not surprised by slight changes in the alignment.

In fact the alignment with dust removed is arguably better than it is with dust present, because the alignment process then is relieved of the tension between keeping the subject lined up and keeping the dust lined up.

The crazy thing about Luisifer's example is that there's so much difference between dust and no dust using ZS dust removal, while there's only a little difference between dust and no dust using PS dust removal.

BTW, I notice in your example that you're getting a significant amount of rotation, judging from the angle of the edge streaks themselves, at lower right. If your rig prevents rotation mechanically, as most high mag setups do, then you should remove the checkmark on Rotate. If this is also high mag, say 10X or above, then typically you should remove the checkmark on Scale also. This is for the reasons discussed at https://www.photomacrography.net/forum/ ... 9878#79878 , that with high mag and shallow DOF, those "corrections" are more likely to introduce alignment problems than to solve them.

--Rik

TheLostVertex
Posts: 315
Joined: Thu Sep 22, 2011 9:55 am
Location: Florida

Post by TheLostVertex »

rjlittlefield wrote: BTW, I notice in your example that you're getting a significant amount of rotation, judging from the angle of the edge streaks themselves, at lower right. If your rig prevents rotation mechanically, as most high mag setups do, then you should remove the checkmark on Rotate. If this is also high mag, say 10X or above, then typically you should remove the checkmark on Scale also. This is for the reasons discussed at https://www.photomacrography.net/forum/ ... 9878#79878 , that with high mag and shallow DOF, those "corrections" are more likely to introduce alignment problems than to solve them.
Thanks for pointing this out. I read that thread some time ago and normally do turn off scale for high mag stacks, but never did for rotation. My rig has no rotation in it so of course it make sense to turn that off :oops:. In that image I had all alignment options on, as I wasnt thinking about it at the time. (old stack that I remembered had easily visible dust, IIRC it was with a mitty 50x SL)

-Steve

Luisifer
Posts: 84
Joined: Wed Sep 05, 2018 1:40 am
Location: Czech Republic
Contact:

Post by Luisifer »

rjlittlefield wrote:Since this is high mag (10X or above?), typically you should remove the checkmark on Scale also. This is for the reasons discussed at https://www.photomacrography.net/forum/ ... 9878#79878 , that with high mag and shallow DOF, those "corrections" are more likely to introduce alignment problems than to solve them.
Thanks for your advice. Yes, rotation was enabled. I will turn it off.
(20x and scale alignment i had turned off already)

Luisifer
Posts: 84
Joined: Wed Sep 05, 2018 1:40 am
Location: Czech Republic
Contact:

Post by Luisifer »

Btw. one more thing. Maybe better than for larger spots to add black borders to mask for preventing strips. :-)

rjlittlefield
Site Admin
Posts: 20760
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

Luisifer wrote:Maybe better than for larger spots to add black borders to mask for preventing strips.
Translating this into my terms...

It is important that the mask covers the entire dust spot, including margins that may be darkened only slightly.

Unlike some other products (Helicon Focus comes to mind), Zerene Stacker does not expand the mask at all. It replaces pixels that correspond to black in the mask, and it does not replace pixels that correspond to white in the mask.

So, if the mask does not cover some edge of the dust spot, then that edge will form a trail of its own.

It is the user's responsibility, as the creator of the mask, to make sure that it covers enough area.

--Rik

Luisifer
Posts: 84
Joined: Wed Sep 05, 2018 1:40 am
Location: Czech Republic
Contact:

Post by Luisifer »

rjlittlefield wrote:It is important that the mask covers the entire dust spot
Maybe it will be better to add visual example what i was thinking by borders in mask:

Image

rjlittlefield
Site Admin
Posts: 20760
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

OK, now I'm not sure what you are thinking.

I have thought about using the mask as a way of preventing edge streaks, by using a different dataflow. For that application, the filling would not get done before transformation. Instead, the mask would get transformed along with the image, so that the narrow black borders turn into large black regions, covering the areas that would otherwise be filled with edge streaks. Then in-filling those regions would result in smooth variations instead of edge streaks, and the stacking process would tend to replace those with real data from other frames.

Is that what you're thinking, or something else?

If something else, I need further explanation.

--Rik

Post Reply