www.photomacrography.net :: View topic - Zerene Stacker now has dust & hot-pixel removal
www.photomacrography.net Forum Index
An online community dedicated to the practices of photomacrography, close-up and macro photography, and photomicrography.
Photomacrography Front Page Amateurmicrography Front Page
Old Forums/Galleries
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
Zerene Stacker now has dust & hot-pixel removal
Goto page 1, 2, 3, 4  Next
 
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Macro and Micro Technique and Technical Discussions
View previous topic :: View next topic  
Author Message
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 20491
Location: Richland, Washington State, USA

PostPosted: Wed Mar 25, 2020 8:48 pm    Post subject: Zerene Stacker now has dust & hot-pixel removal Reply with quote

The latest beta release of Zerene Stacker provides a new feature, “Use dust & hot pixels mask”, for addressing the problem of trails caused by dust and hot pixels.

To use it, you construct a mask image, same size and orientation as your source images, that is pure white everywhere, except black where your source images have dust spots or hot pixels. Then at Options > Preferences > Preprocessing, you check the box for “Use dust and hot pixels mask” and browse to find the file. The mask will then be applied in subsequent stacking operations.

Very briefly, what the feature does is to replace blobs of bad pixel values with a smoothly varying average of surrounding good pixel values. This alone provides a good start at hiding the defects, and then the normal stacking process further improves the result by tending to replace the smoothly varying averaged values with unmodified good values from other frames that happen to have different alignments. The result is that instead of a trail, what typically gets produced is an image that is clean except for one spot of lost or degraded detail where the defect lay in the best focused image.

Here is an example. These crops are from a stack that I shot several years ago, at a time when I think I must have had my camera open during a dust storm.

Left: without the mask; Right: with the mask.



Left: source frame with dust locations marked in yellow; Right: final mask, black on white



Here is another example, this time dealing with hot pixels as a result of long exposure with continuous light.

Left: without the mask; Right: with the mask



Left: cropped mask matching the images shown above; Right: mask for whole frame (dots expanded to be easily visible)




Note that Zerene Stacker makes no attempt to build an appropriate mask. It just applies whatever mask you give to it.

Building the mask is something that you’ll do outside Zerene Stacker, using tools in your favorite image editor such as Photoshop or GIMP.

For the first example, I pulled a source frame into Photoshop, heavily sharpened it to enhance the dust, added a new layer that would become the mask, and used a small brush to manually “dot over” each spot of dust. For brushing I used a contrasting color (bright yellow) to make it simple to see where I had painted. Then to make the final mask, I used Photoshop’s selection and filling tools to turn the yellow spots into black-on-white.

Initially I tried to construct the dust mask automatically from the source images by a combination of sharpening and levels adjustment to bring out the dust spots. But that approach failed because the dust spots had such wide range of appearance, on varied background, that I was not able to get a clean distinction between dust and no dust. It was much faster and better to just dot over the dust spots by hand.

For the second example, I found that it was tedious and unnecessary to make the mask by hand from a source frame of the stack. Instead, I simply made a very long time exposure of a black scene to make the hot pixels easily distinguishable, then used levels-adjustment in Photoshop to isolate hot pixels, selected those, expanded the selection by a couple of pixels to be sure of covering up the “crosses” that are often produced by JPEG compression, and filled the selection to make the mask. Unlike dust, hot pixels don’t move around or accumulate rapidly, so this mask will be usable for a long time in the future with that same camera.

There are a few caveats…
  • At this time the process is computationally expensive, roughly doubling the time to run a basic PMax stack in my limited testing to date. Probably I can speed that up with more clever coding, but the value at current cost is high enough that I’m comfortable releasing it as a production feature.

  • The mask is applied only during stacking. During retouching, source images will be presented in their original form.

  • Error checking is minimal at this time. If the mask is the wrong size, or the wrong orientation, or the mask file can’t be read, or anything else unexpected happens, then either the function won’t work at all, or it’ll do something weird, or you’ll end up with one of the dreaded “Uh-oh, something went wrong” pop-ups.

  • It's best if the mask image is TIFF, even when you're stacking JPEG's. This avoid possible issues caused by JPEG compression around the edges of the black areas of the mask.

  • The dust & hot pixels mask will be a Pro-only feature, so to trial it with a Student or Personal Edition license, you'll have to click "Yes, start trial" when the notification popup appears.

  • If you haven't received an announcement for this beta even though you're already on the Zerene Stacker beta channel, please don't worry about that. I've bundled a bunch of stuff into the current beta, and I decided to post here before pushing out an in-app announcement because of recent questions about hot pixels.

Feedback solicited, of course.

--Rik
Back to top
View user's profile Send private message Visit poster's website
ChrisR
Site Admin


Joined: 14 Mar 2009
Posts: 8496
Location: Near London, UK

PostPosted: Thu Mar 26, 2020 2:01 am    Post subject: Reply with quote

Wonderful, I look forward to trying it!

Well done Rik.
_________________
Chris R
Back to top
View user's profile Send private message
Lou Jost



Joined: 04 Sep 2015
Posts: 4118
Location: Ecuador

PostPosted: Thu Mar 26, 2020 5:50 am    Post subject: Reply with quote

Thanks very much Rik, this will be useful to everybody!
_________________
Lou Jost
www.ecomingafoundation.wordpress.com
www.loujost.com
Back to top
View user's profile Send private message Visit poster's website
Peter M. Macdonald



Joined: 20 Jan 2009
Posts: 175
Location: Berwickshire, Scotland

PostPosted: Thu Mar 26, 2020 6:00 am    Post subject: Reply with quote

Yet another great addition to the tools which you have already put at our disposal. Well done and thank you Rik.

I am aware that Canon has a dust delete feature in many of its cameras. You take an out of focus image, at a large aperture, of something fairly bright, such as a blank area of sky, and then feed the resultant capture into DPPro as a dust file. The software can then apply this information to any other file when asked. Nobody else seems to use this dust information from Canon, so I am not sure if it is something which they have always hidden the details of, or whether the likes of Adobe have considered that it is too much trouble to utilise when other camera manufacturers seem not to offer the same feature.
Back to top
View user's profile Send private message
Beatsy



Joined: 05 Jul 2013
Posts: 1551
Location: Malvern, UK

PostPosted: Thu Mar 26, 2020 7:14 am    Post subject: Reply with quote

Another handy addition to an already indispensable piece of software. Thanks for the continuing updates Rik.

Could I also suggest... Smile

Back to top
View user's profile Send private message
Smokedaddy



Joined: 07 Oct 2006
Posts: 1590
Location: Phoenix, Arizona

PostPosted: Thu Mar 26, 2020 8:35 am    Post subject: Reply with quote

Excellent, thanks. No more spot healing brush in PS. Cool
Back to top
View user's profile Send private message Visit poster's website
Macro_Cosmos



Joined: 15 Jan 2018
Posts: 543
Location: Sydney

PostPosted: Thu Mar 26, 2020 10:55 am    Post subject: Reply with quote

Beatsy wrote:
Another handy addition to an already indispensable piece of software. Thanks for the continuing updates Rik.

Could I also suggest... Smile



In Australia, alcohol of all types... even Vodka in many places are completely sold out.

We all know why Crying or Very sad
_________________
Personal Flickr page: https://www.flickr.com/photos/133023063@N04/
Blog still under construction
Back to top
View user's profile Send private message
abpho



Joined: 17 Aug 2011
Posts: 1479
Location: Earth

PostPosted: Thu Mar 26, 2020 3:44 pm    Post subject: Reply with quote

Nice Rik. Thanks.
_________________
I'm in Canada! Isn't that weird?
Back to top
View user's profile Send private message
Chris S.
Site Admin


Joined: 05 Apr 2009
Posts: 3455
Location: Ohio, USA

PostPosted: Thu Mar 26, 2020 10:37 pm    Post subject: Reply with quote

Rik,

Delightful to see this added to Zerene Stacker! I've downloaded the beta, and am looking forward to trying out dust and hot-pixel removal.

Peter M. Macdonald wrote:
I am aware that Canon has a dust delete feature in many of its cameras. You take an out of focus image, at a large aperture, of something fairly bright, such as a blank area of sky, and then feed the resultant capture into DPPro as a dust file. The software can then apply this information to any other file when asked. Nobody else seems to use this dust information from Canon, so I am not sure if it is something which they have always hidden the details of, or whether the likes of Adobe have considered that it is too much trouble to utilise when other camera manufacturers seem not to offer the same feature.

FYI, Peter, Nikon has a similar feature. However, I've not gotten it to work on the two or three models I've attempted it on. When I try to take the out-of-focus image with my macro lens, the camera seems to think for a while, then eventually tell me to try with a different lens. At that point I've stopped messing with it.

--Chris S.
Back to top
View user's profile Send private message Send e-mail
Scarodactyl



Joined: 14 Apr 2018
Posts: 291

PostPosted: Fri Mar 27, 2020 2:17 am    Post subject: Reply with quote

A very handy feature that I've been getting quite a bit of use from. I had my local camera shop clean my sensor, but they somehow ended up smearing something on it instead. Very vexing on some longer exposures.
Back to top
View user's profile Send private message
RDolz



Joined: 28 Aug 2017
Posts: 83
Location: Valencia (Spain)

PostPosted: Fri Mar 27, 2020 3:49 am    Post subject: Reply with quote

Rik: good job and very helpful. Applause

In my case my main problem is the dead and hot pixels. I work with the Fujifilm X-T1, a camera that does not have the ability to map faulty pixels.
So I had an action in photoshop that eliminates the hundreds of pixels that show defects in long exposures. (action that significantly increases post-processing tasks)

Therefore, having a tool, such as Zerene, that integrates these functions as well as folder processing (Batch) is very interesting because it allows you to obtain the best result without the need to apply other processes.

Only if it serves as a help, in my case if you make a long exposure of many seconds with the lens covered, I can see the defective pixels, but others appear when in addition to making long exposures, I also capture thousands of photos (when I do stacking + stitching), that is, when the sensor has been active for a long time and therefore the temperature increases. I have to comment that normally my exposure times are around 1 / 15s, ... they are not very long exposures, but many damaged pixels appear.

Best
_________________
Ramón Dolz
Back to top
View user's profile Send private message
JW



Joined: 23 Feb 2011
Posts: 160
Location: New Haven, CT, USA

PostPosted: Fri Mar 27, 2020 2:18 pm    Post subject: Reply with quote

Another great feature, but, my friendly admins now keep me from downloading software. Rik, is there a way to side-load the updates?
_________________
It's not what you look at that matters, it's what you see - Henry David Thoreau
Back to top
View user's profile Send private message
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 20491
Location: Richland, Washington State, USA

PostPosted: Fri Mar 27, 2020 7:57 pm    Post subject: Reply with quote

JW wrote:
my friendly admins now keep me from downloading software. Rik, is there a way to side-load the updates?

Windows or Mac?

If Mac, then there's really no installation to be done. Just do the download and unpack the .zip on any machine that can, the write the whole application package tree to a thumbdrive and execute it from there.

On Windows, you can try something similar, except in that case it's probably cleanest to install into some ordinary folder like C:\ZereneStacker, instead of letting the installer put it into Program Files by default. Then copy the folder to thumbdrive and move that.

Unfortunately I think that both of these routes are likely to be blocked also, assuming that your admins are good at their jobs.

Other than these approaches, I have no idea how to try working around administrative limits.

--Rik
Back to top
View user's profile Send private message Visit poster's website
pierre



Joined: 04 Jan 2010
Posts: 254
Location: France, Var, Toulon

PostPosted: Sat Mar 28, 2020 2:21 am    Post subject: Zerene Stacker now has dust & hot-pixel removal Reply with quote

Many thanks for your care and hardwork Rik !
This is definitively a nice add.

Will try it soon.

Stay safe.
_________________
Regards

Pierre
Back to top
View user's profile Send private message
RDolz



Joined: 28 Aug 2017
Posts: 83
Location: Valencia (Spain)

PostPosted: Sat Mar 28, 2020 10:46 am    Post subject: Reply with quote

Hello, I downloaded the Beta version to test the new feature, “Use dust & hot pixels mask”. I have tested it to eliminate the hot pixels and, (as could not be otherwise), it works perfectly!!

It took me a while to make the mask, but in the end, with photoshop, it was easy.

The main problem is detecting pixels that are defective in a single image in the stack. But if we use the result of a stacking it is easy to detect the "little snakes" that each faulty pixel generates.

Here I show you the hot pixel mask for my X-T1




and the difference of a portion of an image before and after applying the new feature.



This image has a lot of noise, because it is a black background and I have increased the levels to make the hot pixels look good.

Best
_________________
Ramón Dolz
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Macro and Micro Technique and Technical Discussions All times are GMT - 7 Hours
Goto page 1, 2, 3, 4  Next
Page 1 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group