www.photomacrography.net :: View topic - Super-resolution from small sensors?
www.photomacrography.net Forum Index
An online community dedicated to the practices of photomacrography, close-up and macro photography, and photomicrography.
Photomacrography Front Page Amateurmicrography Front Page
Old Forums/Galleries
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
Super-resolution from small sensors?
Goto page 1, 2  Next
 
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Macro and Micro Technique and Technical Discussions
View previous topic :: View next topic  
Author Message
Lou Jost



Joined: 04 Sep 2015
Posts: 1513
Location: Ecuador

PostPosted: Tue Jan 26, 2016 5:47 pm    Post subject: Super-resolution from small sensors? Reply with quote

This is a continuation of an earlier post of mine about the possibility of getting higher-than-native resolution from a sensor by making multiple pictures of exactly the same scene and then combining them.

Here is my first test. I used a Nikon D90 (APS-C sensor, 4288 x 2848 pix) focused on a power line on a distant mountain. I used a Nikon 16-85mm lens at ISO 200, f11, 1/500 sec in full sun. I took eleven pictures of the same scene and converted them to uncompressed, unsharpened tiffs using a Photoshop actio. (By the way Chris, Photoshop CC did not open them all at once but processed them sequentially.)


Whole frame, shrunk for forum.

It was a windy day and after taking the images, I realized even the distant trees were moving too much to make useful targets. But the power tower on the right wasn't going anywhere so I concentrated on it, by cropping a vertical strip from the original image. (The tower on the left had clouds behind it that were moving, so they might have screwed up the alignment). Even so, I realize now that the alignment was made from the whole vertical strip, which included much vegetation, and so the alignment might be slightly off.

I upsampled all the images by a factor of three using the bilinear option in Photoshop. Then I put them all into a stack and had Photoshop align them. I then set their transparencies to 100/n where n is the layer number. Then flattened the image. Then cropped to just the power tower. Here is a comparison of the original tower image (from one of the eleven source shots) and the combined image. The original image is at left, the combined image is at right and outlined. The original image is resized to match the combined image. No sharpening or other adjustments were applied.


One of the original images at left. Combined image at right.

The difference is interesting but it may be mostly noise reduction. The noise reduction is striking; the original sky is very noisy in spite of the low ISO and bright sun, while the new sky is smooth. The most striking improvement is in the diagonal white power lines to the right of the tower; these are almost invisible in any individual shot but are visible in the combined shot.

However, I do not see any detail smaller than the original pixel size. I suspect a more sophisticated algorithm using pixel math may be needed for that. Theoretically I do think it is possible. Will do some more experiments and report back.

Edit: If this were film, I guess this is equivalent to using a lower ISO film, approx ISO 25.

Admin edit [rjl]: add link to earlier post
_________________
Lou Jost
www.ecomingafoundation.wordpress.com
www.loujost.com
Back to top
View user's profile Send private message Visit poster's website
Saul



Joined: 31 Jan 2011
Posts: 805
Location: Naperville, IL USA

PostPosted: Tue Jan 26, 2016 7:27 pm    Post subject: Reply with quote

Looks like simplified version of:

http://petapixel.com/2015/02/21/a-practical-guide-to-creating-superresolution-photos-with-photoshop/
Back to top
View user's profile Send private message Visit poster's website
Lou Jost



Joined: 04 Sep 2015
Posts: 1513
Location: Ecuador

PostPosted: Tue Jan 26, 2016 7:48 pm    Post subject: Reply with quote

Yes, that's the article I cited in my other post on the subject. I think my tests are more honest, since I use no sharpening at any stage.

What do you think of the results?
_________________
Lou Jost
www.ecomingafoundation.wordpress.com
www.loujost.com
Back to top
View user's profile Send private message Visit poster's website
Chris S.
Site Admin


Joined: 05 Apr 2009
Posts: 2730
Location: Ohio, USA

PostPosted: Tue Jan 26, 2016 9:38 pm    Post subject: Re: Super-resolution from small sensors? Reply with quote

Lou,

Lou Jost wrote:
I upsampled all the images by a factor of three using the bilinear option in Photoshop. . . .

The original image is resized to match the combined image. No sharpening or other adjustments were applied.

When you resized the original to match, did you do this by upsampling it by a factor of three, or upsizing it in the photoshop display? If the latter, it would be interesting to see the same comparison, but with the original upsampled in the same manner as the inputs to the test images. In fact, it would be interesting to see crops of all the input images. Do they differ visibly?

I get the impression that you photographed the distant mountaintop through a fair degree of atmospheric haze. Was this the case? If so, it might add a variable, as air is not uniform or still. Perhaps it would be useful to repeat the experiment on a close subject.

--Chris
Back to top
View user's profile Send private message Send e-mail
Lou Jost



Joined: 04 Sep 2015
Posts: 1513
Location: Ecuador

PostPosted: Wed Jan 27, 2016 5:15 am    Post subject: Reply with quote

Chris, I upscaled it in the display, because I did not want to add any processing. This way you see exactly the original pixels. But yes, it would be interesting to do it the way you suggested as well. Will do later today.

And yes you are right, yesterday was a bit hazy. We are in a severe drought and that's the time that people like to burn forests, so there is some light haze caused by smoke from distant Amazonian fires.

I am going to redo this using a precisely made resolution target that I am drawing on my computer screen. I can make crystal-clear detail with line thickness that is a known fraction of my camera's pixel dimensions.

But as I think about this, I am pretty sure that one needs to do a bit of pixel math to extract better sub-pixel information. That will require Visual Basic programming. Still, the easy Photoshop method is an interesting improvement, even if it just comes from noise reduction, and there are still some more parameters to play with.
_________________
Lou Jost
www.ecomingafoundation.wordpress.com
www.loujost.com
Back to top
View user's profile Send private message Visit poster's website
benjamind2014



Joined: 18 Oct 2014
Posts: 210

PostPosted: Wed Jan 27, 2016 1:20 pm    Post subject: Reply with quote

One thing I have noticed is that the larger the pixel pitch on the sensors, the less general noise.

However I have compared images to determine low light performance of a few small format sensors (through various sources) and found the Panasonic MN34120 produced great images in low light.

The larger format sensors are assumed to generally produce the better images, but the MN34120 really held its own in this regard.
Back to top
View user's profile Send private message
Lou Jost



Joined: 04 Sep 2015
Posts: 1513
Location: Ecuador

PostPosted: Mon Feb 01, 2016 8:18 pm    Post subject: Reply with quote

I've been researching this more, and finding lots of interesting results. Some people were skeptical that super-resolution could be obtained from multiple randomly-shifted low-res images. It turns out that this is an area of active research with well-documented results. A good search phrase for Google is {super resolution photo algorithm}.

The Olympus technique of increasing resolution by moving the sensor a known amount and taking multiple pictures can indeed be replaced (in theory) by moving any camera by a random amount, and applying sophisticated algorithms to reconstruct the sub-pixel image from the many randomly-displaced images. It is not necessary to know the displacement of each image: there are accurate algorithms to figure that out if one needs to. This method is not limited to a doubling of the resolution either. The resolution can be increased by factors of 8x or more, but more images are required. The difficulties go up very quickly as the extrapolated resolution increases, though. And apparently most of this stuff is done in black-and-white, so it may not be ready for color photography yet.

I have yet to find a clear implementation of any of these algorithms in software that one could download and try. The simple averaging of pixels is not enough to extract the full amount of information present in multiple images. There are some commercial programs that say they use sophisticated algorithms, but it is hard to tell if they really do.
_________________
Lou Jost
www.ecomingafoundation.wordpress.com
www.loujost.com
Back to top
View user's profile Send private message Visit poster's website
Chris S.
Site Admin


Joined: 05 Apr 2009
Posts: 2730
Location: Ohio, USA

PostPosted: Mon Feb 01, 2016 10:00 pm    Post subject: Reply with quote

Lou,
Lou Jost wrote:
And apparently most of this stuff is done in black-and-white, so it may not be ready for color photography yet.

Each color channel is essentially monotone, right? So perhaps simply run the process separately on the color channels?

Or perhaps convert from RGB to lab color, and run the process on the luminosity channel? I often make this conversion for sharpening, then convert back to RGB. My sense is that most of what our eyes see as resolution resides in the luminosity channel. And I suspect that the luminosity channel could be treated as black and white.

Your initial experiment was interesting, despite the simple method for combining the images. Any further results?

--Chris
Back to top
View user's profile Send private message Send e-mail
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 17874
Location: Richland, Washington State, USA

PostPosted: Mon Feb 01, 2016 11:45 pm    Post subject: Reply with quote

Lou Jost wrote:
Some people were skeptical that super-resolution could be obtained from multiple randomly-shifted low-res images.
...
It turns out that this is an area of active research with well-documented results.
...
I have yet to find a clear implementation of any of these algorithms in software that one could download and try.

So, I guess you can count me among the skeptics.

But I'm not really skeptical about "could be obtained". Theory and experiment are pretty clear that it can be.

The skepticism I have relates to how practical and reliable the approach is, particularly for focus stacking which was the original context.

I will be very interested to hear what your further investigations find.

--Rik
Back to top
View user's profile Send private message Visit poster's website
Lou Jost



Joined: 04 Sep 2015
Posts: 1513
Location: Ecuador

PostPosted: Tue Feb 02, 2016 5:45 am    Post subject: Reply with quote

Chris, that was my thought too. But maybe there are issues in getting the absolute levels of the channels to match--maybe relative lightness and darkness patterns are easier to reconstruct. I don't know.

Rik, I suppose the dearth of good consumer-ready programs suggests that you are right, but if it CAN be done, it WILL be done sooner or later.

I'll get back to my experiments as soon as I get sufficiently ahead on my "real" work....
_________________
Lou Jost
www.ecomingafoundation.wordpress.com
www.loujost.com
Back to top
View user's profile Send private message Visit poster's website
soldevilla



Joined: 16 Dec 2010
Posts: 457
Location: Barcelona, more or less

PostPosted: Tue Feb 02, 2016 12:35 pm    Post subject: Reply with quote

This sound like the astronomical photography! The way of use several images is our normal line of work, Sometime in planetary images, thousand...

Test Registax, AutoStakkert, Deep Sky Stacker and many more softwares. they are free, and work better that PS.
_________________
http://www.digitalphotomicro.com
Back to top
View user's profile Send private message
Lou Jost



Joined: 04 Sep 2015
Posts: 1513
Location: Ecuador

PostPosted: Tue Feb 02, 2016 2:17 pm    Post subject: Reply with quote

Soldevilla, it looks like some of these are mainly noise-reduction programs rather than resolution-increasing programs. Do you have any recommendations for resolution-increasing programs?

Edit to note that also some of these programs, like Deep Sky Stacker, are specifically designed to use stars for alignment and won't work without them, and Registax 6 has been crippled with weird artificial limitations compared to earlier versions, making it useless for super-resolution applications.
_________________
Lou Jost
www.ecomingafoundation.wordpress.com
www.loujost.com
Back to top
View user's profile Send private message Visit poster's website
Stevie



Joined: 27 Dec 2008
Posts: 87
Location: Belgium

PostPosted: Tue Feb 02, 2016 10:31 pm    Post subject: Reply with quote

Quote:
it looks like some of these are mainly noise-reduction programs rather than resolution-increasing programs.

Noise reduction in case of deepsky , yes since you use long exposure .
Another reason to do this , when shooting planets , is battling local seeing .
Most planet shooters also use a mono camera with a filterwheel in front of it .
So the shoot in LRGB to gain the full resollution of the sensor for each colorchannel .
Back to top
View user's profile Send private message Visit poster's website
Bushman.K



Joined: 26 Aug 2015
Posts: 162
Location: OR, USA

PostPosted: Tue Feb 02, 2016 10:49 pm    Post subject: Reply with quote

How about trying AutoStakkert! 2?
Back to top
View user's profile Send private message Visit poster's website
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 17874
Location: Richland, Washington State, USA

PostPosted: Tue Feb 02, 2016 11:46 pm    Post subject: Reply with quote

AutoStackkert looks like a good idea.

Here is a more specific URL for the super-resolution function: http://www.autostakkert.com/wp/enhance/

--Rik
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Macro and Micro Technique and Technical Discussions All times are GMT - 7 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group