## Combining images to get better noise or resolution

A forum to ask questions, post setups, and generally discuss anything having to do with photomacrography and photomicroscopy.

Moderators: Pau, rjlittlefield, ChrisR, Chris S.

Lou Jost
Posts: 4859
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:
"No, thats not the reason. All calibration frames (darks, flats, bias) are needed to extract the pure signal from the object. I.e. darks are necessary to subtract the dark current"

Markus, that is what I just said. The reason we need dark frames is because we have to subtract the mean noise from the image in order to get the signal. Averaging multiple images is not enough.

etalon
Posts: 67
Joined: Tue Jan 05, 2016 5:26 am
Location: Germany
Contact:
Lou, sorry. My last post was postet at the same time as yours...
But here is my problem: whatever the noise distribution, its mean value is some fixed nonzero value X, and the mean signal is some other fixed value Y, so no matter how many frames you average, you end up with X+Y and you have to subtract the X. Right?
I don´t know whether I understand it right, but I´ll try to explain it in an other way (sorry for my bad English. It´s difficult for me to explain what I mean in a foreign language )

The detection limit of your camera is the fluctuation range of the noise. If there is any signal with an amount smaller than the fluctuation range of noise, it will not be possible to differentiate the signal from the noise. If you get a better noise statistic, i.e through stacking, the fluctuation range of noise decrease, and the signal, which was formerly hidden in the noise, would be detectable. You reached a better detection limit with your camera. So, there is nothing to subtract. Noise is always present, but in different strength...

Cheers Markus
Last edited by etalon on Tue Feb 09, 2016 11:01 am, edited 1 time in total.

etalon
Posts: 67
Joined: Tue Jan 05, 2016 5:26 am
Location: Germany
Contact:
because we have to subtract the mean noise from the image
Lou, you can´t subtract noise like a constant, because it is in every image different. In one, you will than subtract too much, and in an other frame you will subtract too less at the same pixel...

Lou Jost
Posts: 4859
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:
Perhaps we may be misunderstanding each other because of language, but your English is nearly perfect.

Of course the noise varies from frame to frame. But my point was that the average noise (averaged across frames) is a constant (assuming everything else in the setup is unchanged--temperature, etc). So if you average many frames of an image, the result approaches the mean value of the signal plus the mean value of the noise. You need to subtract the mean value of the noise from the averaged image to obtain the pure signal.

This is what you yourself said above regarding dark current so I don't think we really disagree.

etalon
Posts: 67
Joined: Tue Jan 05, 2016 5:26 am
Location: Germany
Contact:
Well, I think we still disagree a little bit.

Lets do a mind experiment:
We assume a mean value of 100e- per pixel well in an array. Because of random noise, the amount of e- in a single observed pixel well varied between 90e- and 110e-. If you now say, the mean noise over the array is 10e-, and subtract 10e- from each pixel, the pixel well with 90e- have now 80e-, and the pixel well with 110e- have now 100e-. As you can see, you lower the mean value by 10e-, but the random variation between the pixels are still present. Thats the reason, why you can´t eliminate noise. You can make it only less bad...

Lou Jost
Posts: 4859
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:
No, I have always been emphasizing that we are working with the mean values, not any particular image. In your example, we would first work with many dark images. We would find the average value in each pixel well is 10e , in the absence of a signal. Then we make many exposures of our signal, and we find the average is close to 100e. So we subtract 10e from 100e to get the true mean value of the signal, 90e.

Edited to emphasize that we are working with many dark images in this thought experiment, not just one.

rjlittlefield
Site Admin
Posts: 21392
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:
Lou Jost wrote:But here is my problem: whatever the noise distribution, its mean value is some fixed nonzero value X, and the mean signal is some other fixed value Y, so no matter how many frames you average, you end up with X+Y and you have to subtract the X. Right?
The math is right, but the terminology is wrong. Additive signal contaminants that are called "noise" are almost always defined so that their mean is zero. If a contaminant has non-zero mean, then it is typically modeled as two other components, with the mean being called something like "bias" or "offset", and "noise" being the rest.

--Rik

Lou Jost
Posts: 4859
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:
I can see why that might be useful practical terminology, and if that's what the field uses, I have to just get used to it.

But it is quite odd from a statistical and physical standpoint, and would seem to get in the way of understanding sometimes. For example, many noise-producing processes are Poisson processes. In any Poisson process the variance always EQUALS the mean. In other words these two things, the "random component" and the "offset" are actually so tightly linked that they can't be altered independently, and they are both determined by exactly the same statistical parameter. It is misleading to think of them as separate.

rjlittlefield
Site Admin
Posts: 21392
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:
Lou Jost wrote:It is misleading to think of them as separate.
Sure. Some level of imprecision is the price we pay for having models that are simple enough to be helpful. I am reminded of the comment by one of my mentors, many years ago: When a statistician says "assume that", what he really means is "pretend that". Nothing is ever really Gaussian. That particular mentor was a world-class statistician, who basically made his living by figuring out how to make models that were accurate enough to be useful, while still being simple enough to yield insight.

If you want other things to worry about, then consider that the pixel values in an ordinary JPEG or TIFF file are encoded as a power function, courtesy the "gamma" parameter in the color profile. As a result, noise components that may be purely additive in the raw sensor data become not purely additive in the pixel values that get processed later.

This is another reason why all sorts of noise is more visible in dark regions. To take an example, if white is 10,000 captured photons and read noise is 10e, then the total noise is +-100 for the Poisson component and +-10 for the read noise. Those are independent, so rms applies and the total noise for white is about +-100.5. If dark gray is 100 photons, then the total noise for that is +-10 for the Poisson and +-10 for the read noise, a combined total of +-14.1 . At this point there's still a lot more noise for the white pixels, although the signal-to-noise ratio is better. But then gamma gets applied, at which point that 10000+-100.5 turns into 255+-1.3, while 100+-14.1 turns into 25.5+-1.8 (assuming gamma=2). All of a sudden, because of gamma, those dark pixels not only have a lot worse signal to noise ratio, but they also have more noise in the absolute numbers.

I confess, I've never found it very helpful to worry about these subtleties. Of course there may be important insights that I've missed and never known about, but for practical purposes it has always seemed to work very well to just keep straight the basic stuff: what's constant, what's random, and what happens when you add together multiple independent samples.

By the way, gamma can also be your savior, though it's basically an accident. If gamma=2, and read noise is insignificant, then the stdev of noise in pixel values turns out to be essentially independent of brightness, despite being Poisson in its origin. Isn't that a bizarre quirk?

--Rik

Lou Jost
Posts: 4859
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:
I like to worry about these kinds of things. Sometimes (though not always) sorting out the math clears things up more than it confuses. That's the main thing I do in my scientific life (sorting out and fixing the math used to conceptualize fundamental concepts in biology like "diversity") so it is hard to turn it off.

Your comment on other things to worry about is very interesting. Does that mean we should take the logarithm of the intensity before subtracting off the "offset"?

Your comment about gamma=2 is also interesting. I've found something similar in my field-- measures of biodiversity that are based on the squares of detection probabilities have interesting special properties.

rjlittlefield
Site Admin
Posts: 21392
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:
Lou Jost wrote:Does that mean we should take the logarithm of the intensity before subtracting off the "offset"?
I suspect you can figure out the answer yourself, once the question becomes clear.

With that in mind...

What "offset" are we talking about? More generally, what math model are you using, and what do you want to do with it?

By the way, we really need to split this thread. It makes no sense for the current discussion to continue as an extension of "Hello! + First post, questions". I'll do the split to Technical Discussions after you've had a chance to see this note.

--Rik

Lou Jost
Posts: 4859
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:
Got it, go ahead and move it.

etalon
Posts: 67
Joined: Tue Jan 05, 2016 5:26 am
Location: Germany
Contact:
Lou Jost wrote:No, I have always been emphasizing that we are working with the mean values, not any particular image. In your example, we would first work with many dark images. We would find the average value in each pixel well is 10e , in the absence of a signal. Then we make many exposures of our signal, and we find the average is close to 100e. So we subtract 10e from 100e to get the true mean value of the signal, 90e.

Edited to emphasize that we are working with many dark images in this thought experiment, not just one.
Yes, I understand that. But I don´t think, that´s the right way to calculate it. Noise is the variance of stored e- between all pixel wells of the sensor, when illuminated with the same photon flux and an assumed equal efficiency for all pixel wells.
Even if you stack infinite frames, you only reduce the variance of the noise, so it will get closer to an constant, but it will never reach it! Therefor an infinite small amount of noise will remaining in the mean image.

Also, when you stack the same number of frames (darks and lights) to get the mean values in your example (10e- and 100e-), a subtraction would make the noise in the resulting image more bad than in the original mean light frame, because the variance in the mean dark frame would be higher than the variance in the mean light frame.
Additive signal contaminants that are called "noise" are almost always defined so that their mean is zero. If a contaminant has non-zero mean, then it is typically modeled as two other components, with the mean being called something like "bias" or "offset", and "noise" being the rest.
I agree. That is the reason, why each frame have added an offset by the read out amp, because the D/A can not calculate with zero. This offset would be removed with a bias frame in image calibration and is always a constant (physically the bias frame isn´t constant over the entire frame, because of several disturbance, i.e. accumulating more thermal electrons during the readout process for the last pixel loads or electro magnetic disturbance through electronic circuits and the read noise itself).

Cheers Markus

Lou Jost
Posts: 4859
Joined: Fri Sep 04, 2015 7:03 am
Location: Ecuador
Contact:
Markus, of course the noise can't be completely eliminated. No one expects that.
"..a subtraction would make the noise in the resulting image more bad than in the original mean light frame, because the variance in the mean dark frame would be higher than the variance in the mean light frame."
No. If we are thinking at the electron level, and talking about the actual variance (not the variance relative to the mean), then the variance in the averaged light frames is greater than or equal to the variance in the dark frames. The subtraction is necessary to remove the mean value of the dark noise (or the "offset" if you prefer). And by averaging enough frames we can make the variance of the averaged dark frames as small as we want.

Rik, you asked me what my model was. I don't have one, I am just trying to learn about how this works and trying to think carefully about the statistics.

Your comment on gamma is important to this discussion about subtracting. Do you know what the equation is that connects the raw data to the RGB values?

etalon
Posts: 67
Joined: Tue Jan 05, 2016 5:26 am
Location: Germany
Contact:
Lou Jost wrote:No. If we are thinking at the electron level, and talking about the actual variance (not the variance relative to the mean), then the variance in the averaged light frames is greater than or equal to the variance in the dark frames. The subtraction is necessary to remove the mean value of the dark noise (or the "offset" if you prefer). And by averaging enough frames we can make the variance of the averaged dark frames as small as we want.
Lou, I agree completely. But anyway, even if we calculate with the absolute variance, it will increase in the mean light frame after the subtraction compared with the mean light frame without any calibration...
Also, when you stack the same number of frames (darks and lights) to get the mean values in your example (10e- and 100e-), a subtraction would make the noise in the resulting image more bad than in the original mean light frame, because the variance in the mean dark frame would be higher than the variance in the mean light frame.
Here I talk about the relative variance. I didn´t specify it especially, so it was mistakable. My fault...

of course the noise can't be completely eliminated. No one expects that.
I understand you this way, because you want to subtract a constant...
remove the mean value of the dark noise (or the "offset" if you prefer)
In this case it is the amount of dark current.