Slanted Edge MTF Measurements and Interpretation

[Greenfields]

Introduction

Miljenko’s chart of his measurements of the Modulation Transfer Function of the Sigma 70mm lens:

http://www.photomacrography.net/forum/v ... hp?t=22379

prompted me to wonder what the measurements mean.

This post is an invitation for anyone to give me a sanity check because I am not confident of my conclusions.

If someone can tell me how to calculate a diffraction limited curve: that would be even better.

Slanted Edge Software

The post is about the results of software like Imatest which use the slanted-edge method to calculate the MTF of digital systems [lens + sensor + processing], and particularly about understanding the MTF curve produced by such software for a single point.



This is an example of such an MTF curve - for the Sigma 70mm lens used at 1x magnification at a nominal aperture of f/5.6.

With my copy of the lens this aperture produced the highest MTF at 1:1 on its axis. The lens has a pupil ratio of about 1.13 at 1x [it varies slightly with magnification] so the effective aperture is about 9.2 when adjusted for the shortened focal length of the lens [c. 61mm] at 1:1

Most MTF charts published by online testers use software based on ISO12233 which uses the image of a sharp grey/white edge inclined at about 5 degrees to the sensor’s rows and columns to calculate the MTF. The analysis uses tens of rows of several tens or even hundreds of pixels. Because of the slanted edge the several hundred pixels used are all at slightly different distances from the edge enabling the software to determine the edge profile and MTF value with a much greater spatial resolution than the size of the phototites.

The effective resolution of the slanted edge software is much greater than the resolution of the sensor when the senor is used to take pictures.

Imatest is probably the most well known, but has long evolved away from photographers to industrial users and is now very expensive. There are two free applications which can produce the same charts: Image J’s Slanted Edge MTF Plugin and Frans van den Bergh’s MTF Mapper [though a trick is needed to catch the CSV files MTF Mapper generates before it closes].

Sensor Resolution

If you have got this far you probably know that the resolution of a sensor is limited by its Nyquist frequency: the fact that you need a minimum of two rows or columns to records a dark/light cycle. That’s the spatial frequency of 0.5 cycles per pixel marked on the chart.

It practice it’s worse, because the two rows would have to be aligned with the detail and the detail would have to be of high contrast to stand any chance of being detected. There is also the colour filter array which most cameras use, which means that each pixel only samples one colour channel so the missing information has to be interpolated by demosaicing. Tests on video cameras which come in single-senor [with a colour filter array] and three-sensor versions [no colour filter array or demosaicing needed] indicate that the CFA reduces linear resolution by about 30 percent.

To sum up, to detect randomly oriented detail at moderate contrast, in practice you need an absolute minimum of about 3 pixels and probably a bit more.

Interpreting the MTF Chart

Returning to the MTF chart, my first thought is that only the data to the left of point A: i.e. spatial frequencies of below 0.33 cycles per pixel or perhaps even less is of any photographic significance. The rest of the curve to the right of point A is real data - but its of no direct photographic significance because these frequencies cannot be detected by the sensor.

By “no direct” significance I am implying that it is important indirectly: Its important for a lens to out-resolve a sensor by a substantial margin [e.g. a factor of two in limiting resolution] if possible in order to make sure that the MTF at point A is high enough to be seen as detail.

Is a lens merely matches the resolution of a sensor [that seems to be the reference point for some cellphone designs] the image won’t look sharp because the MTF at the limiting frequency will be too low. This eventually happens when a lens is opened out [due to aberrations] or closed down [due to diffraction].

My other thought - and it’s only a suggestion, is that to the right of Point A the slanted edge method is recording real data, not just noise or aliasing, at least until somewhere about B when noise does take over. I suggest that the curve is mainly recording lens performance as attenuated by any low pass filter, but that this performance can be recorded beyond the Nyquist limit because of the super sampling of the slanted edge method despite the fact that it cannot be captured by a conventional image.

Adding a Diffraction Curve ?

Finally, a question: Does anyone know how to calculate a diffraction-limited MTF curve to add to a chart like this to illustrate how closely the performance of the lens is to its physical limit ?

I have tried using the formulae I have found but the results show this lens to be working so far below its diffraction-limited performance that I can’t believe I have it right.

I tried this formula which is said to give MTF vs spatial frequency in clycles/mm



Using this data:

Pupil Ratio at 1:1 magnification : P = 1.13
Nominal aperture fr = f/5.6
Magnification m = 1.0
Effective Aperture, fe = fr (m/P + 1) = 5.6 (0.885 + 1) = f/10.6

... then adjusting for the fact that at 1:1 the focal length of the lens has reduced from 70mm to c. 61mm, which, for a fixed entrance pupil diameter would change the focal ratio in proportion to the focal length to f/9.2

Cut-off frequency νc =1 / (λ fe)
Assume λ = 0.55 micons (i.e. 550 nm)
νc =1 / (.000550 x 9.2)
νc =1 / (.00506)
νc =197.6 cycles/mm.

Each pixel is 0.00641 mm so the cut-off frequency is 1.267 cycles/pixel

I got this blue calculated curve:



I can’t believe it.

The published results for other lenses suggest that at its optimum aperture for sharpness the Sigma should be very close to the calculated curve.

Where have I gone wrong ?

Henry

[rjlittlefield]

Henry,

I've spent almost zero time studying the slanted-edge method of MTF measurement.

However, I've spent quite a bit of time looking at optically magnified images to see what's really in them, beyond what can be captured by current digital sensors.

Just eyeballing your curves, I agree that the blue curve in your last plot looks appropriate for an effective f/9.2 lens on a 5D Mark III sensor (photosite size 6.25 microns). You can see in the optically magnified images HERE and following that even f/11 resolves bars at 1.78X higher resolution than is captured by a Canon T1i camera with photosite size 4.7 microns. So it's perfectly reasonable to me that an f/9.2 lens would not reach cutoff until 2.5X Nyquist on the 5D Mark III.

So, I believe the disconnect lies in these words you wrote: to calculate the MTF of digital systems [lens + sensor + processing].

In the system you're looking at, the overall MTF is limited far more by finite sensor resolution than by optical diffraction. I think all we're seeing in the slanted-edge MTF curve is experimental confirmation of that fact.

Perhaps some useful intuition could be obtained by stopping down the optical system until most of the MTF degradation is due to diffraction, not sensor resolution. I'm thinking of something like f/100, maybe f/200 -- basically as small an aperture as you can use without driving crazy the slanted-edge computation. My expectation is that if you do that, the slanted-edge curve will then come a lot closer to matching the calculated diffraction curve.

--Rik

[Greenfields]

Thanks, Rik,

I had made the assumption that when the lens produces its sharpest images it is diffraction-limited.

That assumption is unlikely to be correct: it is more likely that the constraints of diffraction and aberrations are balanced - at least as far as resolution is concerned.

I am still puzzled by why the measured and calculated curves are so different. I expected that the measured curve would follow the calculated curve quite closely at low frequencies, then fall away broadly somewhere around the Nyquist limit - but because of the OLP filter rather than because of the limiting resolution of the array.

I had in mind measurements made by Carles Mitja [one of the authors of the Image J Slanted Edge MTF plugin] in a paper reporting that the old Nikkor 55mm f/3.5 macro was almost diffraction-limited down to f/4 on the basis of optical MTF measurements.

That published paper was once online - but I can't find it there now. I do have a PDF.

Would it be "fair use" to post a clip of the published chart with a proper citation from that paper this site ? [I am nervous about posting someone else's work]

Henry

[Miljenko]

Hi Henry, nice you've pulled this theme I'm personally involved with almost since day one of Norman Koren's Imatest development. Most of the answers you wonder about can be found within Imatest official pages.
Concerning your particular problem I will stress once again the importance of processing during (raw) photo file reading. You have nicely wrote yourself "...MTF of digital systems [lens + sensor + processing]..." but you forgot about the last factor later on. I have mentioned sharpness (resolution!) restoration within reading process in my macro lens test post and big differences between various raw converters. And there is another factor involved; demosaicing algorithm used. The fact is that most raw converters today use DcRaw math simply because David Coffin was generous enough to share it's work for free with the world. Unfortunately, generic DcRaw doesn't perform any form of resolution restoration after demosaicing. Therefore, the final result is what you got in your chart. There are converters that do just that; convert with DcRaw algorithm and leave detail loss restoration to the user. Some restore it automatically with fixed or variable amount of sharpening. If the converter uses conversion routine other than DcRaw, resolution may vary, sometimes to a large extend. I have compared different converters on few occasions in the past with very interesting results. When it comes to detail restoration, RawMagick Lite was absolutely the best converter ever (yours truly being one of the beta testers). Unfortunately, for reasons unknown to me, authors stopped developing RML some time ago. I can assure you that your MTF line would be much closer to diffraction curve if read with RML.
I am at work right now but when I come home tonight, I will post interesting set of charts where slanted edge target was read with 15+ raw converters. You will see how very different curves look like!

[rjlittlefield]

Would it be "fair use" to post a clip of the published chart with a proper citation from that paper this site ?

Yes, in my opinion this falls well within the boundaries of fair use. Such excerpts are regularly posted in this forum. Just be sure to keep the clip small and to include proper references.

--Rik

[Greenfields]

Thank you.

This chart is an example of what makes me concerned about the difference between the measured MTF of the Sigma 70mm lens and the calculated curve.

These curves are for the Nikkor 55mm f/3.5 lens:



Source:
Relationships between lens performance and different sensor sizes in professional photographic still SLR cameras
by Carles Mitjà, JaumeEscofet, Fidel Vega

The article includes other data which says that the limting frequency of the measured MTF curve is consistently only 5% lower than the diffraction limit.

These curves were determined by optical instruments [details in the original].

At higher spatial frequencies the slanted-edge measurements will be limited by the OLP filter [at least in cameras which have an OLP filter]. The few published curves for OLP filters show them progressively attenuating a broad range of spatial frequencies rather than a narrow band.

Despite that, I am still surprised [incredulous ?] that the OLP filter could produce the difference in MTF seen in the slanted-edge measurements of the Sigma 70mm lens. If it did, the difference between images from cameras with and without OLP filters would be much more dramatic than they are.

My instinct is that if the 55mm is almost diffraction-limited, the Sigma should be too.

I also find it hard to believe that the reason is an intrinsic limitation of slanted-edge testing or someone, surely, would have made this observation before.

Henry

[Miljenko]

Here are the charts I promised. Fast and simple example is comparison between UF Raw (pure DC Raw implementation) and Phase One's Capture One Pro 7 (default settings for this particular camera). Camera used is Nikon D7100, the lens AF-S DX Micro-NIKKOR 40mm f/2.8G at it's sharpest aperture setting, f/4.5. Here UF Raw produces very similar curve you presented above for Sigma 70. MTF50 value is only 50% of the total vertical pixel count. OTOH, C1 gets you exactly all available pixel count.
So, which one ic correct?
We can now dig into tons of theory but let me add one purely subjective view: When aligned side by side, same real life photo looks totally natural, crisp and sharp when read by C1 while DCraw presentation looks like slightly out of focus, with weak local contrast and instead of supersharp macro lens photo seems like it's being shot with the kit zoom lens!



Couple of months ago when I compared raw converters, there were 18 programs that could read D7100 NEF files. Since comparison chart is much larger than 1024 pixel size allowed in Photomacrography forums, I have uploaded it to my Pbase gallery so please check it here: http://www.pbase.com/miljenko/image/153993859/original

You will easily discover which converters use DCRaw algorithm and which use proprietary math. Adobe Photoshop ACR was tried with 3 presharpening settings: 25%, 50% and 100%. For quite some time I use 50% setting as my Imatest default.
When shooting real (macro) pics, Capture One is definitely my converter (and complete photo editing SW) of choice.

[Miljenko]

I am still puzzled by why the measured and calculated curves are so different.

Norman Koren on Imatest documentation pages says: "....Lens MTF response can never exceed the diffraction limited response, but system MTF response often exceeds it at medium spatial frequencies as a result of sharpening, which is (and should be) present in most digital imaging systems.
In addition to lens response, system MTF response is affected by the sensor (which has a null at 1 cycle/pixel), the anti-aliasing filter (designed to suppress energy above 0.5 cycles/pixel), and signal processing (which can be very complex— it can be different in different regions of an image)...."

[Greenfields]

Thank you, Miljenko.

your comments have been very helpful.

I think that you are saying that the MTF of a lens can only be measured objectively by optical instruments. That is the only way to isolate the performance of a lens from effects of the rest of a system.

I was hoping that the performance of a lens was the main factor in the MTF of a camera image, at least at pictorially significant spatial frequencies and when sharpness in processing is minimised. The results we both have obtained seem to demonstrate that the rest of the imaging chain, especially sharpening in development or post processing, has such a powerful effect that MTF values derived from an image are merely indicators of the performance of a lens.

MTF values measured by slanted-edge software just can't be compared with calculated curves.

That does not mean that measurements made under standard conditions, like yours, can't give valuable information about the performance of a lens. What then matters are the relative values for different lenses or different settings of the same lens.

Have you any thoughts on the suggestion that because of the Bayer CFA and other factors, three or more pixels are needed to record pictorially significant detail ? That is a widespread conclusion in photomacrography and seems to mean that only the parts of the MTF curve for spatial frequencies below, say 0.33 cycles/pixel (or even less) matter.

Henry

[Miljenko]

Henry, you have come to pretty sound and logical conclusions. Due to variables you wisely discovered, there is practically no way to establish absolute MTF values for certain using a camera. Worse yet, even relative relations are not always carved in stone. Even when doing one's best to keep all measuring parameters fixed, being forced to change one factor only makes different sessions incomparable. This is why my Imatests performed during 10 year period don't have much in common if you compare latest to the earliest tests.
When we talk about interesting objective to sensor resolution ratio, you should understand the issue is not that straightforward. It has so much to do with color involved. Slanted edge resolution test deal with B/W patterns so every pixel site act as single discrete source of information. This is the only situation where you get full pixel count. Unfortunately, this situation is far from realistic. When the color information is involved, effective resolution drops considerably, ratio being dependent on the actual object color. This is due to different percentage of blue, green and red photosites, and this is the reason why we talk different ratios; 1:2.5, 1:3, 1:3.5 or whatever you like in between.

[rjlittlefield]

I think that you are saying that the MTF of a lens can only be measured objectively by optical instruments. That is the only way to isolate the performance of a lens from effects of the rest of a system.

A general rule of thumb about measurements is that the measuring instrument should be several times better than the item under test. With current digital sensors and good lenses at their optimum apertures, this relationship is reversed.

The f/5.6 lens shown HERE retains significant contrast at a frequency corresponding to about 1 cycle per 0.9 pixel, on a 15 megapixel APS-C sensor. In that particular test, the key technique was to use a 10X microscope objective to optically expand the image to around 1 cycle per 9 pixels. That scaled the pattern to be something that was well within the range of the digital sensor, where the original pattern would have been far beyond the Nyquist limit. One way to think about this is that the microscope optics plus the digital sensor become what you're calling an "optical instrument".

I was hoping that the performance of a lens was the main factor in the MTF of a camera image, at least at pictorially significant spatial frequencies and when sharpness in processing is minimised.

When a good lens is set to its optimum aperture, the MTF of a low magnification camera image is determined mostly by the sensor. I expect that someday this will not be the case, but someday is not today (with the possible exception of 41 megapixel cell phones). This is the reason that teleconverters are still useful.

I had made the assumption that when the lens produces its sharpest images it is diffraction-limited.

That assumption is unlikely to be correct: it is more likely that the constraints of diffraction and aberrations are balanced - at least as far as resolution is concerned.

I agree. However, I think that for this discussion the key point is that even the combined effects of diffraction and aberration can still produce an optical image that is far beyond the sensor's limit. In that case the MTF detected by the slanted-edge test is mostly limited by the sensor, not the lens.

--Rik

[Greenfields]

Thank you Miljenko and Rik for your thoughtful explanations.

I agree now.

Its not that the MTF curves are not useful: merely that they did not mean what I first thought they meant.

Henry