Comparing macro lenses using MTF - Part I

Have questions about the equipment used for macro- or micro- photography? Post those questions in this forum.

Moderators: Pau, rjlittlefield, ChrisR, Chris S.

Miljenko
Posts: 170
Joined: Sat Jun 01, 2013 11:53 pm
Location: Zagreb, Croatia
Contact:

Comparing macro lenses using MTF - Part I

Post by Miljenko »

Image

I believe there is no photographer who didn't at least once compare two or more lenses in a search for better optical performance. So you probably know the procedure: find a distant object with fine textures, mount a camera on a tripod and shoot while changing the aperture. I started comparing lenses borrowed from my friends while we were still shooting on film. At first the moving force was a search for a better lens for my photo bag but later it became kind of obsession. It had something to do with my college days in 70's when Testing and Measuring in Electronics was my favorite faculty course. So along with hundreds of hours spent in testing lab I was trying to improve my lens testing routines and getting USAF 1951 test charts was one of the important moves I've done. Comparisons become quantified, repeatable and doable even between lenses tested many years apart. Of course, testing became more convenient and exact when I bought my first DSLR in 2002.. Along with experience came dissatisfaction with inaccuracy due to iffy resolution assessment relying on the eye and brain. Huge relief came in 2004. when Norman Koren introduced his Imatest testing software and I was lucky to get the early beta copy. The concept bought me at once because finaly I could get exact figures for resolution and other lens parameters. This essential brakethrough nicely coincided with my involvment as contributing editor for the national photo magazine where I was doing lenses testing and reviewing. Since then I have tested more than 100 lenses, half of those being of macro variety.

Image

I didn't plan to elaborate advantages and shortcomings of slanted edge MTF method here but strongly advice reading valuable documentation at http://www.imatest.com/docs/ and other sources. Besides Imatest home edition (officially called Studio and sold for $300) there are other slanted edge programs available online, mostly for free, like SRF Edge, MTF Mapper and QMTF Light. In short, I believe MTF50 way of testing and comparing lenses when compared to visual assessment of photographed patterns gives more objective, exact and repetable results. More than one tester can participate in comparing lenses even when living thousands of miles apart, provided they have the same camera make and model. Even different cameras can give comparable results if test shots are prepared in appropriate way.

Image

My testing procedure is very straightforward:

I shoot classic double edge razor blade in front of diffused back light mounted on a macro rail. Step is 1/20 of a DOF for particular lens and aperture. Usually 20 shots are enough except for the lenses with strong focusing plane curvature where 10 – 30 additional shots are done. For the last year or so I’m using 24MP APS-C Fuji X-T2 camera, shooting raw for this purpose. Pics are read with Capture One with default settings, exported as 100% quality jpegs (when using tiff difference is within error margins). Those are imported to Imatest and final test charts are selected by visual inspection for the max readings at center and both edges. Lenses with variable aperture get multiple runs, but recently I gave up testing full range of apertures, narrowing the selection to f4 – f8 instead.

Image

My method differs only in 2 instances compared to Norman Koren’s recommendations: black to white contrast is higher (does not affect results much) and I do use sharpening before Imatesting. I am strongly against generic Dcraw conversion lacking any sharpening because the results do not correlate well with perceived lens resolution. Besides, Dcraw in general can’t extract full sensor resolution. Solution is raw converter with standardized sharpening. After trying dozen of raw converters, I decided on Capture One for two virtues: It does squeeze out the last pixel from the sensor and it applies just the right amount of default sharpening. Even the sharpest lenses are not oversharpened and get just the right MTF figures that match nicely perceived lens resolution.

Above is a list of macro related lenses tested in a last two years, including some tube (relay) lenses as well. In this series you won’t see resolution figures for all those lenses because I have introduced “stage zero” where lenses with poor subjective resolution (I’m using certain butterfly wing sample) are excluded from MTF testing. The chart you see here (Nikon 63mm) is typical final result summary after complete enlarger lens test. Such chart requires enormous amount of time and since first dozen of tests performed on enlarger lenses showed max resolution in f 4.8 – 5.6 - 6.8 range, I shortened the procedure to just those three. This is why you are going to see just 3 figures for each lens test in this series: max center resolution / max edge resolution / chromatic aberration. Exact, meaningful and comparable.

In part 2 I will unveil few important general findings that arose thanks to number of tested lenses and thorough procedures applied. I believe some of the findings will be very surprizing and unexpected.

BTW, I am not connected in any way to Imatest LLC company or Mr. Norman Koren although we both used to design high end audio gear and share love for photography.

Hoping you will find following parts both interesting and useful.

Miljenko
All things are number - Pythagoras

typestar
Posts: 197
Joined: Sat Dec 12, 2009 1:45 am
Location: Austria

Re: Comparing macro lenses using MTF - Part I

Post by typestar »

Miljenko wrote: Hoping you will find following parts both interesting and useful.
Miljenko,

as many of us like the intensive hard testing-work and the detailled results and comparison of appreciated member Robert O'toole, it will be interesting to see your findings for some of the lenses you list here.

appreciated in advance

typestar
Last edited by typestar on Sat Sep 08, 2018 5:52 am, edited 1 time in total.

Smokedaddy
Posts: 1714
Joined: Sat Oct 07, 2006 10:16 am
Location: Phoenix, Arizona
Contact:

Post by Smokedaddy »

Miljenko,

I've had Imatest for many years but am having problems getting it up and running on a new computer I purchased. Still working with Norman trying to iron things out.

Anyway, my real question is have you ever tried using MTF Mapper too? Just curious.

https://mtfmapper.blogspot.com/

-JW:

Miljenko
Posts: 170
Joined: Sat Jun 01, 2013 11:53 pm
Location: Zagreb, Croatia
Contact:

Post by Miljenko »

Smokedaddy wrote:Miljenko,

I've had Imatest for many years but am having problems getting it up and running on a new computer I purchased. Still working with Norman trying to iron things out.
James, I'm using very old version too and it can't work on Windows 10 PC. This is why I retained one ancient Win 7 laptop just for running Imatest!
Smokedaddy wrote: Anyway, my real question is have you ever tried using MTF Mapper too? Just curious.
Nope, I'm perfectly satisfied with Imatest so had no reason to switch to anything else!
All things are number - Pythagoras

Smokedaddy
Posts: 1714
Joined: Sat Oct 07, 2006 10:16 am
Location: Phoenix, Arizona
Contact:

Post by Smokedaddy »

I have Imatest 3.5.1 Studio. I have a couple of old desktops and will give that a shot. I didn't realize Windows 10 could be the problem.

... thanks.

Online
RobertOToole
Posts: 1720
Joined: Thu Jan 17, 2013 9:34 pm
Location: United States
Contact:

Post by RobertOToole »

Hi Miljenko,

The equipment list is larger than I expected. Thats an impressive amount of gear!

Looking forward to seeing the results the rest of the test.

All the best,

Robert

mjkzz
Posts: 1237
Joined: Wed Jul 01, 2015 3:38 pm
Location: California/Shenzhen
Contact:

Post by mjkzz »

that is a LONG list, can't wait for it.

Post Reply Previous topicNext topic