lens testing setup and presentation

Have questions about the equipment used for macro- or micro- photography? Post those questions in this forum.

Moderators: ChrisR, Chris S., Pau, rjlittlefield

dickb
Posts: 249
Joined: Mon Jul 05, 2010 1:54 am

lens testing setup and presentation

Post by dickb »

Inspired by Robert O'Toole I have finally got (almost) all the components for a decent lens testing setup. Mine consists of a Nikon Multifot stand, an Olympus BH microscope base, 2 Metz 60CT4 flashes and a Godox diffusor circumcised in the way Robert has shown. I use my Canon 5DmkII, tethered to my pc with qDSLR dashboard for precise liveview. As a test subject I've got one of the Dallas Semiconductor chip wafers Robert also has.

I'm trying to get a test system that will allow for easy comparison, ideally both between lenses in my excessive collection and between lenses tested by Robert and perhaps others.

What would be the best way of presenting such lens test data?

dickb
Posts: 249
Joined: Mon Jul 05, 2010 1:54 am

Post by dickb »

As an example, here are two ways of showing a photo with a Canon 35mm f/2.8 MP35 at f/2.8 and 4:1 magnification, the whole photo SOOC but resized to 1600 pixels long
(I have a slight alignment problem, there seems to be a slight tilt somewhere in the system)

Image


A 1mm*1mm square of the subject at 100% from the centre of the image, with a bit of sharpening and CA reduction

Image

ray_parkhurst
Posts: 2656
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Post by ray_parkhurst »

I'm a big fan of animation to show the differences. The method eliminates the need for the user to pixel peep between the images, but has the disadvantages of it being more work, and of the bit depth being that of .gif rather than .jpg. It also works best when there are just 2 to perhaps 4 lenses to compare. Beyond that the animation becomes confusing.

rjlittlefield
Site Admin
Posts: 20872
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

It appears that the crop has been rotated with respect to the full frame.

If that means that the 100% image has been rotated from what came out of the camera, then the result cannot be meaningfully compared to any other image that is close in quality.

This is because slight rotation inevitably implies subpixel interpolation, which always messes up the sharpness one way or the other.

For comparison purposes, you should definitely avoid any sort of resizing or rotation.

I think it's also generally best to treat all images exactly the same, so the only thing different in the workflow is the lens.

I like to believe that people are interested in characterizing lenses so they can choose one that makes good photos. For that purpose, a good case could be made that it would be most relevant to treat each image with different sharpening, CA removal, etc -- whatever is needed to bring out the best in each lens. But given the subjectivity in "best", taking that approach is almost an open invitation to being accused of favoritism if you do it by hand. And if you do it by algorithm, say by just checking a box that says "Remove CA", then your results are not so meaningful for people who don't have the same software. Hence the advice to treat all images exactly the same.

I too am a big fan of animation. But again some care is needed. You cannot resize or rotate the images without messing up the comparison. Whole-pixel shifts are OK, and you should use those to center the two 100% crops. Be sure to include a label in each frame of the animation indicating what setup produced it.

--Rik

dickb
Posts: 249
Joined: Mon Jul 05, 2010 1:54 am

Post by dickb »

Thanks for the suggestions. Animation is interesting. Tricky to get the framing exactly the same, I would think. Especially with the rather large Godox diffusor and short working distances. I'll try a few things to see what works for me.

dickb
Posts: 249
Joined: Mon Jul 05, 2010 1:54 am

Post by dickb »

My first try at animated lens comparisons - well image quality comparisons between aperture values of a known good performer, a Canon MP 35/2.8 at 3:1 magnification, at 2.8, 4.0 and 5.6, actual pixels from a Canon 5D mkII.

Image


Making animations is rather cumbersome, at least with my software so far. The limitations of the picture upload of this forum are very annoying. I had to crop the image so only a few pixels are left from the image. If there is a way to upload files larger than 250kB I'd like to hear it.

For lens comparison images, would you like to see the levels adjusted or is the contrast SOOC interesting information? I'm not sure others like Robert do.

ray_parkhurst
Posts: 2656
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Post by ray_parkhurst »

Yes, it is very cumbersome.

In your sequence, it seems the f/4 is a bit less sharp than the f2.8, while f5.6 is sharper than either f/2.8 or f/4. This seems odd, and contradicts my own tests. Are you sure the order of images is correct? I found that f/4 was best at 2-3x. Could there be a focus issue?

The small shift in the images is not ideal but not a showstopper. I try to get within 1 pixel in my animations, but it's not always possible due to mag shifts vs different lenses or even on same lens vs aperture setting. Of course the best lenses don't shift mag vs aperture but a big % of lenses do, especially near the corners.

I use ezgif.com for my animations. It's free, simple to upload, and the results are pretty decent. I host on photobucket so there is no file size limitation for the group, but indeed photobucket limits animated gifs to something like 4MB, so you must be careful.

rjlittlefield
Site Admin
Posts: 20872
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

As far as I know, animations are cumbersome for everybody, although they do get simpler with practice.

The upload limit to photomacrography.net is 300 KB. That low value is a side effect of this forum's "business model" -- no membership fee, no advertisements, and no tracking of users. In principle we could change any or all of those, but just personally I'd rather not.
For lens comparison images, would you like to see the levels adjusted or is the contrast SOOC interesting information?
This strikes me the same as sharpening and CA removal. Unless all lenses are treated the same, it's hard to know what the results mean. SOOC is not required, but whatever you do, always do the same.

--Rik

dickb
Posts: 249
Joined: Mon Jul 05, 2010 1:54 am

Post by dickb »

It could well be a focus issue, but I see hardly any difference in the others of the focus stack. The sequence is correctly labelled, the decrease in CA is obvious in the larger images, maybe this works:

https://drive.google.com/open?id=1QL8aR ... Yl3taKPvxb


I think part of the problem may be the slight difference in contrast adjustment between the images, making the f.4.0 image look less contrasty. Also, the crop isn't the centre of the image. For some reason the forum image upload doesn't allow me to upload a 683*1024 pixel image, so again:

https://drive.google.com/open?id=1-lXVa ... 3gHcKFMb_S



And the image is a 24*36mm one, so I would expect IQ in that part of the image to increase from 2.8 to 4 and possibly 5.6.


I'm still figuring out what the ideal setup is. The large Godox diffusor works well, but doesn't allow me to change the aperture without lifting the diffusor and that can cause the wafer to move.. Perhaps tape should be the answer..

rjlittlefield
Site Admin
Posts: 20872
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

dickb wrote:For some reason the forum image upload doesn't allow me to upload a 683*1024 pixel image
The file served back by drive.google is length 397 KB. That size would cause the upload to be rejected by photomacrography.net because the pixel dimensions are OK but the file length is not.

If the pixel dimensions were too big also then the upload would be accepted but would automatically be downsized, not helpful for lens comparison work.

--Rik

dickb
Posts: 249
Joined: Mon Jul 05, 2010 1:54 am

Post by dickb »

rjlittlefield wrote:The upload limit to photomacrography.net is 300 KB. That low value is a side effect of this forum's "business model" -- no membership fee, no advertisements, and no tracking of users. In principle we could change any or all of those, but just personally I'd rather not.

I like that business model as well. I wasn't aware of the 300kB limit. It is a bit frustrating when the image upload text gives maximum dimensions of 1024*1024 and when I try and and upload an image with fewer pixels it still refuses to do so. Perhaps mentioning the 300kB limit there is a good idea.

RobertOToole
Posts: 1613
Joined: Thu Jan 17, 2013 9:34 pm
Location: United States
Contact:

Post by RobertOToole »

ray_parkhurst wrote:Yes, it is very cumbersome.

In your sequence, it seems the f/4 is a bit less sharp than the f2.8, while f5.6 is sharper than either f/2.8 or f/4. This seems odd, and contradicts my own tests. Are you sure the order of images is correct? I found that f/4 was best at 2-3x. Could there be a focus issue?

The small shift in the images is not ideal but not a showstopper. I try to get within 1 pixel in my animations, but it's not always possible due to mag shifts vs different lenses or even on same lens vs aperture setting. Of course the best lenses don't shift mag vs aperture but a big % of lenses do, especially near the corners.

I use ezgif.com for my animations. It's free, simple to upload, and the results are pretty decent. I host on photobucket so there is no file size limitation for the group, but indeed photobucket limits animated gifs to something like 4MB, so you must be careful.
Hi Ray,

You can also do everything in Photoshop. You can use a menu command to load all the open images into a single file as layers. Photoshop can then align the images. Finally you can save the file with the layers as a gif, setting the timing options at that point.

I haven't made a gif in Photoshop in a couple of years but its pretty simple once you do it a couple few times.

Robert

ray_parkhurst
Posts: 2656
Joined: Sat Nov 20, 2010 10:40 am
Location: Santa Clara, CA, USA
Contact:

Post by ray_parkhurst »

Yeah, I have a few copies of PS but rarely use it. Never wanted to get over the learning curve, but indeed auto alignment would be very useful. I do the alignment manually, which takes a bit of time.

RobertOToole
Posts: 1613
Joined: Thu Jan 17, 2013 9:34 pm
Location: United States
Contact:

Re: lens testing setup and presentation

Post by RobertOToole »

dickb wrote:Inspired by Robert O'Toole I have finally got (almost) all the components for a decent lens testing setup. Mine consists of a Nikon Multifot stand, an Olympus BH microscope base, 2 Metz 60CT4 flashes and a Godox diffusor circumcised in the way Robert has shown. I use my Canon 5DmkII, tethered to my pc with qDSLR dashboard for precise liveview. As a test subject I've got one of the Dallas Semiconductor chip wafers Robert also has.

I'm trying to get a test system that will allow for easy comparison, ideally both between lenses in my excessive collection and between lenses tested by Robert and perhaps others.

What would be the best way of presenting such lens test data?
Hi Dick,

I can help with that and give the processing workflow I use.

By experience I found the only way to do it, for me at least, is to process each image in a comparison with exactly the same settings so you can see minor differences and it makes testing a lot more repeatable and consistent results.

For example I prefer to have noise reduction set to zero, and also all lens correction turned off. This is a problem with Photoshop and Lightroom though since even with some settings like contrast zero'd out, the baseline is still offset a bit. I do add some mild sharpening in the RAW conversion, 40, 0.4, 0, but I don't apply sharpening to the converted images.

Using this method I can re-test or add another lens and get results that are easy to comparable.

BTW I am in the process of comparing a bunch of tube lenses and a pile of scanner lenses, I think my brain now has a wafer disk image permanently etched into my prefrontal cortex :D

All the best,
Robert

rjlittlefield
Site Admin
Posts: 20872
Joined: Tue Aug 01, 2006 8:34 am
Location: Richland, Washington State, USA
Contact:

Post by rjlittlefield »

RobertOToole wrote:You can use a menu command to load all the open images into a single file as layers. Photoshop can then align the images.
If you're talking about Edit > Auto-Align Layers, then suddenly I have become very uncomfortable due to the sub-pixel interpolation issue discussed above.

But probably I have misunderstood what you're talking about. Can you explain in more detail?

--Rik

Post Reply Previous topicNext topic