www.photomacrography.net :: View topic - lens testing setup and presentation
www.photomacrography.net Forum Index
An online community dedicated to the practices of photomacrography, close-up and macro photography, and photomicrography.
Photomacrography Front Page Amateurmicrography Front Page
Old Forums/Galleries
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
lens testing setup and presentation
Goto page 1, 2  Next
 
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Equipment Discussions
View previous topic :: View next topic  
Author Message
dickb



Joined: 05 Jul 2010
Posts: 189

PostPosted: Mon May 21, 2018 2:51 pm    Post subject: lens testing setup and presentation Reply with quote

Inspired by Robert O'Toole I have finally got (almost) all the components for a decent lens testing setup. Mine consists of a Nikon Multifot stand, an Olympus BH microscope base, 2 Metz 60CT4 flashes and a Godox diffusor circumcised in the way Robert has shown. I use my Canon 5DmkII, tethered to my pc with qDSLR dashboard for precise liveview. As a test subject I've got one of the Dallas Semiconductor chip wafers Robert also has.

I'm trying to get a test system that will allow for easy comparison, ideally both between lenses in my excessive collection and between lenses tested by Robert and perhaps others.

What would be the best way of presenting such lens test data?
Back to top
View user's profile Send private message
dickb



Joined: 05 Jul 2010
Posts: 189

PostPosted: Mon May 21, 2018 3:02 pm    Post subject: Reply with quote

As an example, here are two ways of showing a photo with a Canon 35mm f/2.8 MP35 at f/2.8 and 4:1 magnification, the whole photo SOOC but resized to 1600 pixels long
(I have a slight alignment problem, there seems to be a slight tilt somewhere in the system)




A 1mm*1mm square of the subject at 100% from the centre of the image, with a bit of sharpening and CA reduction

Back to top
View user's profile Send private message
ray_parkhurst



Joined: 20 Nov 2010
Posts: 2217
Location: Santa Clara, CA, USA

PostPosted: Mon May 21, 2018 3:14 pm    Post subject: Reply with quote

I'm a big fan of animation to show the differences. The method eliminates the need for the user to pixel peep between the images, but has the disadvantages of it being more work, and of the bit depth being that of .gif rather than .jpg. It also works best when there are just 2 to perhaps 4 lenses to compare. Beyond that the animation becomes confusing.
Back to top
View user's profile Send private message Visit poster's website
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 19913
Location: Richland, Washington State, USA

PostPosted: Mon May 21, 2018 5:05 pm    Post subject: Reply with quote

It appears that the crop has been rotated with respect to the full frame.

If that means that the 100% image has been rotated from what came out of the camera, then the result cannot be meaningfully compared to any other image that is close in quality.

This is because slight rotation inevitably implies subpixel interpolation, which always messes up the sharpness one way or the other.

For comparison purposes, you should definitely avoid any sort of resizing or rotation.

I think it's also generally best to treat all images exactly the same, so the only thing different in the workflow is the lens.

I like to believe that people are interested in characterizing lenses so they can choose one that makes good photos. For that purpose, a good case could be made that it would be most relevant to treat each image with different sharpening, CA removal, etc -- whatever is needed to bring out the best in each lens. But given the subjectivity in "best", taking that approach is almost an open invitation to being accused of favoritism if you do it by hand. And if you do it by algorithm, say by just checking a box that says "Remove CA", then your results are not so meaningful for people who don't have the same software. Hence the advice to treat all images exactly the same.

I too am a big fan of animation. But again some care is needed. You cannot resize or rotate the images without messing up the comparison. Whole-pixel shifts are OK, and you should use those to center the two 100% crops. Be sure to include a label in each frame of the animation indicating what setup produced it.

--Rik
Back to top
View user's profile Send private message Visit poster's website
dickb



Joined: 05 Jul 2010
Posts: 189

PostPosted: Tue May 22, 2018 12:44 pm    Post subject: Reply with quote

Thanks for the suggestions. Animation is interesting. Tricky to get the framing exactly the same, I would think. Especially with the rather large Godox diffusor and short working distances. I'll try a few things to see what works for me.
Back to top
View user's profile Send private message
dickb



Joined: 05 Jul 2010
Posts: 189

PostPosted: Sun May 27, 2018 11:07 am    Post subject: Reply with quote

My first try at animated lens comparisons - well image quality comparisons between aperture values of a known good performer, a Canon MP 35/2.8 at 3:1 magnification, at 2.8, 4.0 and 5.6, actual pixels from a Canon 5D mkII.




Making animations is rather cumbersome, at least with my software so far. The limitations of the picture upload of this forum are very annoying. I had to crop the image so only a few pixels are left from the image. If there is a way to upload files larger than 250kB I'd like to hear it.

For lens comparison images, would you like to see the levels adjusted or is the contrast SOOC interesting information? I'm not sure others like Robert do.
Back to top
View user's profile Send private message
ray_parkhurst



Joined: 20 Nov 2010
Posts: 2217
Location: Santa Clara, CA, USA

PostPosted: Sun May 27, 2018 11:28 am    Post subject: Reply with quote

Yes, it is very cumbersome.

In your sequence, it seems the f/4 is a bit less sharp than the f2.8, while f5.6 is sharper than either f/2.8 or f/4. This seems odd, and contradicts my own tests. Are you sure the order of images is correct? I found that f/4 was best at 2-3x. Could there be a focus issue?

The small shift in the images is not ideal but not a showstopper. I try to get within 1 pixel in my animations, but it's not always possible due to mag shifts vs different lenses or even on same lens vs aperture setting. Of course the best lenses don't shift mag vs aperture but a big % of lenses do, especially near the corners.

I use ezgif.com for my animations. It's free, simple to upload, and the results are pretty decent. I host on photobucket so there is no file size limitation for the group, but indeed photobucket limits animated gifs to something like 4MB, so you must be careful.
Back to top
View user's profile Send private message Visit poster's website
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 19913
Location: Richland, Washington State, USA

PostPosted: Sun May 27, 2018 12:05 pm    Post subject: Reply with quote

As far as I know, animations are cumbersome for everybody, although they do get simpler with practice.

The upload limit to photomacrography.net is 300 KB. That low value is a side effect of this forum's "business model" -- no membership fee, no advertisements, and no tracking of users. In principle we could change any or all of those, but just personally I'd rather not.

Quote:
For lens comparison images, would you like to see the levels adjusted or is the contrast SOOC interesting information?

This strikes me the same as sharpening and CA removal. Unless all lenses are treated the same, it's hard to know what the results mean. SOOC is not required, but whatever you do, always do the same.

--Rik
Back to top
View user's profile Send private message Visit poster's website
dickb



Joined: 05 Jul 2010
Posts: 189

PostPosted: Sun May 27, 2018 12:17 pm    Post subject: Reply with quote

It could well be a focus issue, but I see hardly any difference in the others of the focus stack. The sequence is correctly labelled, the decrease in CA is obvious in the larger images, maybe this works:

https://drive.google.com/open?id=1QL8aRNZ8NIpWZYz0Ou-0wsYl3taKPvxb


I think part of the problem may be the slight difference in contrast adjustment between the images, making the f.4.0 image look less contrasty. Also, the crop isn't the centre of the image. For some reason the forum image upload doesn't allow me to upload a 683*1024 pixel image, so again:

https://drive.google.com/open?id=1-lXVaJugA2JWHPe14hKeS53gHcKFMb_S



And the image is a 24*36mm one, so I would expect IQ in that part of the image to increase from 2.8 to 4 and possibly 5.6.


I'm still figuring out what the ideal setup is. The large Godox diffusor works well, but doesn't allow me to change the aperture without lifting the diffusor and that can cause the wafer to move.. Perhaps tape should be the answer..
Back to top
View user's profile Send private message
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 19913
Location: Richland, Washington State, USA

PostPosted: Sun May 27, 2018 12:24 pm    Post subject: Reply with quote

dickb wrote:
For some reason the forum image upload doesn't allow me to upload a 683*1024 pixel image

The file served back by drive.google is length 397 KB. That size would cause the upload to be rejected by photomacrography.net because the pixel dimensions are OK but the file length is not.

If the pixel dimensions were too big also then the upload would be accepted but would automatically be downsized, not helpful for lens comparison work.

--Rik
Back to top
View user's profile Send private message Visit poster's website
dickb



Joined: 05 Jul 2010
Posts: 189

PostPosted: Sun May 27, 2018 12:27 pm    Post subject: Reply with quote

rjlittlefield wrote:
The upload limit to photomacrography.net is 300 KB. That low value is a side effect of this forum's "business model" -- no membership fee, no advertisements, and no tracking of users. In principle we could change any or all of those, but just personally I'd rather not.



I like that business model as well. I wasn't aware of the 300kB limit. It is a bit frustrating when the image upload text gives maximum dimensions of 1024*1024 and when I try and and upload an image with fewer pixels it still refuses to do so. Perhaps mentioning the 300kB limit there is a good idea.
Back to top
View user's profile Send private message
RobertOToole



Joined: 17 Jan 2013
Posts: 1260
Location: United States

PostPosted: Wed May 30, 2018 7:36 pm    Post subject: Reply with quote

ray_parkhurst wrote:
Yes, it is very cumbersome.

In your sequence, it seems the f/4 is a bit less sharp than the f2.8, while f5.6 is sharper than either f/2.8 or f/4. This seems odd, and contradicts my own tests. Are you sure the order of images is correct? I found that f/4 was best at 2-3x. Could there be a focus issue?

The small shift in the images is not ideal but not a showstopper. I try to get within 1 pixel in my animations, but it's not always possible due to mag shifts vs different lenses or even on same lens vs aperture setting. Of course the best lenses don't shift mag vs aperture but a big % of lenses do, especially near the corners.

I use ezgif.com for my animations. It's free, simple to upload, and the results are pretty decent. I host on photobucket so there is no file size limitation for the group, but indeed photobucket limits animated gifs to something like 4MB, so you must be careful.


Hi Ray,

You can also do everything in Photoshop. You can use a menu command to load all the open images into a single file as layers. Photoshop can then align the images. Finally you can save the file with the layers as a gif, setting the timing options at that point.

I haven't made a gif in Photoshop in a couple of years but its pretty simple once you do it a couple few times.

Robert
Back to top
View user's profile Send private message Visit poster's website
ray_parkhurst



Joined: 20 Nov 2010
Posts: 2217
Location: Santa Clara, CA, USA

PostPosted: Wed May 30, 2018 7:44 pm    Post subject: Reply with quote

Yeah, I have a few copies of PS but rarely use it. Never wanted to get over the learning curve, but indeed auto alignment would be very useful. I do the alignment manually, which takes a bit of time.
Back to top
View user's profile Send private message Visit poster's website
RobertOToole



Joined: 17 Jan 2013
Posts: 1260
Location: United States

PostPosted: Wed May 30, 2018 7:51 pm    Post subject: Re: lens testing setup and presentation Reply with quote

dickb wrote:
Inspired by Robert O'Toole I have finally got (almost) all the components for a decent lens testing setup. Mine consists of a Nikon Multifot stand, an Olympus BH microscope base, 2 Metz 60CT4 flashes and a Godox diffusor circumcised in the way Robert has shown. I use my Canon 5DmkII, tethered to my pc with qDSLR dashboard for precise liveview. As a test subject I've got one of the Dallas Semiconductor chip wafers Robert also has.

I'm trying to get a test system that will allow for easy comparison, ideally both between lenses in my excessive collection and between lenses tested by Robert and perhaps others.

What would be the best way of presenting such lens test data?


Hi Dick,

I can help with that and give the processing workflow I use.

By experience I found the only way to do it, for me at least, is to process each image in a comparison with exactly the same settings so you can see minor differences and it makes testing a lot more repeatable and consistent results.

For example I prefer to have noise reduction set to zero, and also all lens correction turned off. This is a problem with Photoshop and Lightroom though since even with some settings like contrast zero'd out, the baseline is still offset a bit. I do add some mild sharpening in the RAW conversion, 40, 0.4, 0, but I don't apply sharpening to the converted images.

Using this method I can re-test or add another lens and get results that are easy to comparable.

BTW I am in the process of comparing a bunch of tube lenses and a pile of scanner lenses, I think my brain now has a wafer disk image permanently etched into my prefrontal cortex Very Happy

All the best,
Robert
Back to top
View user's profile Send private message Visit poster's website
rjlittlefield
Site Admin


Joined: 01 Aug 2006
Posts: 19913
Location: Richland, Washington State, USA

PostPosted: Wed May 30, 2018 8:48 pm    Post subject: Reply with quote

RobertOToole wrote:
You can use a menu command to load all the open images into a single file as layers. Photoshop can then align the images.

If you're talking about Edit > Auto-Align Layers, then suddenly I have become very uncomfortable due to the sub-pixel interpolation issue discussed above.

But probably I have misunderstood what you're talking about. Can you explain in more detail?

--Rik
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    www.photomacrography.net Forum Index -> Equipment Discussions All times are GMT - 7 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group