How do you compare lenses?

[saffersteve]

To help me learn a little about how to compare lens I would like to put the following question to you. (feel free to hyperlink previous topics)

If you were given a number of 50mm lens that were unbranded.
How would you compare them to select the best?

Here are some items that may help with choosing the first cull.

-speed of lens
-weight
-macro
-type of lens grouping (Im putting this in but know nothing about this)
-coating/s
-build quality
-blades
-f-stops

Next when taking pictures with the chosen few. How would you rate.

-colour
-bokeh
-overall sharpness
-optics
-user friendly
-etc

What tests would you do.

-just go out and shoot
-optical chart
-dark room/ bright sun?
-etc

[enzodm]

This is a very nice review by Orio: http://forum.mflenses.com/voigtlaender-septon-2-50-for-bessamatic-review-t33501,highlight,%2Bsepton.html , where you can find most of the interesting features of a lens tested in real world examples.

[WolverineX]

[saffersteve]

[saffersteve]

[martinsmith99]

[WolverineX]

[poilu]

read this article to understand value of lens test

http://blogs.zeiss.com/photo/en/wp-content/uploads/2011/07/en_CLB_40_Nasse_Lens_Names_Planar.pdf

[lauge]

[kuuan]

[kds315*]

An old rule is that if the tester and the test procedure is not much better than the object tested (a lens in that case), the results are not worth the effort nor can they be objective!

Therefore I would agree to simply shoot known situations and objects in a somewhat comparable manner and learn from that how lenses behave differently.

A few people may have noticed that when I present lenses here, that I apply that method. I'm too much engineer not to know my weaknesses well, that's why...

[Big Dawg]

[RioRico]

[visualopsins]

[Lloydy]

I recently bought a 'classic' Vivitar Series 1 70-210 f3.5, and after a few days I realised that I'd also got a Vivitar 80-200 f4 that I'd had for a long time and rarely used.
They are different lenses in many respects but they are both from the same retailer, but different manufacturer ( Series 1- Kiron, the other is Kobori )
But they are both are fixed aperture zooms of approximately the same range, they had enough in common that I was curious to see what the difference was, after all, there was a huge difference in price when they were new.

So I tested them back to back, and the 'ordinary' Vivitar won hands down, it was way better than the Series1.

Later that day I took them both out and shot pictures as I would normally but changed lenses and shot twice as many pictures as I normally would.
The Series1 won hands down.

Lens testing as a comparison between lenses is difficult and has to be very carefully controlled.
And ultimately it's going to become a test of individual taste and preference in so many aspects of the way we view the images.
The most accurate tests are of charts in controlled lighting, but I don't shoot that kind of picture. I'm with WolverineX here, and RioRico, my test is, does the image on my screen make me go "wow" ?

[visualopsins]

Also if results are not so good, I put in line to try again another day; sometimes 3 or 4 times do that, before give up say 'I cannot get any good result.' Some lenses I need to learn how to use to best advantage before any good results. Then there are lenses even bad photos look exceptional good, have WOW!

[Orio]

Thanks Enzo for mentioning my Septon review

I think that evaluating a lens (which is different from testing - in my opinion the test word should be reserved to experimentation lead in strictly scientifical ways)
is necessarily the result of the interaction between an individual (me, you, every one of us) with an object.
Therefore, it contains an irrepressible element of subjectivity.
The key in my opinion is to find a balance between each user's individual inclinations and some key objective features of the lens.
This means that the subjective element (how does a lens feel in your hands, how do the focusing or aperture control feel for you, how do you like the bokeh, and so on)
need not to overcome some objective ascertainment (how sharp is the lens, how well does it control flare or CA, how much it distorts, does it shift focus or not, and so on)

The two most common mistakes that I find in people's "tests" of lenses are the following:

1- they set up a quasi scientific test (typical: bookshelf test) but "along the way" they forget some fundamentals of scientific testing, such as:
metering light consistently, balancing colours consistently, taking all shots in the same position/condition, etc.
So you often see those tests with included excuses ("here's my four hours of work test, but sorry, I left AWB on, so you can not really evaluate the colour tint of the lenses,
and oops I did leave automatic exposure on, so exposure happens to not be consistent, and yes in take #4 I did use a too slow shutter time so please excuse the motion blur, and,
yes of course I misfocused the lens X shot, so you can not really compare sharpness with lens Z, and damn, I actually did focus take #6 farther, so the bokeh and CA are not really comparable with the rest...
When I read these I am always reminded of the time when I did teach Latin in Junior High, and the kids did deliver their written works with included excuses such as
"I had to rush out yesterday with my mom" or "my little sister was a pain in the neck I could not concentrate" or "I had that big math homework to make"
The point being: want to try yourself at a scientifical lens test? Ok, do it, but do it well. Make a list of things to go through. Make sure you don't forget about things. Make sure you are consistent in everything.
Else, if you have to do it extemporarily, then better not do it at all. It would come out faulty and a waste of your precious time.

2- some reviewers tend to concentrate their attention to one or two aspects only, and forget the rest.
Typical example, a reviewer makes a lot of tests about sharpness, but does not test CA - or, a reviewer is concerned a lot about bokeh, but does not test flare resistance.

Finally, my advice is that it's always best to evaluate lenses in real use conditions - stay clear of bookshelf or liquor bottle "tests".

[Yebisu]

I mainly use Nikkors so none of my lenses cost enough for me to worry about testing or evaluating them. I like it that way. If I bought an expensive lens, I know I'd worry about how well it performed and whether it was worth the money.

As for evaluating them, I just slap them on the camera and start shooting. If I think the results are nice, then that's good enough for me.

I also agree with Orio about the subjective evaluation of a lens. How easy it is for you to focus, how well it balances on your camera etc. I love the pictures I get with my Nikkor Q 135/2.8 but it feels huge (and heavy) on my little D3000 so I don't use it often.

[jjphoto]

Doesn't it all depend on your reason for testing or comparing lenses. Decide what your reasons are, what you want from the lenses and then test accordingly.

There's no such thing as a 'best' lens, only one best suited for a specific application.

Don't bother testing for things that don't matter to you.

JJ

[saffersteve]

Guys.....thanks for your contributions keep them comming, there is some really good stuff here for people new to lens.

One thing I'd say is, I/we are fortunate that we even find ourselves in the position to be able to compare lens in such large numbers.

I guess for most people it's a case of save for sometime then buy based on maket review/sales hype.

Newbies don't really have a place to learn this stuff and test for themselves.

Just one point I want to comment on.

[Pancolart]

There are two major approaches to lens testing: individual and standardized. Since it's much simpler and possibly more fun to do, the first flourishes on this forum and elsewhere too. When i was Pentaxian we managed to unify camera test settings and other conditions and use them on very similar objects like car registration plates. It was soo cool seeing Lens as variable being extracted from many.

Though all those problems Orio mentioned still occurred (and many others) the results were really interesting.

Point i'd like to make: standardization of testing procedure really makes sense. Of course this is something contrary to individual approach (like Wolverine).

Many on this forum use 5DmkII as main camera. If we could agree on standardized testing procedure (and settings)...

[Orio]

Good remarks Steve and Pancolart.
I guess that my concept can be summarized in a shorter way that I did above. Basically I think that tests as usually meant are useless, because they would require precision tools that most of us don't have, and because most of us make mistakes that undermine the validity of those tests.
Many times I read comments "look at that test, it shows that lens x is sharper than lens y". Only to find out that lens x was badly focused and the whole test was crap. It happened so many times.
Another factor is that most home made tests use one copy of a given lens, often used, of unknown whereabouts. Al least five copies should be used per every type of lens, and they should be new or at least same age and same usage level.
In other words it's impossible to make really useful tests on old often battered lenses and without precision tools.

Much better to make reviews that render your own personal impression of that given copy of a given lens, without the presumption of saying final words or etching facts on stone.
Of course, the review should be as global as possible and not focus only on one or two aspects of a lens.

[revers]

Being a sharpness nut, I shoot a back-lit test chart to compare lenses. Last weekend I won a Canon FD 50/1.4 on eBay for $44 shipping included. I will be comparing it to my Minolta 50/1.4 (which I like a lot) when it comes. Minolta 58/1.4s have so far eluded me.

[walter g]

[s58y]

My only real lens "tests" are done outside, at night, with real stars. Astroimaging often brings out the worst in a lens, since stopping the lens down to f/5.6 or f/8 to get the best results is not usually acceptable, and there is extreme contrast.

The most important factors are CA, field flatness, and off-axis aberrations. (CA can be sometimes ignored for narrowband imaging, however.)

It's nice if the lens has a stiff focus ring, which won't move by itself when the lens is pointing straight up, with filters, lenshood, and dew heaters attached.

Much less important is geometric distortion and cool/warm color. Bokeh and closeup performance are irrelevant. In fact, a lens with a long close focusing distance is good, since the helicoid may have a finer pitch, and manual focusing may be easier.