Home

Please support mflenses.com if you need any graphic related work order it from us, click on above banner to order!

SearchSearch MemberlistMemberlist RegisterRegister ProfileProfile Log in to check your private messagesLog in to check your private messages Log inLog in

Suggestions for comparative testing of bokeh quality
View previous topic :: View next topic  


PostPosted: Thu Aug 13, 2009 8:28 pm    Post subject: Suggestions for comparative testing of bokeh quality Reply with quote

Hi,

I've been thinking of putting together a comparison of my various M42 lenses, especially the many different 50mm's and 135mm's. The part where I'm having most difficulty deciding how to proceed is the testing of bokeh. Obviously it's very subjective and even depends on the scene (e.g. sometimes a “swirly” bokeh creates a nice special effect, sometimes it's very distracting). However, I'd like to come up with a series of “scenes”, which would then all be photographed with each lens, that could be used to form a gallery illustrating the bokeh of each lens in various different situations.

Since each test scene would need to be shot with a couple of dozen lenses, it would need to be fairly stable so that it doesn't change between shots. This probably means artificially lit indoor scenes, which would also eliminate the major difficulty of transporting all lenses at once, but it becomes more difficult to come up with suitable tests scenarios because of limited distances. Perhaps these indoor “lab tests” could be augmented with a selection of similar (but non-identical) outdoor shots, e.g. “forest background with sun shining through the trees”, but the primary goal would be to come up with the indoor scenes I could set up.

So, any ideas on what kind of scenes to set up to obtain the most useful comparison? I mean useful in the sense that it would, for example, help someone decide between two otherwise similar lenses.

So far I've been thinking of something along these lines:

– Small objects close behind the point of focus, e.g. office supplies, perhaps a computer keyboard, etc.

– Glasses and/or glass bottles at different distances behind the point of focus, lit so as to produce some bright highlights. (Should the background be dark or light?)

– The keyboard of a piano with the focus on one key in the middle. (Perhaps too dull?)

- Close-up of a flower or a plant? (Needs extension tubes to work with most lenses, so perhaps unsuitable for a “natural” comparison? Also, close-ups often seem to give a false impression of high quality bokeh, or is it just my imagination?)


Edit: If you have done similar comparisons or tests with your lenses, please post or link to them! (Edit 2: By similar, I mean something which could be re-created and used for comparison; there's an excellent thread already on this forum for showcasing bokeh, which I have read through. =)


PostPosted: Thu Aug 13, 2009 8:50 pm    Post subject: Reply with quote

Hard question I like smooth green background for bokeh tests some people don't so really mater of personal taste.


PostPosted: Thu Aug 13, 2009 9:13 pm    Post subject: Reply with quote

Attila wrote:
...... some people don't so really mater of personal taste.


I'm one. Not matter to me the lenses bokeh. Prefer sharpness and color.

But if have to choose one, Glasses and/or glass bottles at different distances behind the point of focus, lit so as to produce some bright highlights.

Rino.


PostPosted: Thu Aug 13, 2009 9:27 pm    Post subject: Reply with quote

Relayer wrote:
hi, some of my bokeh tests
http://forum.mflenses.com/big-test-of-1-4-50-super-takumar-nikkor-zuiko-rokkor-x-t15694.html


Thanks! Using Christmas lights is a very good idea for showing the “plain” circles of confusion, similar to city lights at night, I think I'll copy that as one of the tests. =)


PostPosted: Thu Aug 13, 2009 9:34 pm    Post subject: Reply with quote

estudleon wrote:
I'm one. Not matter to me the lenses bokeh. Prefer sharpness and color.


I would test other characteristics of the lenses as well, but sharpness, distortions, etc are easier since they can (mostly) be measured objectively. In any case, I think I can come up with the suitable tests for these things myself, but bokeh is more difficult because it is subjective and and there are so many different ways to show it (front, back, distance, aperture, brightness, colour, etc in all combinations).

So, the this question is more along the lines of “if all the other tests were already done, and assuming you care about bokeh, how would you like to see it tested so that it would help you choose between lenses that are otherwise equally good to you”. =)


(As for my own opinion, I care very little about sharpness and mild distortions. Bokeh and colour are at the top of my list when considering a lens.)


PostPosted: Thu Aug 13, 2009 10:10 pm    Post subject: Reply with quote

Why do you need to 'test' it? I mean, isn't it better to simply go and shoot your usual subjects, and then see whether you like the results? Bokeh is never 'good' or 'bad'. It depends widely on shooting conditions, background, and personal taste of the photographer / viewers. I for one have a lense with extremely smooth bokeh (Samyang 85/1.4), and find that portraits I shoot with it sometimes lack the 'punch', but sometimes (depending on the background and subject to background distance) find it very pleasing. I have lenses with harsher bokeh (such as Zeiss 85/1.4), and find them perfectly usable in one situation yet a loss in others.

Having too many lenses does not help either. Isn't it better to know your gear well and what to expect of it in various conditions?


PostPosted: Thu Aug 13, 2009 10:17 pm    Post subject: Reply with quote

aoleg wrote:
Why do you need to 'test' it? I mean, isn't it better to simply go and shoot your usual subjects, and then see whether you like the results? Bokeh is never 'good' or 'bad'. It depends widely on shooting conditions, background, and personal taste of the photographer / viewers. I for one have a lense with extremely smooth bokeh (Samyang 85/1.4), and find that portraits I shoot with it sometimes lack the 'punch', but sometimes (depending on the background and subject to background distance) find it very pleasing. I have lenses with harsher bokeh (such as Zeiss 85/1.4), and find them perfectly usable in one situation yet a loss in others.

Having too many lenses does not help either. Isn't it better to know your gear well and what to expect of it in various conditions?


Good point ! I never made any comparative test too much work and boring too see them this is my point of view.


PostPosted: Thu Aug 13, 2009 10:31 pm    Post subject: Reply with quote

btw. I noticed, that many modern DSLR cameras do an automatic boost of microcontrast to make the pictures look better, but it quite harms bokeh Confused


PostPosted: Thu Aug 13, 2009 11:04 pm    Post subject: Reply with quote

aoleg wrote:
Why do you need to 'test' it? I mean, isn't it better to simply go and shoot your usual subjects, and then see whether you like the results?


Yes, it is better to do that, and that's what I've been doing for my own pleasure. One reason I have so many inexpensive lenses is that discovering their various properties and quirks inspires me to take more photos than I probably would if I just had one “universal” lens for each use. =)

So, I don't “need” to test these lenses for myself, but I think it would be very interesting to see a “full” comparison of all the lenses I have, tested under the same, controlled circumstances. (I am a scientist, does it show? =)

For myself, I'm thinking such a comparison might reveal interesting details I haven't otherwise noticed, e.g. because they might not be something I normally pay attention to and would only notice when I see them side by side with other pictures. And perhaps for the “greater good”, posting such a large comparison might serve as a guide for people deciding which lens to buy (while they're still at the stage where they believe one is enough ;>). I do have most of the popular M42-mount normal lenses and several of the well-liked 135mm's, so there might be general interest in this “shootout” (of which bokeh would be just one part, but that's the one I'm having most difficulty with, hence this post).


aoleg wrote:
Bokeh is never 'good' or 'bad'. It depends widely on shooting conditions, background, and personal taste of the photographer / viewers.


Yes! This is the exact reason why I posted the question here; if I just rate the bokeh of the lenses myself, it doesn't say anything for anyone else. But if we can come up with a reasonable set of test scenarios and I post the bokeh samples from each lens for everyone to see, everyone who is interested can decide for themselves. But because it depends on the shooting conditions and background, it's difficult to come up with a “definitive” set of tests that would be informative for “everyone”. Again, hence the question.


Attila wrote:
Good point ! I never made any comparative test too much work and boring too see them this is my point of view.


It is a lot of work, yes, that's one reason why I've been toying it the idea for years but haven't really committed to actually doing it. =)

Still, I totally disagree about them being boring to see. I enjoyed very much going through the “best bokeh” thread in this forum, and I think it would be awesome to be able to see some of the samples reproduced with another (interesting) lens.

Also, having the same scene shot with each lens offers interesting possibilities like “blind” comparisons (i.e. rate photos shown by computer without seeing the names of lenses, find out which you liked the best—this would be a very simple program to write as an online “poll”).


aoleg wrote:
I for one have a lense with extremely smooth bokeh (Samyang 85/1.4), and find that portraits I shoot with it sometimes lack the 'punch', but sometimes (depending on the background and subject to background distance) find it very pleasing. I have lenses with harsher bokeh (such as Zeiss 85/1.4), and find them perfectly usable in one situation yet a loss in others.


I, too, have the Samyang 85mm f/1.4 and one important reason I parted with the £133 I paid for mine (which is more than I'm usually willing to pay for a lens) was the bokeh. I consider this an example of the usefulness of being able to compare bokeh, although admittedly I would not have considered the Zeiss as an alternative due to its price, and I did not have a side-by-side comparison of the lenses. Still, I think it would have been very interesting to see such a side-by-side comparison of the bokeh of these two lenses… Don't you?


PostPosted: Thu Aug 13, 2009 11:06 pm    Post subject: Reply with quote

no-X wrote:
btw. I noticed, that many modern DSLR cameras do an automatic boost of microcontrast to make the pictures look better, but it quite harms bokeh :?


Interesting; I have not encountered this myself (or at least noticed it). Do they do it even when shooting raw? One would hope not.


PostPosted: Thu Aug 13, 2009 11:55 pm    Post subject: Reply with quote

measuring the brightness as you go outward from the center of a defocused distant (or extremely close) point light source.

most modern lenses are overcorrected for distant OOF point sources, so there's a hard edge to their OOF highlights which is easy to measure.


PostPosted: Fri Aug 14, 2009 12:03 am    Post subject: Reply with quote

orly_andico wrote:
measuring the brightness as you go outward from the center of a defocused distant (or extremely close) point light source.


The problem with this measurement is that the result doesn't say anything about the subjective quality. It would reveal the extreme cases like a mirror lens with central obstruction vs Minolta STF, but everything in between depends too much on the circumstances and personal opinion. (And indeed, in some cases the bokeh of a mirror lens may be interesting as an effect, and some strange people think the Minolta STF is “too smooth”… =)


PostPosted: Fri Aug 14, 2009 12:12 am    Post subject: Reply with quote

To offer you a little guidance, I would suppose the rendering of highlights is the most contentious area of bokeh judging. In my experience, most people believe these make or break a photo and most photographers can't see past them (or in front of them more correctly). So that would be something to test. Other types of tests seem less than worthwhile: Both renderings are smooth, but which is SMOOTHER? Slightly better oof areas shouldn't make a photo. Bokeh isn't a subject. If it was, set your lens wide open, deliberately mis-focus, and click away.

That being said, I understand your intentions and your desire to approach this scientifically perfectly. I'm as guilty as everyone in searching for 'better' lenses (meaning they fit my criterion as 'better'). In this case I would encourage you, as aoleg had, to test your lenses through your shooting. Time would be better spent working on your skills as a photographer (this is not meant as an insult in any sense - I'm confident you're a better photographer than I) and seeking out better subjects than a constructing a test that couldn't possibly encompass the infinite amount of circumstances that may arise.


PostPosted: Fri Aug 14, 2009 12:48 am    Post subject: Reply with quote

themoleman342 wrote:
Other types of tests seem less than worthwhile: Both renderings are smooth, but which is SMOOTHER? Slightly better oof areas shouldn't make a photo. Bokeh isn't a subject. If it was, set your lens wide open, deliberately mis-focus, and click away.


I don't think the subjective quality of bokeh is just about the smoothness. Some people deliberately want “visible” bokeh for some shots, and it can be a subject, or at least an important element (e.g. a frame) in a photo. With this in mind, I think it would be helpful to illustrate exactly how the each lens copes with a given situation, not just to determine which is the “best” but also to see what kind of “effects” it can produce. For example, some lenses produce a “swirly” bokeh that forms kind of a circle around the centre of focus, and this is sometimes an effect that is sought after even though it's not smooth at all. (I've heard of some people deliberately buying a lens just for that effect, and others complaining of “sea sickness” from looking at such a photo. =)


themoleman342 wrote:
That being said, I understand your intentions and your desire to approach this scientifically perfectly. I'm as guilty as everyone in searching for 'better' lenses (meaning they fit my criterion as 'better'). In this case I would encourage you, as aoleg had, to test your lenses through your shooting.


I think one thing that seems to be misunderstood about my goal is that I'm not really seeking to test these lenses for myself, and I'm certainly not thinking of doing a systematic test instead of using them “regularly”. (Actually I try to make a point of taking a different lens with me every day and taking a couple of pictures with it whenever I have the time, e.g. on my way to work, etc.)

My primary intent is to put together a comprehensive test and a gallery of each lens, including the typical tests of sharpness etc, “real” shots that I've taken with the lenses, and also a systematic test of bokeh. While I personally find it interesting as an experiment, the major befit of this would probably be as a “buyer's guide” to several popular M42 lenses (as I do have most of the popular/common normal lenses in that mount). And like I've said before, this bokeh gallery/comparison/whatever would just be one part of it, but that's the part I'm having most difficulty with, and that's why I'm asking to see if anyone else has thought about testing it systematically.

Unfortunately the discussion now seems to be more about the sensibility of the test itself, so perhaps it helps if I say that I'll do the test anyhow, but let's say that if I happen to have a lens you (that is, the readership in general) consider interesting, what kind of a test would you like me to perform with it to help you decide about its bokeh (assuming that you care about such things)?


PostPosted: Fri Aug 14, 2009 5:36 am    Post subject: Reply with quote

http://www.rickdenney.com/bokeh_test.htm is a nice link - interesting (considering the nature of this forum) lenses tested.

Still, it is somewhat subjective - I kind of like this bokeh too:



PostPosted: Fri Aug 14, 2009 6:18 am    Post subject: Reply with quote

Arkku wrote:
Unfortunately the discussion now seems to be more about the sensibility of the test itself, so perhaps it helps if I say that I'll do the test anyhow, but let's say that if I happen to have a lens you (that is, the readership in general) consider interesting, what kind of a test would you like me to perform with it to help you decide about its bokeh (assuming that you care about such things)?


For 50-200mm lenses, I'd prefer portraits of beutiful female models (different models for each test are welcome!) shot with a set of backgrounds such as: trees (green leaves are a tough subject), crop field (wheat works great), bushes (branches are, again, a tough bokeh subject). The distance should be varied: facial portrait, half body length, full body, and maybe a smaller scale (this depends on the lens of course).

For long lenses (300mm and longer), wildlife shots with leaves and branches; shots of a dog/cat on the grass.

For wider angle (35mm and wider)... well, I just don't think bokeh is important here except for synthetic tests. You cay consider some low-light pictures with point light sources in background.

Like this: http://club.foto.ru/forum/view_topic.php?topic_id=261862


PostPosted: Fri Aug 14, 2009 6:33 am    Post subject: Reply with quote

aoleg wrote:
I'd prefer portraits of beutiful female models (different models for each test are welcome!)

me2!


PostPosted: Fri Aug 14, 2009 8:29 am    Post subject: Reply with quote

Anu wrote:
http://www.rickdenney.com/bokeh_test.htm is a nice link - interesting (considering the nature of this forum) lenses tested.


Thank you, I had forgotten about that test, although I had seen it a couple of years ago. That's exactly the kind of test setup I've been considering.

Anu wrote:
Still, it is somewhat subjective - I kind of like this bokeh too:


Exactly why I wish to put together a gallery of bokeh examples with different kinds of scenes photographed with each lens instead of just proclaiming my own opinion about the bokeh… Opinions differ and in some scenes the “bad” bokeh can create an interesting effect. I hope the test would not only show which has the smoothest bokeh, but reveal new ideas on what kind of “effects” can be produced with the bokeh of each lens.


PostPosted: Fri Aug 14, 2009 8:36 am    Post subject: Reply with quote

aoleg wrote:
shot with a set of backgrounds such as: trees (green leaves are a tough subject), crop field (wheat works great), bushes (branches are, again, a tough bokeh subject).


Good ideas, although I'm still hoping I could simulate “similar bokeh” with indoor objects, because of the difficulty of transporting several dozen lenses to each location and the varying lighting outdoors (possibly leading to very different highlights). However, I may be aiming to be too scientific, and may be that I'll supplement the test with some shots of similar (but not identical) outdoor scenes.


aoleg wrote:
For 50-200mm lenses


The main emphasis would be on these lenses, probably in three categories: 50-58mm's, 85-105mm's and 135mm's, since those are the lenses I've have enough of to make an interesting comparison. =)