SLRGear.com: Interactive Lens Reviews and Opinion.
Digital Camera ReviewsDiscuss lenses in the SLRgear.com Forums (separate login)


Important points to consider in interpreting our test results

Test results are difficult to compare between different camera platforms
By its nature, DxO Analyzer is measuring the end results of the entire optical/image processing chain. The camera and its settings are therefore an integral part of the overall measurement. Different cameras and different settings will produce different BxU numbers! This means that the BxU numbers we report are NOT absolutes, but must be taken as relative values, a product not only of the lens in question, but of the specific camera and settings used to capture the test images as well.

Because of the relative nature of the DxO blur measurements, you can't directly equate results obtained on different camera platforms. By careful choice of the camera settings and the assignment of a relative scaling factor, we've brought the blur numbers for the different camera platforms we use into reasonably close agreement. We have no adequate way to precisely calibrate the BxU numbers between platforms though.

We arrived at the multipliers by looking at best-case performance with the same lens on the different camera platforms. The scaling factors were set to make this best-case performance roughly the same between the cameras in question. As this is written in early October, 2005, our test platforms are the Canon EOS-20D and the Nikon D2x, although we expect to be adding a Canon 5D to our "stable" of test platforms in the not too distant future, and will hopefully begin testing with Olympus and Konica Minolta cameras soon as well. This means that numbers within a BxU unit or so of each other between the two platforms have to be considered as nearly equivalent. That is, it's meaningless to say (for instance) that a particular Canon lens tested on the EOS-20D and showing a best-case BxU value of 1.5 is "better" than a Nikon lens tested on the D2x body and showing a best-case BxU value of 1.8. For this reason, we show only graphical results rather than numeric ones, and actively discourage people from making quantitative comparisons between lenses based on our data.

What can you tell from our tests? - An example:
So if exact comparisons aren't possible, what sorts of things can you tell from our tests? - In practice, it turns out that a difference of one BxU unit or so is quite a bit less than the differences in sharpness caused by even relatively minor changes in settings for in-camera sharpening or by surprisingly minute focus errors.

Fortunately, the loss of sharpness from typical optical defects and design shortcomings is glaringly obvious, amounting to much more than just one or two BxU units. Also, what really separates one lens from another is not so much how sharp they each are in their respective "sweet spots" (the best-case combinations of aperture and focal length), but rather how well they maintain optimal or near-optimal performance across their focal length and aperture ranges.

Lets take a look at a case in point, comparing a relatively expensive, high-quality lens (the Nikkor 17-55mm f/2.8 constant aperture) to a less expensive lens by the same manufacturer (the Nikkor 18-55mm f/3.5-f/5.6)

Nikkor 18-55mm f/3.5-5.6
at 18mm, f/3.5
Nikkor 17-55mm f/2.8
at 17mm, f/2.8

The worst case for most lenses is wide open, at maximum wide angle. This situation is shown above for the two lenses in question. In the case of the inexpensive lens, while it's actually very sharp in the center of the frame, note how dramatically the blur increases in the corners. This is very evident visually in photos shot with it at this combination of focal length and aperture. In contrast, observe how much more uniform the blur is in the case of the 17-55mm f/2.8 optic. There's a very slight rise in the plot along the left edge, and the corners are slightly softer than the center, but the average sharpness across the frame is significantly more uniform.

Nikkor 18-55mm f/3.5-5.6
at 55mm, f/5.6
Nikkor 17-55mm f/2.8
at 55mm, f/2.8

Here's another example, comparing these same two lenses. Again, the 18-55mm f/3.5-5.6 is on the left, the 17-55 f/2.8 is on the right. In these shots, we're looking at blur performance with each lens set to its maximum telephoto position of 55 mm, and again both are being shot wide open. Notice here how the 17-55mm's blur is uniformly low across the frame, while the 18-55mm has a pronounced "hill" on the right side of the frame.

Nikkor 18-55mm f/3.5-5.6
at 55mm, f/5.6
Nikkor 17-55mm f/2.8
at 55mm, f/5.6

Also, consider that "wide open" for the two lenses at this focal length means f/2.8 for the 17-55mm, but only f/5.6 for the 18-55mm. To make the comparison a little more fair, take a look at the blur plot above for the 17-55mm at f/5.6. An aperture that's at the limit of the 18-55mm's performance curve is solidly within the "sweet spot" of the more expensive 17-55mm optic.

Nikkor 18-55mm f/3.5-5.6
at 18mm, f/8
Nikkor 17-55mm f/2.8
at 17mm, f/8

But lest one completely dismiss low-cost lenses out of hand, take a look at the two plots above. Here, we're again looking at the performance of the same two lenses at wide angle, but this time with both stopped down to f/8. When operating within its "sweet spot," we can see that the 18-55mm actually comes very, very close to the performance of the more costly 17-55mm, at least in terms of image sharpness. (Chromatic aberration might be another matter entirely, though.)

The examples above perfectly illustrate the sort of comparisons between lenses that our data allows. While absolute, numerical comparisons can't be made across different camera platforms, the overall behavior of lenses and their individual quirks and oddities are dramatically evident.



 





This document copyright ©2014, SLRgear.com, all rights reserved.