How to Read our Image Stabilization Test Results
There's no commonly-accepted way of describing the detailed behavior of image stabilization (IS) systems, so we at SLRgear.com have had to come up with our own. We have a more technical article here which explains in greater detail how we go about measuring and analyzing IS performance; but in this article, we just want to describe how to read and interpret the graphs we use to present our findings, and how to translate those graphs into real-world results. (If you have questions not answered here or in the Image Stabilization Testing White Paper, we invite you to ask them in the IS Testing discussion forum thread.)
IS performance is such a complex phenomena that there's really no practical way to characterize system performance for all possible combinations of camera, lens, shooter, and shutter speed. What we do instead is try to measure and describe performance for conditions that should simulate the experience of most of our readers. That is, we test system performance when being used by a rather steady and a rather shaky shooter, and in the case of zoom lenses or cameras with body-based stabilization, we test at both shorter and longer focal lengths.
realize that many readers just want the basic numbers, so we make those
available at the very top of each IS review page, in the form of tables
showing the performance our "steady" and "shaky" testers measured.
Here's an example:
Canon 70-200mm F/4L IS, at 70mm
(Slowest average shutter speeds for motion blur of 0.5)
Canon 70-200mm F/4L IS, at 200mm
(Slowest average shutter speeds for motion blur of 0.5)
Because the performance of IS systems can vary quite a bit depending on how steadily a person can hold the camera, we always show results for both "steady" and "shaky" shooters.
The IS Off and IS On numbers in these tables show the slowest shutter speed that each tester could shoot at, while getting an average shake-produced blur of 0.5 blur units. (See below for what a "blur unit" looks like.) The difference between the lowest usable shutter speeds with IS On versus IS Off is the improvement, which we translate into stops of improvement in the last column.
You can get an idea of what you might be able to expect from an IS system by comparing the slowest speed you can hand-hold a lens of the same focal length with the slowest speeds each of our testers was able to use with the IS turned off. If your hand-holding experience matches that of our Shaky tester, then your results with the IS stem in question will probably parallel his. Likewise if your performance matches our Steady tester.
For reference, the 1/FL column shows what the "1 divided by the focal length" rule of thumb shutter speed is for the lens and camera in question. (Calculated based on the effective focal length of the lens on the body it was being tested with.) In the example above, our "shaky" shooter's IS-off results followed the 1/FL rule pretty closely, while our "steady" shooter turned in a performance that was quite a bit better than that benchmark. As seems to frequently be the case, the IS system in the lens therefore didn't help the Steady shooter quite as much, but he was nonetheless able to shoot at dramatically lower shutter speeds with the IS enabled than he was without its help.
These tables will probably be all that 90 percent of our readers want to know about our IS test results: They give a good idea of what to expect from a system, based on how steadily you can hold a camera. We do have many readers who want to know more of the technical details, though, so we've prepared more in-depth graphs for them. If you're part of the 90 percent, you can stop reading here and go back to the IS review you were just reading. For the remaining 10 percent, read on below.
those wanting a little more detail on how an IS system tested-out, the
graphs you'll see in the reviews reveal more of our findings, including
data points for each individual shot captured during our testing.
This graph is
the one we use to summarize our test results over the photographically
relevant range of shake-induced blur. The two curves show the average
increase in blur that resulted from camera shake, plotted as a function
of shutter speed. (Note that this isn't lens sharpness, this is just
the increase in blur due to camera shake.) The red curve
shows the blur we found with the IS turned off, the blue curve shows
the blur with the IS system turned on. The dashed vertical line marks
the shutter speed that corresponds to 1 divided by the effective focal
length of the lens being tested, a common rule of thumb for the slowest
shutter speed that can be routinely hand-held. The two green arrows
show how much the IS system improved the blur results, depending on how
much blur you can tolerate for your application. The upper blur
threshold of 1 blur unit would apply if you're only making relatively
small prints from your pictures, perhaps a maximum of 5x7 inches or so.
The lower threshold of 0.5 blur unit is a more generally-acceptable
standard, resulting in images useful for making larger output. When
comparing systems to each other, we use the more stringent criteria,
requiring that shake-induced blur be less than 0.5 blur unit.
Details: How Many Shots were Sharp or Blurry?
It's important to note that these graphs show average
blur performance. Shooting at moderately slow shutter speeds, you'll
generally find that some shots are blurry, while others are relatively
sharp. What IS does is shift this balance in your favor, resulting in a
better percentage of sharp shots than you'd get on your own. We
experienced this in our testing, and to help you know what to expect in
your own shooting, we show you how many of our own test shots were
sharp or blurry. If you roll your cursor over the summary graphs in our
reviews, you'll see a number of points appear, against a colored
background. The image below shows the colorized version of the graph
above, without the rollover, to make it easier to examine as you read
point on this colored graph represents a single test shot: The
left/right position shows the shutter speed the image was shot at, and
the height shows how sharp or soft the test image was. Low dots
correspond to sharp images, high dots to blurry ones. To help you
interpret the results, the shaded background shows levels of blur that
are generally acceptable (green), marginal (yellow) or unacceptable
(red). (Note for the data geeks: This view offsets the data points
slightly from the plotted curves, to simplify the presentation of IS
improvement. Read our Image Stabilization White Paper for an explanation of this.)
This view makes clear how much variation there is from shot to shot. In the example shown above, our tester managed to get some very sharp images with the IS system turned off, even at a shutter speed as slow as 1/60 second. (The red dot that's almost on the baseline, just before the 1/50 second mark.) At the same time, even with the IS system engaged, he still got one or two relatively soft shots at shutter speeds as high as 1/120 second. (The blue dots that just edge into the yellow area, just to the left of the 1/100 second shot.) It's easy to see, though, that the IS system was a considerable help, in that the majority of shots from 1/200 down to about 1/30 second were acceptably sharp when it was engaged, vs. only two or three sharp shots in that range when the IS system was turned off.
More Gory Details: The "All Shots" Graph
graphs just shown display our results with IS On and IS Off across a
range of sharpness that's photographically useful. For those readers
who want to see all the data we captured, we provide links to larger
graphs, that show the full range of our experimental results.
this graph, there's no offset between the plotted curves and the data
points, and the data values extend to much higher levels of blur. The
data at high blur values isn't too relevant to day to day photography,
since the images there are so blurred. In the interest of full
disclosure, though, we provide links to these graphs for people who may
be interested in how the IS systems handle very long exposure times. In
our IS reviews, we offer these graphs as links only, so they won't
clutter the pages for the average reader.
Still want to know more?
If the results tables alone met the needs of 90 percent of our readers, the tables plus the graphs just described are all that probably 99 percent of people care to know. For the remaining one percent, we offer our Image Stabilization Testing White Paper, which explains our IS testing methods and data analysis techniques in full detail.