Tuesday, July 31, 2007

How to Test Autofocus

This post ended up so large, I split it between entries. The follow up entry (much delayed) is here.

As I mentioned in my 20D review, I am concerned that the auto-focus in my 20D is out of calibration. So I decided to test it... at least somewhat scientifically. If you are one of those people that likes to read the last page of the book before the first, I'll give you the end before the beginning:

My 20D's autofocus is fine, or at least not worth sending to Canon to 'fix'.

But, you may be asking, how do I figure out if my autofocus is working correctly?

How Autofocus Works:

This discussion only really relates to SLRs, specifically Canon digital SLRs. Most fixed-lens point and shoots use an active autofocus system (for more information, see here).

Canon dSLRs use a passive, TTL (through the lens), optical system. In other words, your autofocus judges focus much like you do: it looks through the lens and determines when the image looks the sharpest. But unlike a person, who will keep rotating the focus wheel until it looks sharp, Canon's system can calculate where focus should be after one reading, then commands the lens to move to a position which should give the correct focus. The gory details are in Gordon McKinney's autofocus post. There's also an interesting discussion about the 30D autofocus in this DPC thread. And here are two more threads with lots of good info.

To compute focus, Canon has placed special sensors on the bottom of the mirror box (using a secondary mirror underneath the half-silvered main mirror). These sensors can measure the focus as long as there are high contrast regions in the image projected onto the sensor. In other words, if you point your camera at a blank wall, the sky, or a polar bear in a blizzard, there will be no contrast and the system will be unable to find focus (instead, it will hunt through the full range of focus hoping to find something it can use). Also, the focus regions are small strips of the viewfinder area, so it is possible to 'miss' the object you are trying to focus on (resulting in a confused camera).

There are three take-home messages about the 20D (and 30D) autofocus:
  • Autofocus is not perfect. In fact, much of the time you'll be lucky to get the object used to focus within the range necessary for a good image. Because Canon cameras measure the 'focus' and then command the lens to move open-loop, it isn't hard to be off in focus, especially since there is some inaccuracy built in. A feedback loop around focus would be great, but for some reason Canon didn't do that (probably too hard given the range of lenses and focusing systems).
  • The 20D and 30D autofocus system isn't really a step up from the 350D. For some reason, autofocus is an area where these cameras seem to have poor quality control. This means you need to be tolerant (almost expect) a bit of autofocus slop in a 20D.
  • There are techniques which can help you compensate for poor focus performance of the 20D.
So, what can you do?

Test your camera, of course! In my opinion, anyone who gets a 20D or 30D should run some simple tests on their camera to verify it doesn't have a (major) focus problem and identify any specific focusing issues.

Testing Procedure:

The testing procedure I used is modeled after Francis Poon's test (and uses his target). Go to his site for the details. An alternate test and target are here and the idea is the same (but the target construction is advanced origami).

The basic procedure is:
  1. Print a test pattern and assembled the 45 degree target. I taped mine to a clipboard.
  2. Set the target on a 45 degree slant which will face the focusing target directly at the camera. I leaned mine on a box and secured it with scotch tape.
  3. Make sure you have plenty of light on the central focusing target. I used a lamp for the full area, then a flashlight on the focusing target. In all honesty, I should have gotten more light on it. See the image that opened this post for the full setup.
  4. Set your autofocus to use your center sensor. You could run the test on the other sensors too, but start with the center one.
  5. Focus, using manual or autofocus depending on the test, and then take a picture. Better yet, take a bunch of pictures, with different lenses, different focusing techniques, and keep track of what you did.
  6. Look at the images on the computer (usually 100% crops) and use the scale to the left and right of the primary focusing target to determine where the depth of field is. The letters ('a') are the best guide to seeing the region of focus. The scale is designed to give millimeter depth measurements if the image is tilted at a 45 degree angle (I didn't do anything special when I printed mine, so it could be a little off).
  7. Compare your measurements to the expected degree of field for your focal length and aperture measurement. I used the DOF calculator here for my computations.
It may seem complicated, but it really isn't. The hardest part is determining what you will test and keeping to the scientific method. Generally, you should try to keep everything as similar as possible between test photos except the variable you are testing. This includes controlling your lighting, camera to target distance, viewing angles, etc.

If the focus is good, you'll get an image like this (click to see the unedited 100% crop of the JPEG):

Note the 'blurriness' of the 'a' line. That was my main guide for finding the degree of field.

If the focus is bad (for reasons listed below) you'll get a crop like this:

The image above shows a front-focus of about 12 mm. Depending on the lens and aperture, that may not even be out of the depth of field.

Seriously, this test isn't that hard. Just takes a little organization and a few hours.

Possible Areas of Failure:

There are a number of areas in a camera which can be miscalibrated. Luckily, pretty much everything can be tested with the target set-up above. Just come up with your own sequence of images and procedure (I'll list mine in my next post). Make sure you check the following:
  1. Viewfinder plane calibration. This isn't often a problem, but you'll want to know if the sensor plane is at a different distance than your viewfinder plane. Just manually focus on the target and shoot a few shots (clearing your focus each time). This ends up being an eyesight test too, but if you take your time and have decent eyes, you'll definitely get within the depth of field.
  2. Test the autofocus calibration. Test your best lens, the longer the focal length the better, since longer focal lengths have a narrower depth of field and the autofocus will be more accurate. Autofocus from both focus extremes (close and far) and see how well it does. If there is a consistent focus offset that is significantly larger than the DOF for that focal length and aperture, you may have a problem. Apparently, this is more common than problem #1, since the floating mirror has some play in it. A little offset, especially on a 20D or 30D (of about 1 DOF) is probably to be expected, any more than that and you probably ought to have Canon calibrate it if it is under warranty.
  3. Test each of your lenses. Since the camera works with the lens, there can be significant variation between lenses. If one lens is much worse than the others, it needs to be calibrated.
  4. Test each focus sensor. I drew horizontal and vertical lines of different thicknesses, and tested to see how well the different focus sensors picked them up. If one of your sensors is faulty, this is the way to find the problem.
  5. Test for viewfinder screen offset. While the plane of the viewfinder screen is usually well constrained, there is a decent amount of play in the horizontal direction for the sensor boxes. So see if you can focus with the sensor indicator slightly above or below a feature -- with some experimentation, you can determine how your viewfinder indicators are offset from the actual sensors. Unless it is a huge amount, just keep that in mind when you are using your camera. Also, keep in mind that each sensor is usually three times the length as the indicator, but relatively narrow in the other direction.

That covers it, stay tuned for my next post about my results.

No comments: