Auto-focus accuracy: a scientific cross brand analysis (guest post)
The next guest post is by Lionel Humbert who took a scientific approach when examining the AF accuracy acros camera brands:
Many discussions emerge on the Internet about image quality, noise, focus accuracy… when a new camera hits the market. Some people move from one brand to another based solely on some brief, vague and not justified reports viewed on the web. Generally tests published on the Internet are by design not reproducible and biased. A scientific approach will required multiple bodies and lenses, and at least more than one sample for each model because of small manufacturing differences that may influence results.
In this article I will discuss the accuracy of focus with a rigorous analysis using modern statistical methods on data collected from a well known website (www.lenstip.com, 8000 visits per day). This analysis is focused on their tests because it is the only website that reports the accuracy of focus for each lens. They get a percentage of error, measured as the number of times the resolution on the target is less than 20% of the maximum MTF50 obtained by this lens. The data cover 18 bodies of six brands, 114 lenses from 11 manufacturers for a total of 162 bodies/lenses tests.
Without rigor we could draw a bar plot with on the X axis the percentage of auto-focus error and on the Y axis the frequency of lens tested showing this auto-focus error (Figure 1) for each brand. Samsung was removed in this figure, since only one test was available.
Figure 1: Barplot of Auto-Focus error per brand
Based on this figure the conclusion is : Nikon gets the best results with 70% of tests with a low auto-focus error (between 0 and 5% of error) and only 10% of tests with high auto-focus error (more than 10% of error) , followed by Sony and Canon, Pentax and Olympus with worst results (45% of good results – 0-5% error – and 25% of high levels of error – > 10% -) . However this figure is terribly misleading, because other variables can influence these tests: the model of the body, the lens manufacturers, lenses model, and aperture…
Many discussions emerge on the Internet about image quality, noise, focus accuracy… when a new camera hits the market. Some people move from one brand to another based solely on some brief, vague and not justified reports viewed on the web. Generally tests published on the Internet are by design not reproducible and biased. A scientific approach will required multiple bodies and lenses, and at least more than one sample for each model because of small manufacturing differences that may influence results.
In this article I will discuss the accuracy of focus with a rigorous analysis using modern statistical methods on data collected from a well known website (www.lenstip.com, 8000 visits per day). This analysis is focused on their tests because it is the only website that reports the accuracy of focus for each lens. They get a percentage of error, measured as the number of times the resolution on the target is less than 20% of the maximum MTF50 obtained by this lens. The data cover 18 bodies of six brands, 114 lenses from 11 manufacturers for a total of 162 bodies/lenses tests.
Without rigor we could draw a bar plot with on the X axis the percentage of auto-focus error and on the Y axis the frequency of lens tested showing this auto-focus error (Figure 1) for each brand. Samsung was removed in this figure, since only one test was available.
Figure 1: Barplot of Auto-Focus error per brandBased on this figure the conclusion is : Nikon gets the best results with 70% of tests with a low auto-focus error (between 0 and 5% of error) and only 10% of tests with high auto-focus error (more than 10% of error) , followed by Sony and Canon, Pentax and Olympus with worst results (45% of good results – 0-5% error – and 25% of high levels of error – > 10% -) . However this figure is terribly misleading, because other variables can influence these tests: the model of the body, the lens manufacturers, lenses model, and aperture…
No comments:
Post a Comment