[Aavso-photometry] Sources of accuracy and precision in photometric measurements

[Radu Corlan]

> There are 4 main areas which affect doing accurate, high-precision 
> photometry:
> 
> 1) the sky background estimate
> 2) the partial pixel algorithm
> 3) the centroiding algorithm
> 4) the random error estimates (sigma of magnitude) must be valid

Michael,

These are certainly important areas, and neglecting any one of them 
can introduce unanted errors. I've added a few comments below, in the 
intention of showing that unforunately there are no "fast" solutions.


> 
> These issues have the strongest effect at the faint end where many of us do 
> the interesting work. Here are some specifics:
> 
> 1. Sky estimation:
> Mira offers 3 methods: Mean, Median, and Mode. Knowledgeable photometrists 
> have their favorite. And no single method is necessity the best in all 
> sorts of images. But generally speaking, the mode method is the best 
> choice. But just because a software authors says "ours measures the mode" 
> does not mean it measures the mode or gives the same mode as other 
> software. Measuring the mode is conceptually straightforward, but in real 
> data, it is a real challenge. One person's "mode" is not another person's 
> "mode", but that is another topic! If calculate correctly, the mode can 
> give the most consistent results when stars and hot/cold pixels contaminate 
> the sky measuring annulus. Our website at http://www.mirametrics.com has an 
> example showing the effect of background contamination when doing 
> photometry of the minor planet Xanthippe (this is linked from the Product 
> Briefs menu item and also at the upper right corner of each product page). 
> In this study, 9 images were measured for a moving object. In the first 
> couple of measurements, a star 0.5 magnitudes brighter than Xanthippe is 
> located in the sky annulus. This is usually a horror story for a photometry 
> algorithm. With Mira AL, the magnitude estimate of the object was biased 
> only 0.01 magnitudes by the contamination. That is darned good performance 
> for a sky rejection algorithm.

Mode does have good outlier-rejection properties. There is however one 
big issue with it: it's badly defined in the presence of noise. by 
definition, the mode is the peak of the intensity distribution (the most 
frequent value). with a limited number of samples and in the presence of 
noise, it's obvious from just looking at the distribution that the peak 
is not the value one wants. So various authors use different ways of 
estimating the mode - which does pose a problem: how do you know the 
properties of a particular method of mode-ing? in many cases you don't.

My second problem with mode is that it's quite bad with a tilted 
background: for an uniform tilt, the distribution of values is uniform - 
the mode cannot be defined at all!

> 
> 2. Partial Pixels:
> Around the rim of the measuring aperture is a tail of pixels that only 
> partially sample the image. How they are used in the magnitude calculation 
> is very important. Only Mira handles partial pixels *exactly*--- no 
> approximations. No other software on the planet does this for the general 
> case (elliptical or circular apertures in Mira AP and Pro, and circular 
> apertures in Mira AL). The mathematics is outrageously complex to 
> implement. We know that Mira does it right because the algorithm has proven 
> in the field by more than a decade of use. How do we know each measurement 
> does this correctly? The area of an ellipse is (pi * a * b) using the 
> aperture size the user specifies. A circle has a = b, so area = pi * r^2. 
> Mira tallies the sum of areas as well as the weighted sum of pixel 
> intensities based on the partial pixel areas. If the difference is 0.001 
> pixels or more, Mira pops up an error box asking the user to contact us. 
> During the Mira AL beta testing cycle someone contacted us about that 
> error, but it turned out there was a different bug that allowed the 
> measuring aperture to be only partially on the image. So the algorithm is 
> time-proven. And particularly with small apertures, and for faint stars, it 
> does make a difference in the precision of the photometry.

Ah, but apportioning the flux of a pixel to an aperture based on the 
fraction of the area inside the aperture is only "exact" if the pixels 
were uniformly sensitive _and_ uniformaly illuminated. In any practical 
case (particularly with smallish apertures), especially the second 
hypothesis is far from being true. So in this case we are left with 
determining whether the error introduced by using some approximation of 
true circular or elliptical apertures is significant compared to error 
introduced by sampling. My tests have shown me that there is a 
detectable improvment in using "true" apertures vs whole pixels, but do 
detectable improvment over the simplified "irregular polygon" algorithm 
of iraf phot. The author of phot also seem to be holding the same view. 
I have to say however that in these days of plentyful computing 
resources, there is not much justification in rejection an even 
marginally superior algorithm on the basis of complexity alone.

> 
> 3. Centroiding noise:
> This is a *widely* neglected source of photometric error. Placing an 
> aperture on the center of the light distribution would seem a simple thing 
> but it is not, especially for faint stars. I would invite everyone to look 
> at a study I did of centroiding noise as a part of the photometric error 
> budget. This is shown on our website at http://www.mirametrics.com. On the 
> left side menu, click on "Virtual Library" and then see the 7th item down 
> the list of topics. In item 1 above, I mentioned an example on our website 
> showing measurement of a moving object. The position of moving object and 
> standard star were both automatically centroided by Mira on each image. The 
> internal consistency of the photometry between images is near the milli-mag 
> level, and this value doubtless includes some variation in the asteroid 
> brightness itself due to rotation. If Mira did not do robust centroiding 
> with high accuracy, the centroiding noise itself would have dominated and 
> the magnitude consistency between the images could never have been anywhere 
> near the milli-mag level.

No argument on this: centroiding is very important unless one has the 
luxury of being able to use rather large apertures (and even then).

Radu


> 4. Good estimates of the magnitude uncertainty:
> Of course, the estimates of magnitude errors (random errors!) must be done 
> correctly for doing science. But, like the other 3 items, getting error 
> estimates also is not straightforward. To give photometrists the 
> information they need, Mira quotes 2 errors: an empirical error measured 
> from the noise in the data, and a theoretical error based purely upon 
> detector properties, aperture area, and the signal. The two estimates are 
> independent except for the magnitude value (which is not important in the 
> issue of independence). The empirical error estimate should scatter around 
> the theoretical values and, the brighter the star, the more they should 
> agree. In Mira, they do.
> 
> Michael Newberry
> 
> 
> ----- Original Message ----- 
> From: "arne" <arne at aavso.org>
> To: "AAVSO Discussion group" <aavso-discussion at aavso.org>
> Sent: Wednesday, December 14, 2005 6:36 AM
> Subject: Re: [AAVSO-DIS] Canopus software for variable star work
> 
> 
> >>Tom Richards wrote:
> 
> [...]
> 
> >Arne wrote:
> >
> >The AAVSO has been considering doing some tests.  The problem is that
> >every vendor is sure that their algorithm is correct, and if you
> >don't get the right answer, then (a) you aren't using the latest version
> >of the software, (b) the software was not designed for that specific
> >case (such as trailed asteroids), or (c) you aren't using the
> >software correctly.  These are valid points, of course, but published
> >tests are always going to result in vigorous discussions.
> >
> >The easiest first step is to supply a set of images that everyone can
> >use to test their software.  That is what I am working on right now.
> >Arne
> >_______________________________________________
> >
> >Aavso-discussion mailing list
> >Aavso-discussion at mira.aavso.org
> >http://mira.aavso.org/mailman/listinfo/aavso-discussion
> >
> 
> _______________________________________________
> 
> Aavso-photometry mailing list
> Aavso-photometry at mira.aavso.org
> http://www.aavso.org/mailman/listinfo/aavso-photometry

-- 
Radu Corlan      

   You can still escape the "Gates" of Hell!
                 Use Linux!