[Aavso-photometry] Sources of accuracy and precision in photometric measurements

[Michael Newberry]

Excellent points and discussion, Radu. I hope everyone is getting good info
out of this. I have 2 points below to follow on what you say.

Michael

----- Original Message ----- 
From: "Radu Corlan" <radu.corlan at visionresearch.com>
To: "Michael Newberry" <mnewberry at mirametrics.com>
Cc: <aavso-photometry at mira.aavso.org>
Sent: Wednesday, December 14, 2005 4:13 PM
Subject: Re: [Aavso-photometry] Sources of accuracy and precision in
photometric measurements


>> >Mode does have good outlier-rejection properties. There is however one
>> >big issue with it: it's badly defined in the presence of noise. by
>> >definition, the mode is the peak of the intensity distribution (the most
>> >frequent value). with a limited number of samples and in the presence of
>> >noise, it's obvious from just looking at the distribution that the peak
>> >is not the value one wants. So various authors use different ways of
>> >estimating the mode - which does pose a problem: how do you know the
>> >properties of a particular method of mode-ing? in many cases you don't.
>> >
>> >My second problem with mode is that it's quite bad with a tilted
>> >background: for an uniform tilt, the distribution of values is uniform -
>> >the mode cannot be defined at all!
>> >
>>
>> I agree with the essence of your points about the mode. This is what I
>> meant
>> about the huge difference between the concept of the mode and actually
>> implementing its measurement in real data.  As I said, "one man's mode is
>> not another man's mode. It depends on whether the calculation is robust
>> in
>> the presence of noise and tiled background. Mira uses a fairly robust
>> algorithm that is insensitive to noise. When the background is tilted,
>> indeed, the mode becomes smeared by
>> convolution with a rectangle function. And once again, the calculation
>> has
>> to be robust. If the background is tilted so much that it affects Mira's
>> mode results then I would suggest people not be trying to to aperture
>> photometry on the image in the first place.
>
> Well, average combined with outlier rejection works pretty well in these
> cases; so does full-frame background fitting.
>
    I agree about mean + rejection, as a concept, anyway. But good or bad
perfoemance depends upon what level of precision one is trying to obtain. If
a photometrist is trying to do 0.1 magnitude work, almost anything will
work. Getting down to 0.01 magnitude is harder. Getting nearer to 0.001
magnitude pushes all algorithms into the panic zone. I work in the realm of
pushing the envelope. When trying to approach the milli-mag level, using
outlier rejection is like measuring the mode: good on paper, not easy to get
"expected results" using real world data. Here are some limitations of
outlier rejection (also called "clipping"): Unless the sky samples a *lot*
of points, outlier rejection still has a tendency to pull the sky value in
the direction of the outliers. Said another way, it requires a very large
sample for outlier rejection to reject *just* the outliers. You can do
better by increasing the statistical significance of the sample. How do you
do that? Add more points to the sky sample. But adding points enlarges the
sky region, which then moves it away from the object. That introduces 2 more
problems: 1) CCD images are not completely flat unless you really have
beaten your brains out on the flatfielding issues, and 2) the sky further
from the object samples a different background of diffuse unresolved
objects. To getting outlier rejection to work as theorized, you are trading
precision for accuracy, which is not good either.

>>
>>
>> >>2. Partial Pixels:
>> [...]
>>
>> >Ah, but apportioning the flux of a pixel to an aperture based on the
>> >fraction of the area inside the aperture is only "exact" if the pixels
>> >were uniformly sensitive _and_ uniformaly illuminated. In any practical
>> >case (particularly with smallish apertures), especially the second
>> >hypothesis is far from being true. So in this case we are left with
>> >determining whether the error introduced by using some approximation of
>> >true circular or elliptical apertures is significant compared to error
>> >introduced by sampling. My tests have shown me that there is a
>> >detectable improvment in using "true" apertures vs whole pixels, but do
>> >detectable improvment over the simplified "irregular polygon" algorithm
>> >of iraf phot. The author of phot also seem to be holding the same view.
>> >I have to say however that in these days of plentyful computing
>> >resources, there is not much justification in rejection an even
>> >marginally superior algorithm on the basis of complexity alone.
>> >Yes, with big fat stars
>>
>> It all depends on the ratio of PSF size to pixel size. With stars are
>> large
>> relative to the PSF, i.e., "poor seeing" or poor sampling, you and the
>> author of RPHOT are correct about using a trapezoidal approximation. But
>> this claim becomes progressively less valid as you reduce the ratio of
>> PSF /
>> Pixel. As you say, the exact aperture solution is not perfect in that
>> case
>> either, but it gives you a better result with greater magnitude
>> consistency
>> than from using trapezoidal approximations as in RPHOT.
>
> I have to differ here. what i meant is that the error due to sampling
> rises much faster than errors due to shape approximations especially
> when undersampling. That being said, i'd love to be proven wrong ;-).
>

Indeed, but I'm focusing not on how fast errors rise, but on the total error
budget. Everything has to be done "right" to make the total error budget
approach the magical milli-mag level of quality. If algorithms can do that
you know they are good for less stringent work.

Regards,

Michael


> Radu
>
>>
>> >>
>> >>3. Centroiding noise:
>> [...]
>> >
>> >No argument on this: centroiding is very important unless one has the
>> >luxury of being able to use rather large apertures (and even then).
>> >
>> >Radu
>>
>> _______________________________________________
>>
>> Aavso-photometry mailing list
>> Aavso-photometry at mira.aavso.org
>> http://www.aavso.org/mailman/listinfo/aavso-photometry
>
> -- 
> Radu Corlan
>
>   You can still escape the "Gates" of Hell!
>                 Use Linux!
>
>
>
>