[Aavso-photometry] [AAVSO-DIS] Software comparison

[Michael Newberry]

I don't really know enough about software other than Mira to comment on 
them. But I would say that Mira's photometry package has been evaluated over 
and over again by both professionals and serious amateurs for many years and 
the feedback we have received is that it gives accurate magnitudes and 
decent error estimates. You can see how consistently the Mira results agree 
over the course of a couple hundred images by reading about Ron Bissinger's 
detection of a 0.003 magnitude exo-solar planet eclipse observing from 
"Silicon Valley", California (see 
http://www.mirametrics.com/news_hd149026.htm).

As Wolfgang said in a later post, perhaps the stars were not being measured 
in both software packages using identical radii for the object, inner, and 
outer sky apertures. But supposing that the radii were all the same, then 
the magnitude differences boil down to a few basic concepts:

    1. How is the background measured?
    2. How are the partial pixels measured around the aperture rim?
    3. How accurately centered is the aperture on the star?

Discussing these topics in serious detail is a VERY big subject---like, book 
size. My suggestion is never to treat software as a black box and take what 
you get without validating it or knowing someone you trust has validated it. 
Compare reported measurements with visual validation of the same objects in 
the same images. For example, make a row intensity profile through a 
measured star and compare your "eyeball" sky brightness with the value 
reported by the software. Or have the software draw contours on the star 
profile, zoom up the image, and visually compare the reported star center 
coordinates with what the contours---do they agree?.

Regarding the partial pixels around the aperture rim, as for the background 
and centroid measurements, the devil is in the details---and the details are 
extensive. I know that Mira has handled partial pixels "exactly" for both 
circular and elliptical apertures for 15 years. For about the last 5 years, 
IRAF has handled the circular aperture case exactly. Other than that, 
software packages use varying degrees of approximation for the partial 
pixels. The degree to which a particular treatment is important to the 
measurement depends on the FWHM and aperture radius. In general, the larger 
the fraction of a star's light is measured through partial pixels rather 
than in whole pixels interior to the aperture, then the larger the effect a 
particular treatment has on the measurement.

Michael Newberry

----- Original Message ----- 
From: "Bill Dillon" <dillon1 at houston.oilfield.slb.com>
To: "aavso-discussion" <aavso-discussion at mira.aavso.org>
Sent: Wednesday, May 02, 2007 7:54 AM
Subject: Re: [AAVSO-DIS] Software comparison


> Actually, I'm impressed they only differ by ~0.01 mag (my random errors
> are often bigger than that, depending on the target brightness, exposure
> time, etc.).
>
> You could do a test with an artificial star, but the key is how the
> software handles noise: background stars, merged stars, hot pixels,
> partial pixels in circular apertures, etc.  Perhaps better would be to
> take a field with good comps and, one by one, pretend the comps are a
> variable -- see how close your estimates are to the given value, using
> the same image and different software packages.
>
> If you want algorithm transparency, then maybe IRAF is a way to go.
>
> --Bill
>
>
> Peter Nelson wrote:
>> I seem to recall a post somewhere about some software that could produce 
>> an
>> artificial field with known magnitudes.  If this existed, could we then 
>> test
>> reduction software against this?
>>
>> Regards
>> Peter
> _______________________________________________
>
> Aavso-discussion mailing list
> Aavso-discussion at mira.aavso.org
> http://mira.aavso.org/mailman/listinfo/aavso-discussion
>
>