Photometric Accuracy/Precision - A Perspective

Affiliation
American Association of Variable Star Observers (AAVSO)
Sun, 01/04/2015 - 23:59

Photometric Accuracy/Precision – A Perspective

Frankly there is no truth out there, in a manner of speaking.  In other words there is probably not any such thing as a 100% absolutely accurate photometric measurement.  Photometric Precision is an illusion (that is not to say that the observing and measurement techniques cannot be precise).  

While a bit of a generalization, the system is a bit of a house of cards starting with original ~ equatorial  PEP measurements many years ago that established calibrated standards for comparison stars (there had to be an arbitrary point of beginning) and then those standards were extrapolated using all sky photometry to other regions of the sky.  Then when CCD's came along they adopted then those measurements and used them for larger sky surveys.  (Each step of the way has its own variance or uncertainty in each measurement and those uncertainties can very well compound themselves but such compounded uncertainty data is rarely carried forward).

Then there are the differences that occur with calibrated fields from different surveys.  One survey or survey period might have a measurement of a specific star at 12.358 while another survey might have it at 12.398 or another at 12.312 and another at 12.418 while another shows 12.456.  Different locations, different times at the same location, varying seeing and different equipment all contribute to differing values for a specific star’s calibrated measurement.

Most observers then will start their differential photometry processing using preselected sequence data from a single (for a given magnitude range) survey (VSP, in the case of an AAVSO chart); data that may or may not agree with a different survey of the same stars.

The observer then has choices to make:  A good comp and check star (color and magnitude matching, if available) ; or a 15 star ensemble; or a three or five star ensemble; whether or not to average 3 or more measurements; an aperture measurement  radius of 1.2xFWHM or 1.5xFWHM or  2xFWHM, etc…

All of the above permutations and combinations will generate different answers; not a lot different, we hope, but, never the less, different answers.  Which ones or more accurate?

If we spend some time at it we can even, on an individual basis, depending upon the comps, their colors and their own original photometric calibration, as well as seeing, clouds and or fog, discover that possibly just one combination or permutation gets a statistically improved answer as opposed to another, for the existing conditions; but is it really an absolute precise accurate photometric result?

Probably not; but it is, and this should apply pretty much to what other combination and permutation might be used in its stead, very useful scientific information and this is, PROVIDED, that the choices the observer makes are consistent from observation to observation, both in the observing technique and the processing technique (the other assumptions being that the observer is not under sampling, misidentifying the target/comp star or making other processing errors/faults). [Common CCD observer mistakes, errors and faults are for another discussion]

It is the changes that we measure from observation to observation which contain the scientifically useful data; therefore consistency of measurement techniques is probably more important than the specific approach used by any one individual observer (or at a minimum, equally important).

With this said, my final presumption is that the CCD observer will try to avoid methodology that results in large uncertainties.  OK, then, what is to large? Well, experienced visual observers can be accurate to .1 magnitude so I think for most CCD observations, unless unusual circumstances (mag ~18+ target for average setup or poor conditions and needed data) that we should be able to beat that.   .01 or less is the ideal uncertainty but there is really nothing wrong with uncertainties out to ~ .05; in between that and .1 is sort of the wild west, in a manner of speaking, and requires best judgment for the situation.  I think I one time reported an uncertainty of ~.3 on a magnitude 19 target; there was a need for the data and that was the best that I could do; sometimes the data is simply: “it is what it is.” (Uncertainty discussions can really get long and involved so this topic is purposely abbreviated herein).

Because of the various differences in equipment, location, observing and processing options, that each observer makes, the data, for a given time, may differ from another observer by some hundredths, or at worst a tenth or two.  Never the less, for the most part, assuming that the individual observers have been consistent (and without fault), the data from the various observers can generally be reasonably curve fit for analysis, which after all is the end goal use of the gathered data.

‘nough said

Tim R Crawford, CTX

Never Assume! Never Presume! Check, Double Check,Cross Check and Verify!

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Photometric Accuracy/Precision - A perspective

Tim,

Eloquent summary.

Thanks, 

Brad Walter, WBY

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Thank You

Brad,

Thank you for your kind remark.

I only hope that more observers will at least read the post.

Per Ardua Ad Astra,

Tim Crawford, CTX

PS, to quote a friend of mine, I wonder how you calculate “how many angels can dance on the head of a pin?”

Affiliation
American Association of Variable Star Observers (AAVSO)
Photometric Accuracy/Precision – A Perspective

Tim,

I have found your summary very instructive. Incidentally we covered this issue last year when you were helping me get the transformation coefficients of my system. I fully agree that, aside from errors that must be avoided in CCD observing, the thing that matters is the analysis of the light variation of the target. Scatter will continue to exist in light curves as it reflects different sources of noise, seeing etc. among observers. Nevertheless all properly taken data have scientific value. I have recently realized that by following some exoplanet transits. Although the light curves I got show a scatter of 0.01-0.02 mag they fit perfectly well the catalogue geometry of the observed systems even if the depth of the transit is very small (in the order of 0.01 mag).

Gianluca

Affiliation
American Association of Variable Star Observers (AAVSO)
Aperture Radius

Hello Tim

Very well put.  You covered a lot of ground.  One thing I wanted to comment on:  You said :(the observer must choose between)  an aperture radius of 1.2xFWHM or 1.5xFWHM or  2xFWHM, etc…

The AAVSO and Arne has always recommended that observers used 2-3 FWHM for the aperture radius.  Of course "it depends" and there may be sepecial circumstances that dictate otherwise, but 2-3 FWHM is the recommendation for most of our measurements.

While single star measurements may lead one to a different conclusion, when doing time series it is most important to start with the 2-3 FWHM baseline.  

WGR

Gary Walker