[Aavso-photometry] Cosmic Rays

Sat Jan 28 10:36:16 EST 2006

Ben,

I appluad your efforts to get clean images and produce the best 
measurements. But I don't think your method is giving you what you think you 
are getting. I see what you are striving for but I just don't believe it can 
be done. I'll tell you why.

You are trying to distinguish an outlier from ordinary deviations within a 
random distribution. It is a probabilistic process. If you detect a value at 
4-sigma's from the mean, is it a radiation event? What do you expect at 4 
sigma's. Suppose there are ZERO outliers. The chances of a value drawn 
randomly from a Gaussian distribution to have a value 4 or more sigma's 
above the mean is about 1 in 30,000. This is rare, but it can happen. But 
for a CCD with 2 million pixels, it means you will have 60 to 70 of 
them---and they are valid, not outliers. This scales with the noise. Suppose 
the noise is 6e-. Then this means a pixel value of 24e-. If the noise is 
12e-, then this means a value of 48e-, and so on. When you get a pixel value 
of 48e- and the total image noise is 12e-, what is it: random deviation or 
outlier? This is where the probabilistic process comes in. You know that 
there should be around 60 to 70 pixels in this realm of 4 sigma's above the 
mean. Do you delete that outlier or not? Is it a cosmic ray? If it shows a 
horizontal track then you can be pretty sure it is a cosmic ray. If it is a 
point, then roll the dice... If you count 20 such points in a 2 megapixel 
image (you would get 60 to 70 purely from chance), then it is probably safe 
to leave them in. If you count 200 of them, then more than half are probably 
cosmic rays. But which ones? Certainly, erroneously rejecting a few 4 sigma 
pixels is not going to ruin your measurements because there are so few of 
them to carry much weight in combination with other pixels. When you 
consider rejecting yet smaller deviations of 3 or even 2 sigma's, you are 
potentially throwing away a significant fraction of the random population. 
There just aren't enough cosmic rays to go around that could populate an 
image to that degree---unless the camera was on the robot that went into the 
Chernobyl containment vessel. :o)

To detect an outlier you need contrast against the random noise. It is a 
matter of seeing n sigma's against 1 sigma. You want to detect 6e- 
"outliers" so how many sigma's is that? Let's look at the noise budget of an 
image;

Thermal noise: A liquid nitrogen cooled CCD is required to get a dark 
current in the neighborhood of 6 or fewer electrons in 30 minutes. The noise 
component from 6 thermal electrons would be sqrt(6) = 2.5e-. If your camera 
is TEC or liquid cooled, the thermal signal will be much higher. Let's say 
it is fairly "quiet" and gives 1e-/minute. That makes 30e- signal in 30 
minutes, and thus a noise of sqrt(30) = 5.5, or call it 6e-.

Readout Noise: This is a combination of 2 effects: what the chip contributes 
and what the electronics contribute. Getting 6e- noise from the on-chip 
amplifier is near state of the art. Add to that the noise from the camera 
electronics which is of the same level in a rather exceptional camera. The 
combination of the two would give 8.5e- readout noise. Getting a total noise 
from the sum of the two which is near 6e- requires an exceptional CCD and 
camera. I don't mean a top-end SBIG or FLI camera here---which are good 
products in their own right, but they are simply not in the technology class 
we're talking about.

Digitization Noise: This results from converting n electrons to 1 ADU (or 
"count"). Maybe I should include it in the "readout noise" but I will 
describe it separately. When chopping electrons to a smaller number of 
ADU's, count resolution is lost, and this adds uncertainty or noise to what 
the ADU value is actually measuring. This noise value is given by n / 
sqrt(12). If your gain is 2.5, then this gives 2.5/3.5 = 0.6e- of 
digitization noise. That is quite small but everything counts when looking 
at a total noise budget under 10e-.

Photon noise: You mentioned darks so this is not a contributor. But when 
measuring any amount of light, there is noise inherent to the number of 
detected photons (as electrons). The noise of this type is given by the 
square root of the number of electrons detected. If the sky signal is quite 
faint, at only 10 ADU, and the gain is 2.5e-/ADU, then the photon noise in 
the sky background is sqrt(10 x 2.5) = 5 e-. Such a faint sky means you are 
taking short exposures and observing only bright stars.

The total noise in an image is given by the square root of the sums of the 
square of the noise components. Assuming a state of the art CCD in a state 
of the art camera operating at LN2 temperatures, the readout noise in a 30 
minute dark frame would be around the level of 6e that you mentioned.

Combining 3 darks by the median gives a noise that is scarcely lower than 
the noise in one image, and it is subject to small number fluctuations, That 
makes it likely to get many large but random deviations from non-outliers. 
If the noise level of one image really is 6e-, then the median of 3 such 
images gives back a noise level of around 4 to 5 e-. But since it is only 3 
images being combined there are huge fluctuations from one pixel to the 
next.

So far I have dealt with the intrinsic noise of an image---that noise which 
is not contributed from any *other* image. When images are calibrated you 
must correct for bias, dark, and flat field effects. This compounds the 
noise giving the image moise noise than its intrinsic noise. The noise in 
the bias correction (however you do it) is added to the intrinsic noise of 
the dark, flat, and data frames. In the next step, the noise of the 
processed dark correction is added to the intrinsic noise of the flat and 
data frames. And finally, the noise of the processed flat field correction 
frame is added to the intrinsic noise of the data frame. By "added", I don't 
mean added in a strict linear sense, but in a rather complex sense that 
cannot be modeled in a simple way. Let's just say that there is a pyramid of 
noise that compounds into the random noise of the final, processed data 
frame from which a star is measured.

When computing the median, the images MUST have been exactly normalized to 
have the same signal level over a large sample area. If not pre-normalized, 
the median returns simply the image in the middle, not a combination of 
pixel values from the separate images, and hence the noise is not reduced. I 
will assume that the median is correctly computed and applied before median 
combining (I have seen various software packages do some strange things in 
this regard).

Putting all of this together, detecting non-random "outliers" at nearly the 
same level as the random noise means catching everything above 1 sigma on 
the positive tail of the random noise distribution. This rejects 16% of the 
pixels even if, in fact, there were 0 cosmic rays.

Nature provides those random deviations, both large and small, positive and 
negative, as estimations of the true pixel value. Trimming them away biases 
the signal to higher positive values with a truncated distribution having no 
positive tail. If you are trying to do precision photometry then It is not 
clear to me what you would be getting.

The above issues are described in a  paper I published in PASP 1991, vol 
103, p. 122. You can get it on the Harvard reprint server here:
http://articles.adsabs.harvard.edu/cgi-bin/nph-iarticle_query?db_key=AST&bibcode=1991PASP..103..122N&letter=.&classic=YES&defaultprint=YES&whole_paper=YES&page=122&epage=122&send=Send+PDF&filetype=.pdf
It is a little technical, but not too bad, involving mostly summations and 
algebra in the math details.

Michael Newberry

----- Original Message ----- 
From: "Ben Davies" <ben at davies.net>
To: <aavso-photometry at mira.aavso.org>
Sent: Friday, January 27, 2006 10:55 PM
Subject: Re: [Aavso-photometry] Cosmic Rays

> You say cosmic rays at "half a dozen electrons". That's a little tight. 
> What is your noise level?
>
> Hi Michael,
>
> The noise level is essentially 0.  Try this for yourself.  I think you 
> will like it.
>
> -Make 3 dark frames of 30 minutes each.
> -Median combine them
> -increase the magnification to 400% and  blink one of the darks with the 
> median.  It is absolutely amazing the 'vermin' you will see revealed in 
> the image.  At least you will if you have the same sort of radiation 
> hitting the chip that I do.
> - Then subtract the median from one of the darks.  This leaves you with an 
> image that has only the cosmic ray artifacts (no asteroids here) and near 
> 0 background level.  The background is going to bounce around a bit from 
> pixel to pixel (-1 to +1) because the median is of 3 images and you are 
> only subtracting it from one of them.
>
> The process with actual starfield images is as I described in a previous 
> post.  As Wolfgang points out, there needs to be a better combining 
> routine than averaging, but that will do until one comes along.
>
> Ben Davies
>
> _______________________________________________
>
> Aavso-photometry mailing list
> Aavso-photometry at mira.aavso.org
> http://www.aavso.org/mailman/listinfo/aavso-photometry
>
> 