[Aavso-photometry] CCD Linearity testing

Wed Jul 8 14:01:36 EDT 2009

Hi Tom,

A nice elegant first go at measuring CCD linearity.  You have tied it down 
pretty well and your data show the expected drift in the system (combined 
effect of changes in light intensity and CCD response).  The drift amounts 
to +0.4% in the first 5 or 6 test duty cycles followed by a -0.6% drift for 
the remaining.  I'll send you a plot so you can see this.  It's a 
consequence of performing the experiment over many hours of time.  I suggest 
you just take a subset of the data between the 4th and 9th cycles 
(inclusive) and plot those data since over this time interval the system was 
most stable.

Richard Berry's comment below that "If you can afford to, the standard 
method now is to use a pulsed LED, and use the number of pulses as the 
intensity of the light falling on the CCD, so 50 pulses is 50 times more 
light than 1 pulse. You can buy standard sources like this for $$$."  It 
turns out that you can build a linearity tester for just a few $ as 
demonstrated by John Saxton privately.  [Am copying John this note.]

There are lots of advantages in going the pulsed source route:  (a) it is 
very quick so you can complete the 'imaging' in a relatively few minutes 
thereby avoiding the drift problem, (b) a constant exposure is used for the 
CCD camera so the same darks apply to all the measures and you can avoid 
having to take bias frames, (c) the lightsource tends to be more stable as 
it does not gradually heat up (or cool down in falling ambient temperatures 
over a longish period), (d) the exposure time of the CCD is kept short (say 
just 4 seconds) so then it is less affected by cosmic ray hits and hot 
pixels, and (e) you eliminate the shutter open-time as another uncertainty 
factor - this is especially true at the bottom end of the linearity range, 
where for example you have used a 0.5-sec exposure.  A quartz crystal 
timebase determines the duration the LED is switched on and in John's setup, 
a thumbwheel switch allows you to preselect the integral no. of flashes.

Coming back to your setup.  One problem with the top end of the range (say 
 >50,000 adu) is that if you select a large area of the CCD then you start 
getting more pixels reaching 65535 'early' as you increase the exposure time 
and this makes it look as though the response is dropping off even though it 
isn't.  You refer to this effect as well.  I would be particularly careful 
to select a uniform part of the CCD frame which is not too affected by hot 
pixels then you can calibrate closer to the 65535 cut-off.

Richard Miles
Golden Hill Obs., Dorset, UK

----- Original Message ----- 
From: "Tom Krajci" <tom_krajci at tularosa.net>
To: <aavso-photometry at aavso.org>
Sent: Wednesday, July 08, 2009 4:27 PM
Subject: [Aavso-photometry] CCD Linearity testing

> I'm doing more detailed/denser evaluations of CCD linearity.  It's easier 
> these days with scripts and good software.
>
> And, working with the folks that make Photometrica software...we may use 
> this info to linearize CCD output and improve photometry data quality. 
> Maybe this can be applied to ABG CCD's?...at least up to a point?
>
> Below is an email exchange I'm having with Richard Berry.
>
> If anyone wants the spreadsheet (about 250Kb size) of my first test-run, 
> just ask.  (Same for the Orchestrate script I used to take the light 
> frames.)
>
> I'm open to suggestions on how to do this analysis better.
>
> Thanks in advance,
> Tom Krajci
> Cloudcroft, NM. 9,440FT
>
> ===========================
> Tom Krajci wrote:
>> Attached is my first attempt at dense testing of CCD linearity.  You'll 
>> see the sequence of exposure times in the spreadsheet...from 0.5 to 80 
>> seconds...covering about 55 different exposure times in a geometric 
>> progression...stepping up/down the exposure times in an alternating 
>> pattern similar to that described in the AIP4WIN Handbook.  I took almost 
>> 1000 light frames through the night...repeating the exposure sequence 
>> until about 530AM.
>>
>> AIP4WIN's Series Analysis Tool examined a large ROI on the frames (about 
>> 80% of the total pixel count).
>>
>> This CCD is a new ST-8, bin 1x1, -10C.  Using AIP4WIN's Characterize 
>> Camera routine (and measuring multiple sets of bias/flat/dark frames), I 
>> get read noise of 18.9e-, and gain of 2.72e-/ADU.
>>
>> The spreadsheet plot of exposure time versus ADU rate looks pretty 
>> consistent   The light source seems to remain stable to a little less 
>> than the 1% level, and repeated measurements may help improve confidence 
>> below the 1/2% level.
>>
>> If you want a simple number, it looks like this CCD is linear up to about 
>> 44,000ADU...typical for this camera/bin mode.  But if you want to try 
>> your hand at fancy curve fitting...I think this CCD can be used reliably 
>> up to about 58,000ADU.  (I've glanced at a few of the longest exposure 
>> time light frames that had average values near 61,000ADU...you can see 
>> that some pixels are saturating, the frame looks streaked/mottled...I 
>> would not trust the statistics for these images, they are probably very 
>> deviant.)
>>
>> Note that I *did* use auto dark matching in this test run.  However, my 
>> 120-second dark frames have the extreme hottest pixel at about 
>> 26,000ADU...this is well within the linear limit of this CCD.  I hope 
>> this means that dark scale matching worked well for all but the longest 
>> exposure light frames.
>>
>> I have not tested/evaluated residual bulk image effects.  If this ST-8 is 
>> like the ST-7 and ST-10 I've been using recently...it has this 
>> problem...but then, what CCD does not have this problem?  (How much does 
>> residual bulk image effect spoil my measurements?  Is it a factor only at 
>> the lowest light levels?)
>>
>> How can I improve upon what I've done?  It's monsoon...there are many 
>> nights of CCD testing remaining.
>>
>> Tom Krajci
>> Cloudcroft, NM. 9,440FT
>> ==============================
>> Richard Berry wrote:
>>> Hi Tom--
>>>
>>> What you're suggesting seems reasonable. Several comments:
>>>
>>> 1.) Test the whole procedure on a small number of images at each 
>>> exposure, perhaps 10, and check that it all works as planned.
>>> 2.) Don't do automatic dark matching, use matching dark exposure times. 
>>> Dark matching makes assumptions about the linearity of hot pixels with 
>>> integration time, and that's not necessarily true when the ratio of 
>>> exposure times becomes large, like 120-second darks with 0.5-second 
>>> flats. Use 0.5-seconds with 0.5-seconds, and so on for all exposure 
>>> times.
>>> 3). Instead of a shooting 100 each of time series like 0.5, 1, 2, 4, 7, 
>>> 10, 14, 20, 28, etc., space the exposures closer together--steps of 1.2 
>>> times--and shoot maybe 32 at each time. Same total number of exposures, 
>>> but it will fill the linearity curve with more points.
>>> 4). Check for residual bulk image effects. With my KAF3200, when I shoot 
>>> flats and flat-darks, the darks have higher mean levels when I 
>>> interleave the flat-darks with flats than when I shoot them separately, 
>>> and the flat-darks sometimes display patterns especially following the 
>>> I-filter flat exposure.
>>> 5.) Test the constant light source by including exposures of the same 
>>> duration in every data set. LED output depends on temperature, and LEDs 
>>> lose output over long run times. It's very hard to make a light source 
>>> constant. If you can afford to, the standard method now is to use a 
>>> pulsed LED, and use the number of pulses as the intensity of the light 
>>> falling on the CCD, so 50 pulses is 50 times more light than 1 pulse. 
>>> You can buy standard sources like this for $$$.
>>> 6.) Use a large ROI in the middle of the frame. Exposure will be more 
>>> uniform, and the edges of CCDs are usually not as free form defects, 
>>> etc. as the middle.
>>>
>>> --Richard
>>>
>>>
>>>
>>> Tom Krajci wrote:
>>>> Hi Richard,
>>>>
>>>> It's monsoon - I am going to take my CCD linearity testing to the next 
>>>> level.
>>>>
>>>> Here's what I'd like to do:
>>>> - set the CCD to one temperature
>>>> - set a constant light source to give about 8,000ADU in a ten-second 
>>>> exposure
>>>> - take a bunch of flat-light frames of varying exposure time...from 1/2 
>>>> to 80 - 100 seconds...for most of the night...hundreds of them, running 
>>>> a camera script
>>>> - take a bunch of bias frames to make a master bias
>>>> - take a bunch of 120-second darks to make a master dark
>>>> - make a 'dummy' master flat frame...same x/y size as the other 
>>>> frames...just make all the pixels the same value, such as 1.000000
>>>>
>>>> Set up advanced calibration in AIP4WIN...master bias, dark, and dummy 
>>>> flat.  (My hope is to bias subtract and dark-match-subtract all the 
>>>> various flat exposure times...but not apply any flat frame correction, 
>>>> hence the dummy flat.)
>>>>
>>>> Multi-Image | Auto-Process (calibrate only) all the flat light 
>>>> frames...save the calibrated versions of them in a new directory
>>>>
>>>> Measure | Series Analysis Tool...for all these calibrated images 
>>>> (entire image, or a large ROI in the middle of the frame?)
>>>>
>>>> Plot the results in a spreadsheet...determining ADU rate from average 
>>>> pixel value and exposure time.
>>>>
>>>> Am I making any mistakes in my protocol?  Are there any ways I can 
>>>> streamline this process further?
>>>>
>>>> Thanks in advance,
>>>> Tom
>>>>
>
> _______________________________________________
>
> Aavso-photometry mailing list
> Aavso-photometry at mira.aavso.org
> To change options or unsubscribe, goto 
> http://www.aavso.org/mailman/listinfo/aavso-photometry
>
> The Santa Barbara Instrument Group (www.sbig.com): Take the next step. 
> SGS,
> the self-guiding spectrograph. 