Software for photometry (image calibration)

Affiliation
British Astronomical Association, Variable Star Section (BAA-VSS)
Wed, 11/01/2023 - 21:16

I am making some headway into this complicated (for me) CMOS photometry.
Now I need software to calibrate raw FITS file photometry images.
There is much advice online. Most is centered around taking fantastic images.


Is there anything (perhaps user friendly for this old brain and less complicated)
suitable for calibration of images for photometry.
Everybody has their favourite so quite happy to take a poll.
I am in the AAVSO and they have some great stuff but as far as I can see, oddly nothing for this.
I have downloaded AIP4WIN initially as a viewer for FITS images, but can't seem to get far with it.
Thanks again for invaluable advice on these forums
Kevin

Affiliation
American Association of Variable Star Observers (AAVSO)
AstroimageJ

Hi Kevin I was in the class you just took. AstroimageJ is the software to use.  This youtube link (https://www.youtube.com/watch?v=GW--rE5O-c8) provides all the info you need. Note, AstroimageJ is what the AAVSO Exoplanet team uses.  I have a good background in astrophotography (pretty pictures) and use
only Pixinsight, but was cautioned against using it for photometry.

I used to AstroimageJ, this week, to calibrate my own images of NGC7790 and applied the coefficients to my telescope system.

If you already own MaximDL that will work also. AstroimageJ if FREE.

Steve HSTG

Affiliation
British Astronomical Association, Variable Star Section (BAA-VSS)
Thanks

Hello Steve,

Thanks for the information.

Calibration using Standard Fileds for filter/color transformation and extinction coefficient will be essential.

Does it also do calibration of raw light images by application of flats, darks, flat darks, and bias frames?

Thanks

Kevin

Affiliation
American Association of Variable Star Observers (AAVSO)
it calibrates your lights,darks,flats and bias frames

Be sure to use ASTAP too, it's an essencial step and it's FREE software.

Steve

Affiliation
American Association of Variable Star Observers (AAVSO)
AstroimageJ can plate solve your light frames but its painfully

SLOW.

Additionally ASTAP rearranges my FITS headers and makes them better for VPHOT to interpret. I will attach before and after ASTAP FITS headers screenshots for you to examine.

On second thought I won't because unlike the AAVSO course we just took I cannot attach images here. If you ask for them and your email address is visible to AAVSO members I will send them to you via email.

 

Steve

Affiliation
American Association of Variable Star Observers (AAVSO)
A point of clarification.

In AstroImageJ I only calibrate my lights,darks,flats and bias frames. Then load the calibrated images into VPHOT and create the Sequence, loading Comps, Target, assigning Check ect.

And again I plate solve my images with ASTAP as the guy in the AstroimageJ video suggests.

Also the TG program makes creating coefficients childs play.

 

Steve

  

Affiliation
American Association of Variable Star Observers (AAVSO)
ASTAP and FITS headers?

Steve,

I'm an extensive user of AIJ and have good results stacking and plate solving with AIJ.  If a file doesn't plate solve, sometimes a double take is necessary.  Normally I will calibrate with AIJ and also do multiple aperture photometry with AIJ as the results can be put into a measurements file and later easily inserted into a spreadsheet for subsequent processing and storage.  And yes I will indeed put the calibrated files into VPhot for AAVSO reports but VPhot doesn't produce as much info as AIJ. 

The only problem I've had with AIJ and FITS headers is that neither the filter or calibration status isn't available for VPhot, at least not in keywords that it recognizes.  Calibration status doesn't matter as it's not reported anyway, and the filter can be set prior to doing VPhot photometry.  Does ASTAP resolve these header issues?  I've thought about using ASTAP, but thought it would be redundant.

Clear skies,

Steve

Affiliation
American Association of Variable Star Observers (AAVSO)
CALSTAT

which coincides with the CAL column in VPHOT is a field limited to users of MaximDL. I know of no other imaging/calibration package that populates this FITS header field. I had an email conversation with Ken and he said it can be ignored, that everyone assumes your data is calibrated.

Maybe the AstroImageJ developers could be persuaded to add this field and populate it too?

As for the filter, SGP populates FITS header fields FWHEEL (CFW-5-7) and FILTER ('B       ') and I never have to do anything to it.

 

Steve 

Affiliation
American Association of Variable Star Observers (AAVSO)
which coincides with the CAL…

which coincides with the CAL column in VPHOT is a field limited to users of MaximDL. I know of no other imaging/calibration package that populates this FITS header field.

ASTAP populates the CALSTAT FITS property.  VPhot seems to use it correctly

Affiliation
British Astronomical Association, Variable Star Section (BAA-VSS)
Flat darks

I have installed AstroImageJ, found the DP button but there is no option to create flat darks.

I have been advised to calibrate my raw lights using darks, flats and flat darks (not use bias frames for CMOS).

The windows are pretty overwhelming for info.

So I have found various tutorials but I'm getting the impression they are for CCDs which do use bias frames I believe.

Any help appreciated.

Kevin

 

Affiliation
American Association of Variable Star Observers (AAVSO)
I was on a ladder collimating my telescope before

I have a solid astrophotography background, see my astrobin page - https://www.astrobin.com/users/sink45ny/
I have a mono CMOS camera - QHY600MM
I calibrate my light frames with BIAS, Flats and Darks.
I have good pixInsight experience but have been cautioned not to calibrate with it by an AAVSO teacher.
It was recommended I use AstroImageJ for my AAVSO work, so I do.
My imaging software is Sequence Generator pro SGP. 
I use ASTAP to plate solve my light frames like the following youtube recommends.
https://www.youtube.com/watch?v=GW--rE5O-c8
I do not use AstroImageJ to set my apeture.
I use VPHOT to set Apeture,annalus and sky annalus.
I use VPHOT to create the sequences.
I have successfully imaged NGC7790, this past week, with BVRI filters and calibrated them with process mentioned above.
The images were upload to AAVSO.
Using VPHOT as mentioned above a sequence for my NGC7790 images was created and a VPHOT Time Series created.
The Time Series was exported and imported into the AAVSO Transform Generator (TG windows executable program), which produced an INI file containing the coefficients for my system.
The INI coefficients was imported and applied to the AAVSO telescope I defined.

If you want explain to me where you are in relation to the stuff I described.

Steve

Kevin,

I use AstroimageJ…

Kevin,

I use AstroimageJ for callibration and photometry of time series images take with a mono CMOS ZWO ASI camera (like yours). I use darks and flats, and never use bias frames or flat darks. AstroimageJ is fine for both CCD and mono CMOS cameras.

Roy

Affiliation
American Association of Variable Star Observers (AAVSO)
 Of course! You might want…

 Of course! You might want to look into options such as AstroJ Image or MaximImageDL if you're looking for easy to use photometric image calibration software. It is more user-friendly and can make the calibration process easier. Although AIP4WIN is an appropriate choice, the alternatives allow a better experience. You can always reach out to a custom software development agency if you run into any problems.

Affiliation
British Astronomical Association, Variable Star Section (BAA-VSS)
Update

Update
I threw caution to the wind and ignored the Defender warning for Astap.

After I have had literally a play with the following with a view to just being able to just calibrate.
AstroimageJ
TychoTracker
DeepSkyStacker
AIP4Windows
Astap

ASTAP seems at first to be the most user friendly for someone of my limited experience of image processing and has
a dedicated routine for flat darks which seems to be missing from the others.
I loaded some lights, flats, darks and flat darks and ran the program which seemed to work(Furthest I have got so far).
It output a pretty nice "looking" (but huge image 62Meg)
I have attached the text of the file file that appeared in the window.
I assume this is the FITS header. Only concern is the opening line which states:

SIMPLE  =                    T / file does conform to FITS standard             
Which I know is a bit of a deal breaker as they have to conform. So that's where I am. Any thoughts please?

I'm sure that the others may prove much more powerful and useful when I am a bit more advanced.
The problem for me is that the more advanced and versatile the software the more parameters there are to play with.
Thanks to everyone for useful input for future plans. I will almost certainly ask some more questions.
Kevin

*********************************TEXT OF FILE below************************************************

SIMPLE  =                    T / file does conform to FITS standard            

BITPIX  =                  -32 / number of bits per data pixel                 

NAXIS   =                    2 / number of data axes                           

NAXIS1  =                 4656 / length of data axis 1                         

NAXIS2  =                 3520 / length of data axis 2                         

EXTEND  =                    T / FITS dataset may contain extensions           

COMMENT   FITS (Flexible Image Transport System) format is defined in 'Astronomy

COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H

BZERO   =                    0 / offset data range to that of unsigned short   

BSCALE  =                    1 / default scaling factor                        

CREATOR = 'ZWO ASIAIR Plus'    / Capture software                              

OFFSET  =                   50 / camera offset                                 

XORGSUBF=                    0 / Subframe X position in binned pixels          

YORGSUBF=                    0 / Subframe Y position in binned pixels          

FOCALLEN=                  597 / Focal length of telescope in mm               

EGAIN   =     4.96000003814697 / Electronic gain in e-/ADU                      

XBINNING=                    1 / Camera X Bin                                  

YBINNING=                    1 / Camera Y Bin                                  

CCDXBIN =                    1 / Camera X Bin                                   

CCDYBIN =                    1 / Camera Y Bin                                  

XPIXSZ  =     3.79999995231628 / pixel size in microns (with binning)          

YPIXSZ  =     3.79999995231628 / pixel size in microns (with binning)          

IMAGETYP= 'Light   '           / Type of image                                 

RA      =             359.9055 / Object Right Ascension in degrees             

DEC     =              61.3554 / Object Declination in degrees                 

DATE-OBS= '2023-10-22T19:50:58'/ Date and time of the start of the observation.

FILTER  = '5V      '           / Filter used when taking image                 

INSTRUME= 'ZWO ASI1600MM Pro'  / Camera model                                  

GUIDECAM= 'ZWO ASI120MM Mini'  / Guide camera model                            

GAIN    =                    0 / Gain Value                                    

FOCUSPOS=                51183 / Focuser position in steps                     

TELESCOP= 'EQMod Mount'        / Telescope name                                

CTYPE1  = 'RA---TAN-SIP'       / TAN (gnomic) projection + SIP distortions     

CTYPE2  = 'DEC--TAN-SIP'       / TAN (gnomic) projection + SIP distortions     

CRVAL1  =      0.0388451298405 / RA  of reference point                        

CRVAL2  =        61.1352115421 / DEC of reference point                        

CRPIX1  =        1752.88039144 / X reference pixel                             

CRPIX2  =        2003.05200195 / Y reference pixel                             

CD1_1   =   -0.000364622297243 / Transformation matrix                         

CD1_2   =   -2.20059030484E-06 / no comment                                    

CD2_1   =    2.29198883849E-06 / no comment                                     

CD2_2   =   -0.000364661707781 / no comment                                    

A_ORDER =                    2 / Polynomial order, axis 1                      

B_ORDER =                    2 / Polynomial order, axis 2                       

AP_ORDER=                    2 / Inv polynomial order, axis 1                  

BP_ORDER=                    2 / Inv polynomial order, axis 2                  

A_0_0   =                    0 / no comment                                    

A_0_1   =                    0 / no comment                                    

A_0_2   =   -1.47621961487E-07 / no comment                                    

A_1_0   =                    0 / no comment                                    

A_1_1   =   -3.75089029194E-08 / no comment                                    

A_2_0   =    1.75644526303E-07 / no comment                                    

B_0_0   =                    0 / no comment                                    

B_0_1   =                    0 / no comment                                    

B_0_2   =   -3.90101991191E-08 / no comment                                    

B_1_0   =                    0 / no comment                                    

B_1_1   =    1.08100451613E-07 / no comment                                    

B_2_0   =   -5.73414276682E-08 / no comment                                    

AP_0_0  =   -6.14189484914E-05 / no comment                                    

AP_0_1  =   -2.85105297785E-09 / no comment                                    

AP_0_2  =     1.4756298653E-07 / no comment                                    

AP_1_0  =    7.17213298529E-08 / no comment                                    

AP_1_1  =    3.75402795125E-08 / no comment                                    

AP_2_0  =   -1.75532685305E-07 / no comment                                    

BP_0_0  =    5.67595011944E-05 / no comment                                    

BP_0_1  =    3.98326758671E-08 / no comment                                    

BP_0_2  =    3.90197129377E-08 / no comment                                    

BP_1_0  =   -5.39693815989E-08 / no comment                                    

BP_1_1  =   -1.08055171149E-07 / no comment                                     

BP_2_0  =    5.72864068235E-08 / no comment                                    

IMAGEW  =                 4656 / Image width,  in pixels.                      

IMAGEH  =                 3520 / Image height, in pixels.                       

COMMENT 1   Written by ASTAP. www.hnsky.org                                    

PEDESTAL=  0.000000000000E+000 / Value added during calibration or stacking    

CALSTAT = 'DFBS'                                                                

DATE-END= '2023-10-22T21:13:44'/ Date and time of the end of the observation.  

JD-AVG  =  2.460240355801E+006 / Julian Day of the observation mid-point.      

DATE-AVG= '2023-10-22T20:32:21'                                                

EXPTIME =                   60 / Total exposure time in seconds.               

COMMENT     D=master_dark_20x30s_at_0C_2023-10-23.fit                          

COMMENT     F=master_flat_corrected_with_flat_darks_V_20xF_20xFD_2023-10-25.fit

HISTORY 1   Stacking method AVERAGE                                            

HISTORY 2   Processed as gray scale images.                                    

SET-TEMP=                    0 / Average set temperature used for luminance.   

LUM_EXP =                   30 / Average luminance exposure time.              

LUM_CNT =                    2 / Luminance images combined.                    

LUM_DARK=                   20 / Darks used for luminance.                     

LUM_FLAT=                   20 / Flats used for luminance.                     

LUM_BIAS=                   20 / Flat-darks used for luminance.                

DATAMIN =                    0 / Minimum data value                            

DATAMAX =                65152 / Maximum data value                            

CBLACK  =                  469 / Black point used for displaying image.        

CWHITE  =                  737 / White point used for displaying the image.    

END                                                                            

Affiliation
American Association of Variable Star Observers (AAVSO)
Only concern is the opening…

Only concern is the opening line which states:

SIMPLE  =                    T / file does conform to FITS standard             
Which I know is a bit of a deal breaker as they have to conform. So that's where I am. Any thoughts please?

Are you sure you are reading the comment correctly?  It says the file *does* conform to the FITS standard.  I don't see a problem.

I have used ASTAP for calibration and stacking and never had an issue with VPhot.

Affiliation
British Astronomical Association, Variable Star Section (BAA-VSS)
Thanks

Thanks for pointing out my senior moment there.

I had been playing with various files and I felt sure that I had read one that stated that it did not conform.

Obviously not that one. That's great. Have moved forward another small step.

I think my worry was that maybe I had done something wrong.

I was expecting to have an output of master flats, master darks and master flat darks and then having to do something with them.

So the image it outputs, is that a calibrated image. Is it possible to tell?

What does the software do to the original uncalibrated lights?

I have no feel for what s going on under the bonnet.

Thanks

Kevin

Flat darks

Kevin,

If you use an artificial light source for your flats so that exposure times for your flats are short, you should not need flat darks. I use a dimmable screen on a tablet set at 20% of maximum intensity and my flats exposures are less than 1 second. I never use flat darks.

If ability to process flat darks is the only reason you are leaning toward ASTAP, make sure you really do need flat darks before your final decision.

Roy

 

Affiliation
British Astronomical Association, Variable Star Section (BAA-VSS)
Flat darks

Thanks Roy,

I have CMOS camera and was advised to take flat darks because the short(zero) exposures needed for bias frames is a problem with CMOS cameras. The advice was to instead take flat darks with the same exposure as the flats. Hence your right I did need software that offers this.

So I am initially restricted to DeepSkyStacker and ASTAP. ASTAP seemed the more intuitive of the two although as my previous post states, it would be useful to know what it's doing with the files I plug into it

I bought a light panel which (with appropriate T-shirt and sheets of paper) enabled me to take flats at about 2.3s. Again my advice was to aim for 2-3 sec exposures to avoid the problems inherent at the normal zero exposures.

So I have followed this advice, which came from BAA members. If you are interested I can post the text here.

I'm not wedded to ASTAP, rather it's the only software that at present I don't find overwhelming. (Screens full of parameters I have no knowledge of are very off-putting. I'm sure that as I get more familiar with the process I will try the more advanced software that many of you are recommending. So I do appreciate all the advice on here and will return to it. I'm very much working hand to mouth with each problem I encounter on this steep learning curve, hence the number of posts, but I think I am getting there.

Regards and thanks

Kevin

 

Flat darks

Kevin,

I'm puzzled by the advice given to you that you need flat darks when it has also been recommended that your flats exposures are only 2-3 seconds. As I understand it, the purpose of flat darks is to correct for dark current 'noise' in the flats. In such short exposures, I would have thought that was negligible, particularly in images from a cooled camera, hence my comment that flat darks should not be necessary.

As you noted, some software does not even provide the opportunity for the use of flat darks (including AIJ and AIP4Win, which I use or have used).

As to the choice of software, it makes sense to use a package you feel comfortable with.

Roy

Affiliation
American Association of Variable Star Observers (AAVSO)
AIJ Calibration

Roy:

I'm not specifically supporting subtracting flat darks in this case or not BUT:

1. IF you have to wonder " I would have thought that was negligible", would it not be more accurate to always make the correction no matter how small the correction may be? I think the answer is yes BUT when we suspect/know that a correction is very small we often ignore it only because it is easier to do! The same argument applies to extinction correction on images with a small field of view. Amateurs generally ignore it. Professionals rarely ignore it. IMHO, it unfortunately comes down to what we are willing / able to do? Note that I do personally ignore some corrections when I discover that the corrections are smaller than my normal precision.

2. You stated "some software does not even provide the opportunity for the use of flat darks (including AIJ..). Actually if you look at AIJ, there is a correction line for bias, darks and flats and darks can be scaled. It is not immediately obvious whether flats are corrected for scaled darks in AIJ but it is certainly possible. I was not able to find the help menu for this correction to confirm what is done? I'm guessing it does but perhaps someone could confirm this one way or the other?

Ken

Ken,

Thanks for your…

Ken,

Thanks for your comments (they are always useful and always provoke thought).

"... would it not be more accurate to always make the correction no matter how small the correction may be?"

You posed this as a question. My answer is "no", because of your qualification "no matter how small ...".

If a correction makes no statistical difference to the outcome, then the result is not going to be more accurate. Of course, without testing, I don't know if that's the case.

Roy

AstroimageJ and scaling of darks

Ken and all,

The following paragraph from the AstroimageJ User Guide is pertinent.

"6.3.2.3
If the scale option (for darks) is enabled, the master dark pixel values will be scaled according to the ratio of the
science image exposure time (or raw flat image exposure time in the case of master flat creation) to the
master dark exposure time. The master dark image must be bias subtracted and the exposure time must be
available in both FITS headers for this feature to work as expected. See the FITS header Updates section
for compatible exposure time keywords."

My interpretation after reading the above is that, if the "scale" option for darks is checked, the darks will be scaled not only to the science images, but scaled also to the exposures of the flats, and applied as flat darks. Note specifically, that for this to work the master dark image must be bias subtracted.

The user guide can be found by scrolling down the web page at: http://www.astro.louisville.edu/software/astroimagej/]

Final comment: if my interpretation as above is correct, it would be possible to run a test in AstroimageJ with and without the 'scale' option selected for the darks to see if there was any statistical difference in (say) the mean and standard deviation of the magnitude of the check star in a time series.

Roy

 

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Equally enjoy!

Roy:

I equally enjoy your responses!  ;-)

In fact as I mentioned, I personally make that judgement about 'how small' a correction may be. If I 'think/guess'' that it is smaller than my normal total precision, I tend to ignore the correction too. It generally (but unfortunately not always) does not make any significant difference. (I'm actually thinking about more than just calibration here.)

However, it really comes down to the fact that precision (random error) and accuracy (systematic error) are not the same. But since they do impact the total error (random and systematic), I normally use my opinion rather than fact, to take the easy way out.

Interestingly you then said "Of course, without testing, I don't know if that's the case." It begs the question: If you really don't know if it is a significant correction or not, shouldn't you do the testing OR make the correction so you don't have any doubt? Professionals like Arne Henden set up their own analysis pipeline to 'always' make all the corrections. I have always felt that we (amateurs) should also have a good commercially available photometry product that does this easily. We are just too small a niche. Oh well!  ;-(

Ken

Testing...

Ken wrote:

"Interestingly you then said "Of course, without testing, I don't know if that's the case." It begs the question: If you really don't know if it is a significant correction or not, shouldn't you do the testing OR make the correction so you don't have any doubt?"

Yes, of course, I agree completely, and you will see in my other post re AIJ and scaled darks I suggest doing just that (testing).

But in the moment, during exchanges in this Forum, the data isn't immediately available!

Roy

Affiliation
American Association of Variable Star Observers (AAVSO)
I used ASTAP before for…

I used ASTAP before for calibrating, alignment and plate solving. However, since I use a smart telescope, exposure is just limited to 4 secs - so I needed to manually stack frames for time frames of 2 or 5 minutes as sub-stacks to improve SNR prior to photometry.

I now use Tycho Tracker to preprocess my images, and do sub-stacks with photometry on them on the fly without having to manually do sub-stacks and save them on the drive - makes my workflow much faster. I still use ASTAP though to extract RGB channels, after calibrating, aligning and plate solving with Tycho. Plus, I use it to detect comets as well.

Raymund