Skip to main content

Need advice on image processing pipeline

KTC's picture
KTC
Offline
Joined: 2010-12-08

I'm setting up more non-AAVSONet scopes here, which means that I must do my own calibration and processing of images. 

My current methods are not what you could call automated.  (Example:  AIP4WIN, unlike Maxim, won't calibrate an entire folder...looking at FITS header filter names and applying the correct flat frame.  I gotta process all B images, then manually select all V images and process them with a different master flat, etc.) 

Also, if I want to upload processed images to VPHOT, I need to add some FITS keywords with the right values...or VPHOT is not happy.  (Some examples:  TELESCOP = name1, CALSTAT = BDF, USERNAME = name2).

What advice can you give me?  Can Windows software can do this in a streamlined, automated way?  Or should I man up, get a Linux box and IRAF and read the 1500 page manual and build routines/libraries/scripts from scratch?  Or is there another way?

Thank you in advance.

Hi Tom, I got started using
b3davies's picture
b3davies
Offline
Joined: 2010-12-23

Hi Tom,

I got started using IDL because that is what San Francisco State encourages it's students to use.  A student license is $115 per year and a permanent license is $2,500.  The premanent license is expensive, but it is also cheap; depends on how much use you make of it.

I observe fast varying bright stars (~mag 6) and get between 1,000 and 1,600 images a night.  I've written a pipeline in IDL that allows me to throw all images into a single folder and push the buton, so to speak.  It reads the names and headers and sorts for filters, time of exposure, targets, calibrates and does aperture photometry.  I have a number of other procedures for inspecting the images, batch renaming of files, determining extinction coefficients, plotting orbital paramaters, and an embarassingly long list of other stuff that indicates entirely too much free time devoted to fooling around.

I'd be happy to share this stuff as it is pretty well generalized and the parts that aren't would be easy to fix.  Also IDL comes with the ability to export compiled programs and provides a free engine to run them.

I should mention that some of the professors at SFSU have IDL available but nevertheless do thier work using IRAF inside a Python wrapper and are very happy with that.  But those guys are more than smart and do this stuff for a living.  Comparatively, IDL is dead easy, equally powerful as IRAF, and intuitive - particlarly if you've ever written any Fortran code.  One thing I really like about IDL is that the code is completely transparent.  No black boxes.

Ben Davies

Image processing versus image analysis
Matthew Templeton's picture
Matthew Templeton
Offline
Joined: 2010-03-12

Hi Tom,

First, I can confirm what Ben said above: IDL is a nice software package, and the Astrolib package is a nice tool. I can also confirm that it's exhorbitantly expensive.  The AAVSO Chart Plotter used to be written in IDL -- one of the reasons we contracted Clockwork to write a new VSP in PHP was because of the licensing cost.  You might be able to compile Astrolib with GDL: http://gnudatalanguage.sourceforge.net/

Another possible no-cost alternative is CFITSIO, and in particular the Perl interface for it.  Perl makes file-handling incredibly easy, and once you learn FITSIO you should be able to do *basic* file processing (e.g. anything that involves arithmetic operations on whole frames like flatting and bias/dark subtraction) easily enough.

Honestly, IRAF isn't terribly hard for basic operations either.  Where IRAF starts to get difficult is in doing automated photometry (like DAOPHOT) or spectrum reductions.  Basic operations like flat-fielding and image combinations are straightforward.  And for Macintosh at least (and I assume Linux now) IRAF installation isn't anywhere near the agony it used to be.

For now it's Maxim or CCDSoft
KTC's picture
KTC
Offline
Joined: 2010-12-08

Thank you to all those that gave advice, on this forum, via private email, etc.

For now I'll use either Maxim or CCDSoft for the processing pipeline.  That's because my current tasks are rather simple:  calibrate (bias, dark, flat), and add a small number of static/fixed FITS keywords to make VPHOT happy during bulk upload.

If my tasks change, I'll reassess, and may use other software.

If the number of images per night starts to bury my current computer, I may look for a faster processor.

Thanks again.

Software
TCB168's picture
TCB168
Offline
Joined: 2010-07-26

KTC wrote:

Thank you to all those that gave advice, on this forum, via private email, etc.

For now I'll use either Maxim or CCDSoft for the processing pipeline.  That's because my current tasks are rather simple:  calibrate (bias, dark, flat), and add a small number of static/fixed FITS keywords to make VPHOT happy during bulk upload.

If my tasks change, I'll reassess, and may use other software.

If the number of images per night starts to bury my current computer, I may look for a faster processor.

Thanks again.

The other option is MIRA. Between the price of the 2 suggested above and does preprocessing very well. Also good for photometry but doesn't make the best AAVSO files.

Cheers

 

Terry

AAVSO 49 Bay State Rd. Cambridge, MA 02138 aavso@aavso.org 617-354-0484