Have I wasted my time and money?

Affiliation
American Association of Variable Star Observers (AAVSO)
Tue, 07/11/2017 - 02:40

Hi guys,

I joined AAVSO a few days ago, partly because I'd heard of this wonderful VPHOT software. So far I am terribly disappointed. It has taken 2 days to even manage to upload an image, and I still can't do anything with it, it just shows white and won't let me open comparison star catalogue or anything.

Before I waste any more of my life on this, am I just in the wrong place entirely? I'm imaging with a Nikon D3200 DSLR on the back of a Skywatcher 100ED refractor, I have nice enough images. But I can't get anywhere in VPHOT.

Is the program only for CCD imagers? When I try to upload my FITS images, for example, I leave the filters blank, 'cos I'm not using any.

Got as far as shrinking//processing my images in other software and uploading them to VPHOT, even WCS box is ticked on my last effort (not Cal box though), but then nothing.

Where is the user guide/getting started guide? The one on the website is a joke?

Thanks, Phil, Salisbury, UK.

 

Affiliation
American Association of Variable Star Observers (AAVSO)
DSLR and VPHOT

Gee, I don't know if they go together. I have been meaning to try it someday. Does the DSLR manual offer any insites to make images work for VPHOT?

https://www.aavso.org/dslr-observing-manual

The thing that I have learned over the years is that it takes a few months of tinkering to make all the systems work together reliably. Hang in there. Also CHOICE courses are useful.

Looks like there are CHOICE courses for both DSLR photometry and VPHOT comming up soon:

https://www.aavso.org/choice-course-schedule-2017-non-member

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Uh, no it doesn't :-) But

Uh, no it doesn't :-) But everyone on the web said that VPHOT was the "state-of-the-art", and surely if I can take a good quality image of a variable star with plenty of comparison stars in the FOV it should be able to give some results?

 

Affiliation
American Association of Variable Star Observers (AAVSO)
VPHOT can easily do that.

VPHOT can easily do that. Just provide it with a calibrated and plate-solved image of a known variable, use comps that have colors that match your filters (many comps don't have all UVBRI) and you should get a result. Perhaps the problem is that the DSLR filters don't appear in VPHOT, VSP, LCG. That is the thing that stops me from going to an inexpensive DSLR. It seems there is a way to transform from TB, TG, TR to BVR. Maybe that's the problem to solve. Don't know if  there is  some literature on that. Maybe it's just as simple as treating the DSLR as any other photo-sensor, then using a standard V filter and transforming it against M67.  Pages 70-73 of the DSLR manual indicate that you can use the Green channel if you understand the star and comp's spectra so to do photometry only on fields approprate to DSLR. That works when the different bandpasses  of G and V are minimized by using stars that look more like a black-body.

It takes me lots of  time to figure these things out.

Affiliation
American Association of Variable Star Observers (AAVSO)
Blank image in VPhot

Your image is showing blank in VPhot because its not an image type appropriate for photometry; it's some kind of 3d luminance image. Please read the DSLR manual: https://www.aavso.org/dslr-observing-manual .

VPhot is more optimized for CCD images, but can handle DSLR images. These DSLR images need to be raw, not processed, and they need to be de-bayered to isolate the colors.

Affiliation
American Association of Variable Star Observers (AAVSO)
Thanks for that guys.

Thanks for that guys.

I'm really lost now! What I'm doing at the moment is taking a single raw image, saving it as a TIFimage in Deep Sky Stacker ('cos that's the only way I know to convert the raw to TIF), cropping it, then plate-solving on Astronometry.net, then downloading the FITS file from there. I then use a FITS header editing program to add the missing DATE-OBS and EXPOSURE/EXP-TIME headers fields. 

I then load that FITS file into VPHOT with no filter selected (I have created my own telescope).

As I say, I've got as far as getting it to load and the WCS box is green, the CAL box is red. Any ideas where I go from here?

Many thanks, Phil.

 

 

 

Affiliation
American Association of Variable Star Observers (AAVSO)
You are “not lost now”

You are “not lost now”, you just need more information and education in what you are trying to do. To upload some images in VPHOT and to get the scientific results automatically is my dream too...

Try to get more information and skills first. One of the possibilities is to participate to CHOICE courses as,

September 4 – 28: DSLR Photometry

October 2 - November 3: Photometry Using VPhot

Then you will be free to decide whether you wasted your time and money.

Velimir

Affiliation
American Association of Variable Star Observers (AAVSO)
hello. another option is to

hello. another option is to use IRIS which is also described in the DSLR manual. I use IRIS along with DSLR and get great results. if you need help with IRIS let me know and i can walk you through how i use it

Affiliation
American Association of Variable Star Observers (AAVSO)
Encouragement for Phil

Phil -

I hope you have found the answers and insight here to be helpful. There is indeed a learning curve when processing images. The suggestion of the CHOICE courses is an excellent one. You not only get direction from an "expert" but work through specific examples with the insight and help (and even temporary frustrations) of others who are in the same boat as you.

To everyone who has offered advice to Phil - THANK YOU! You demonstrate why the AAVSO is such an amazing organization.

 

KL

Affiliation
American Association of Variable Star Observers (AAVSO)
Uh, thanks for that guys, but

Uh, thanks for that guys, but no-one has actually helped me to go forward from where I am at the moment! The AAVSO DSLR manual does not explain how to use VPHOT. Using IRIS rather defeats the object of me joining to get access to VPHOT in the first place! Just stuck getting started really :-)

Affiliation
American Association of Variable Star Observers (AAVSO)
Hi Phil!

Hi Phil, send me one picture (raw) let me see if I can come up with a workflow for you (or not) 

my email is 

pablotwa@earthlink.net

I also recently had all kinds of problems with Vphot and I was able to work them out.

I'm a recent newbie like you and I feel your pain

Just ONE picture raw in it's original format.

best!

 

Pablo

Affiliation
American Association of Variable Star Observers (AAVSO)
This manual does not provide guidance for any single software...

Ladies and Gentlemen from the DSLR Manual "Because software is constantly changing, this manual does not provide guidance for any single software package."

Phil Edmonds problem is just that, he's using a "mechanical" intervalometer and no computer so his RAW images do NOT have a properly formatted header or a header at all...

I respectfully suggest that we create a forum specifically designed for image capture programs that will work with Vphot. On that Forum members can share their experiences with different Capture techniques/software that will allow a new member to create properly formatted subs. I use Sequence generator pro to capture and it took me a while until I was able to set it up to generate subs with the proper header , and it was not easy. 

I'm sure there are many members who use capture software packages other than Maxim DL.....There's Sharp Cap, Atik, Orion,ZWO,SBIG, Backyard EOS, and virtually hundreds of other capture software. Everything should be explored so when you have a new member with little experience we can point her/him in the right direction right away minimizing frustration.

I recommended to Phil tthat he initially should get get Sequence Generator Pro 45 day free trial  because that's what I know and he's building an Observatory and will be able to use all of the features eventually,

Maybe there are easier techniques, free software easier header editors to modify a series of headers at the same time?

Who knows, a forum specifically geared for capture software and header formatting techniques for beginners I think would be useful.

Best!

Pablo Lewin

Affiliation
American Association of Variable Star Observers (AAVSO)
There are open source

There are open source software packages like UFRaw that build on DCRAW (so a wide range of RAW formats are supported) and can output FITS after all sorts of image manipulation. What is needed here is just de-Bayer (*without* fancy whitebalance) and splitting into color channels, plus setting the FITS headers right (= as  expected by VPHOT). It should be possible to build a stripped-down, easy to use version of this software that does just that with batches of raws. Once you have batch-converted Raws to Fits (with correct header infos), ther rest of the workflow (including calibration, plate solving etc) should be identical to a CCD workflow. I do something similar using IRIS ( http://www.astrosurf.com/buil/us/iris/iris.htm ), but I'll admit it has a bit of a peculiar user interface.

Fitswork ( http://www.fitswork.de/software/softw_en.php ) is another package that can do batch processing on Raws and can generate (obviously) fits.

If you stack your images with third party software (and I would be a bit careful with DSS because it has so many options that are tuned towards "pretty pictures", make sure you don't use anything that screws up the photometry by doing anthing more "smart" than adding/averaging pixel values) , you might not want to bother with the FITS headers for the individual frames and just set them manually on the end-product, the calibrated and stacked frame(s), before uploading to VPHOT.

CS HBE

Affiliation
American Association of Variable Star Observers (AAVSO)
Fits are not his problem, fits headers are...

The main challenge with vphot is getting the fits header to speak the same language as Vphot. Making fits it's easy, getting the header to be properly formatted irrespective of the software you use to convert RAW to fits now that is his problem. There should be a "Quick Start" procedure/software/class/centralized forum to properly format  ANY fits header irrespective of whether CCD/RAW so it will conform to Vphot specs...A sort of unversal translator from fits to Vphot...

 

Affiliation
American Association of Variable Star Observers (AAVSO)
AstroImageJ

Hello Phil, I guess, for those things you're looking for (debayering + FITS updates) you could use AstroImageJ (see: http://www.astro.louisville.edu/software/astroimagej/). It is a Java software and therefore runs more or less on any platform. It also does pre-processing for you more or less in automated way... And last but not least, it does also very high-quality photometry.

Best wishes,
Tõnis Eenmäe

Affiliation
American Association of Variable Star Observers (AAVSO)
Hi Tonis,

Hi Tonis,

Many thanks for this, it has sent me off on a new direction and I am spending time exploring the capabilities of AstroImageJ. This looks more promising, at least it can load and plate-solve (via astronometry.net) my images. Many thanks for the tip :-)

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Many thanks guys, lots of

Many thanks guys, lots of useful stuff here, and a lot to take in.

I'm a complete beginner at this, so please bear with me! I think I now know how to convert my images (one way or another) to FITS files, and also that I must have the correct information in the FITS header, I have found a utility to allow me to edit the FITS header, so some progress!

But there is more fundamental stuff I don't understand. I do understand the concept of the Bayer matrix, but what is "de-bayering" and how do I do it to one of my DSLR images? Is the aim to produce separate files for the R,G and B channels and then load them separately into Vphot and stack them together there?

Many thanks, Phil :-)

Affiliation
American Association of Variable Star Observers (AAVSO)
A few years ago, I helped

A few years ago, I helped some with the VPHOT project and I unclogged the server using an RDC many times.  The biggest problem was bad headers in the FITS files.  Unfortunately, I haven't had time to help out for the past couple of years.

AAVSO is a wonderful group.  I would point out that one really has to enjoy working at these things in order to get anything out of it.  Many of the things we do are technically challenging and it's not unusual for people to keep working on their equipment and software all the time even after they get to the point where they can generate data.

Glen

Affiliation
American Association of Variable Star Observers (AAVSO)
But there is more fundamental

But there is more fundamental stuff I don't understand. I do understand the concept of the Bayer matrix, but what is "de-bayering" and how do I do it to one of my DSLR images? Is the aim to produce separate files for the R,G and B channels and then load them separately into Vphot and stack them together there?

So as you know, Raw frames include information from several color channels interlaced in a single image, where the pixel position can be used to reconstruct whether a given pixel is recording Red, Green or Blue color. For photometry, you want to measure a star's brightness separately in a given color (filter), so when doing aperture photometry as VPHOT does, you want all the pixels in the measurement aperture to represent the same color, and not measure a mix of colors. That is important. The code of the filter will be part of the FITS header of the image given to VPHOT.

So you are right, at some stage you need to split the interlaced, all-color-at-once "Bayer filter" image into separate color channels.

If the software that you use for stacking can handle Bayer-Filter frames (e.g. if it allows to stack Raw images without going thru intermediate file formats), then you only have to do this color separation once, after stacking. If your stacking software is not capable of stacking raws directly , you need to "de-bayer"  the individual frames and then stack the converted frames.

Most of the time, the way software deals with Bayer-filtered RAW images to split them into separate color channels is a 2 step process:

1) Take the raw image and convert it into a RGB image of the same pixel dimensions (!) but with each pixel having all three RGB color informations. This is the de-bayering mentioned before.

2) Take the RGB image and split it into three images , one for each color, again maintaining the image pixel size.

There are a few catches here with step 1).

For simplicity, let's assume you have a fiducial 1 Mega Pixel sensor: 1024 x 1024 pixels. Because of the Bayer-filter, usually half of them are recording the green channel , 1/4th are for red and 1/4th are for blue. The de-bayering will generate an image that has the same dimensions, so the output image will have 1024x1024 pixels EACH with RGB info, that is MORE information than in the original raw image. The way this works is that for each pixel, the selected de-bayering algorithm (there are usually several alternatives) will try to guess reasonable values for all three color values at that pixel position by looking at the Bayer-filter pixels of that color that are either at this very pixel position (if that happens to be the right color) or neighboring it (this guessing is called interpolation). There are several algorithms for this interpolation, some are more tuned to "pretty pictures" than faithfully preserving the original information. You should always select the method that has "linear" in it's name, not the fancy ones, for photometry.

Another trap you can fall into  is that during this step, software usually performs the "white balance" correction of the image, sometimes automagically behind the scenes using the whitebalance setting that was active on your camera during exposure. This setting is saved in the RAW file als meta-data, it doesn't have an influence on the pixel values in the raw. But the Debayer-step might apply the white balance and re-mix the colors in your image, which is something you do NOT want, so one has to be careful to configure the software to not perform this white-balance color shifting during de-bayering.  

Another drawback of this 2-step process (devbayer, then split) is that your images are needlessly large (because the interpolated color pixels are included) and processing takes longer.

Some software packages (like IRIS, see above) have functions that allow you to split the raw directly into color images without any interpolation or white balance, which is really what you would want. So IRIS can convert our 1024x1024 pixel RAW into 4 images: one 512x512 Red, one 512x512 blue , and two (!) 512x512 Green channel images. So you have less data to process, but the disadvantage of having two separate green channel sub-frames per raw frame to deal with.

CS

HBE

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Thank you so much for this,

Thank you so much for this, it must have taken you ages to write it!

Your post may put me on the right track, I think I can now see what I have to do to get started. I'm going to analyse your post in more detail over the next few days, but I think it's pointing me towards where I need to go next.

Cheers, Phil :-)

 

Affiliation
American Association of Variable Star Observers (AAVSO)
VPHOT

phil

YES  you can upload DSLR image to VPHOT as long as image is plate solved(PinPoint etc) and other fields on header are edited.  Succeeded last night in uploading DSLR image to VPHOT. I also image with DSLR and have been for 7 years.  would be glad to share my experiences, off line.  my email address is Mikerph1951@gmail.com