VPhot not processing images

Affiliation
Astronomical Society of South Australia (ASSAU)
Sat, 11/01/2014 - 12:51

VPhot appears to not be processing any images. The following shows what the situation has been for a couple of hours:

Thanks.

David 

File Upload
vphot-Q.png86.98 KB
Affiliation
Astronomical Society of South Australia (ASSAU)
Stuck again?

The queue seems to have been reporting this one for awhile:

Calibrated-T31-steenodde-0020-30-20141114-222009-Luminance-BIN1-E-180-015.fit 

David

Affiliation
American Association of Variable Star Observers (AAVSO)
VPhot Seems Broken - Doesn't upload ANY images

I was having a problem yesterday that V Phot would not accept more than 3 images. I had uploaded a total of 8 it  showed only the last three images uploaded. I uploaded 4 new images and only the last 3 of the 4 showed and the earlier images were lost as well as one of the 4 in sthe second upload So I processed those three and uploaded the 4th and it replaced one of the origninal three. I processed the 4th. All of these images had full headers and were plate solved. I had checked one of them beforehand using the upload wizzard. All were from the same telescope. Each image of the 4 image set had a different filter, different time and different airmass. All of the first 8 had different times and airmasses but there were 2 each B, V, R and I. These were from a normal monochromatic CCD camera taken in a I, R, V, B, B, V, R, I set. 

Last night I deleted all images (only 3) This morning I tried uploading 4 images. The quick upload page showed they had uploaded but they didn't show up on my image list. The list was empty. I tried uploading 1 of the 4 images. Again the quick upload page told me the image had uploaded and again no images appeared on the image list. In all cases I clicked the refresh button when I didn't see the images just to be I was seeing the latest status. 

I tried a test upload of the same image using the upload wizzard and it told me everything was fine and the test image uploaded and all fields were populated correctly. 

I was having the same problem yesterday but it was intermittant. Today it is consistent. 

Guys it appears that VPhot is simply broken. Could you be out of allocated storage space or something like that? My images are  24 MByte each, which although large, isn't unusual for astro images.  

I would appreciate it if staff would look into this. 

Thanks

Brad Walter WBY

Brad, I think that the size

Brad, I think that the size limit for what VPHOT will upload is somewhere around 20 megabytes.  Your images may be close to the limit if some made it through and others disappeared.  I have run into this before.  Two possible solutions:  crop the image and upload the cropped image -or- bin the pixels 2x2 and try again.

 

 

Affiliation
American Association of Variable Star Observers (AAVSO)
VPHOT upload

Hi Brad,

There are several possible reasons why you are having difficulty, not all of them related to VPHOT.  Now that the Fall meeting is over, I'm planning on working with the VPHOT programmers to improve the uploading process.  If everyone will be patient for a week or two, I'll post with what was discussed and our interim/final solutions.  In the meantime, keep submitting images - we will be monitoring the queue on a sub-day basis and will fix any problems that might exist.

Arne

Affiliation
American Association of Variable Star Observers (AAVSO)
Uploading files

What were the file sizes and how long did it take to upload 900?

My files are 24MB each and take 5 minutes each to upload. I have the fastest DSL I can buy at 600 kbits/sec upload and T1 rates download. That is the fastest internet available at my location. Perhaps we are simply overrunning the Queue or perhaps my uploads are too slow.  I really have no idea, but it wasn't taking my images even though the test in the upload wizard said they were fine when I ran it to find out if there was something in the header info that was creating a problem and it has taken them in the past. 

Strangely my images showed up about 3 hours later. I don't know if manual intervention was involved in making that happen. the first 8 images that I uploaded yesterday never showed up. 

I am going off now to sacrifice some files to appease the data gods hoping that will satisfy them and my uploads will be welcome for a while. 

Brad Walter, WBY

Affiliation
American Association of Variable Star Observers (AAVSO)
3 Is Still the Limit

Well I tried uploading 4 x 24 MB files again this morning. The first time around they didn't upload even though the uploader said they had. The second time around only the BVR of a BVRI set loaded and when I Uploaded the I as a single image it replaced the V and the V disappeared. . I am absolutely certain that I did not upload a duplicate image. I checked the fits heaers of the BVand R images before I uploaded the I image and I checked the I image after it uploaded. They were the correct images of a BVRI set. These images are combined images of 4 BVRI sets on a single night so their times,  althought not identical are very close to each other. The field coordinates are essentially identical. The filters are different and the EXPTIMEs are different. 

Brad Walter, WBY

Affiliation
American Association of Variable Star Observers (AAVSO)
Uploading Files to VPhot

New information that may be helpful. When I was having trouble uploading the server traffic was moderate or heavy. Last night and this morning the server traffic in the upload box showed "None" and not only did the files upload but now they don't replace earlier uploaded files> I now have 12 images in my image list and it lets me upload at least 4 images at a time (I haven't had more than 4 at once to try). 

I have noticed that there can be a delay between uploading and the files showing up in the list, even when you refresh it. when there is no server traffic the delay seems to be short. With significant traffic It has taken more than half an hour. I can't say how much more. I wasn't timing the completion of upload carefully.

I have also determined that my speed of upload does not seem to be limited by the AAVSO server. Rather it is bandwidth on my internet connection. Last night when server traffic showed as "None" I could only upload at 61 kB/sec. That was early Saturday evening when there is a lot of traffic and internet movie streaming. at 4:30 a.m.. this morning the rate was back up to 76 kB/sec, the same value  as during the week when uploading to the server showing moderate or heavy traffic. 76 kB/sec agrees with the max rate on my DSL of 600 - 700 kbits/sec upload speed. 

Don't know if this added information is useful for diagnosis of problems or not. 

Brad Walter, WBY

Affiliation
American Association of Variable Star Observers (AAVSO)
Further Upload Experiences

Well, after about 09:00 CST, the server load became heavy. In a dozen tries, over 6 hours I have been able t oload only 3 of my last 4 images. The last 4 tries were a  B image that stubbornly refuses to appear on the file list.

Sorry, but once I get my last transformation image uploaded, I think I will wait until, the upload process is significantly improved before becoming a regular user of VPhot. 

Brad Walter, WBY

Affiliation
American Association of Variable Star Observers (AAVSO)
Upload Size

Brad:

I can understand your pain! At least get your transforms done with VPhot and PTGP!

I'm sure you want to make use of a beautiful big CCD camera BUT for your photometry is there anything you would consider to shrink the image size (24 MB!!)?  I assume you have your FWHM set to yield about 2.0 to 2.5 pixels? Can you set your imaging software (maxim?) to save only the center (1/2?) of the image? What is your FOV?

Hope things improve for you. As you asked me previously, my images are small by comparison (3MB from my 6303 chip on a 12.5" F/8). I bin 2x2 to match my pixel size (9 um) and seeing (3-4").

As I mentioned, I can upload 100s+ w/o problems. I do have a fast FIOS connection. Of course, I may cause high loads during my uploads but they do go reasonably quickly. Today there was 1000+ (sorry!) which took a few hours with other loads by others.  ;-(  They all did arrive on my account.

Ken

Affiliation
American Association of Variable Star Observers (AAVSO)
Image size

 I get about .75 arcsec per pixel which is just about right for good nights when I have 2-arcsec seeing. It gets that good perhaps 20% - 25% of the time. On bad nights when I hit 3 arcsec FWHM I can bin 2x2 

In this case I could download only the center part of the image. I didn't want to bin because the field is crowded. In other cases where comps are sparse or I want to use as many good comps as I can get  in an ensemble, I need most of the chip. I don't understand why the images are 24MBytes. I would expect them to be just a bit more than half that with 16 bits per pixel + header + a couple of dark rows. at the edges. I don't understand why it works out to four bytes per pixel instead of two. Your 3 MB files for a 6303 chip binned 2x2 seems right. My 24MB seems to large for a 6303 binned 1x1 by a factor of two. 

I am going to ask SBIG what the extra file size is for. My old ST7 camera also used 4 bytes per pixel.  

Brad Walter, WBY

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Image Size

Aha! i discovered why the images seem 2x the size they should be. it isn't the camera. It is the calibration software. It saves maste calibration frames with 32 bits per pixel. I assume it does that so that data isn't truncated during the creation of the master frames. however, when it uses those to calibrate the normal 16 bit light frames it saves the result as 32 bit as well automatically even though I never asked it to perform a data type conversion. I never noticed it before. I will have to convert back to 16 bit. 

Can anyone advise if this should be 16 bit signed or unsigned? 

Brad Walter

 

Affiliation
American Association of Variable Star Observers (AAVSO)
image calibration

Hi Brad,

Always use "unsigned" when dealing with 16-bit numbers.  Otherwise, you actually have 15-bit numbers. :-)

Arne