Skip to main content

VPhot stuck or...? Processing question

21 posts / 0 new
Last post
hgeagle
VPhot stuck or...? Processing question

I tried to upload a number of images, but they do not show up in my "images" list. Uploading seemed to work (I got some processed fine during the day using the "wizard", and they showed up quickly in the images list).

However, in the evening I tried to use the "quick upload" since I was confident that my format/fits headers were similar to the ones tried successfully earlier. The processing queue appeared to be "busy", but eventually reached my images. They disappeared from the queue, but did not appear in the images list even after "refreshing" for "most recent" images at quite some time after the queue was emptied.

Finally my question: If the images disappear from the "server processing queue", should we expect that they will be available in the images list shortly after, or can it still take a substantial time before they show up there due to some further processing?

Cheers,

Helmar

 

MZK
MZK's picture
Wizard vs Quick

Helmar:

Two issues here:

After images pass through the queue they should appear on your image list quickly. At least that is my normal observation. You stated that the images did move through the queue, so it is not stuck. I assume you saw other images make it through the queue also? More importantly, you stated the images appeared in your list after you used the wizard.

However, your assumption to "use the "quick upload" since I was confident that my format/fits headers were similar to the ones tried successfully earlier." may not be accurate. You are asked to input any missing fits info when you use the wizard. Did you add such info in the wizard? The fits info may in fact not be present in the other fits headers and thus the VPhot processing after the quick upload was not successful.

I'm a bit confused why you felt you needed to try different upload procedures and I'm pretty sure you have used VPhot before? So what changed today in terms of your images or process?

Also, you might try searching in "more" for a specific image target, system and date to find the images in question.

Ken

hgeagle
Re: Wizard vs. Quick

Hi Ken,

Thanks for your quick response!

>" I assume you saw other images make it through the queue also?"<     Yes, they did.

>"You are asked to input any missing fits info when you use the wizard. Did you add such info in the wizard?"<   I did not have to add anything in the wizard. In fact I learned some time ago that VPhot is quite sensitive to some missing or changed header info. E.g.,  AIP4Win appears to change/forget some info during its processing (such as the "Exposure time" which in the original data file is existing, but then set to "0" during/after processing). Of course, VPhot needs the "Exposure time"...

>"I'm a bit confused why you felt you needed to try different upload procedures and I'm pretty sure you have used VPhot before? So what changed today in terms of your images or process?"<   Yes, I have used VPhot before; I wanted to change to "Quick" because it is... wink. The wizard takes more time with the added step(s), especially when you have a larger amount of files to submit. 

>"...you might try searching in "more" for a specific image target..."<    Believe me, I did. 

Also, I resubmitted later some of the same data with the wizard (no complaints), they went throught the queue, and disappeared also.  I hope that is not the "known bug" regarding similar images you mentioned before in another thread...

Here is another question:    If I successfully used the wizard to upload images without having to add any info, should I not assume that the "quick" uploader would work also (after choosing the correct telescope)?  

I am just running another "experiment" with three images of S Sex, uploading with the wizard and being in the queue...

Best,

Helmar (AHM)

 

 

hgeagle
Re:Re: Wizard vs. Quick

reply to my own reply:

VPhot does not like certain ways of naming files - but what is the correct way?

1. My experiment with "S Sex (VSX34975)_V_AHM_2015-03-20_00001.fts" passed the wizard, got into the queue without complaint and disappeared after being processed. It is not the "parental filter": The same file named "S Sex_V_AHM_2015-03-20_00001.fts" went through completely and showed up in the images list. My conclusion: VPhot does not like the brackets (even if the brackets show up in the fits header and later in the images list) - should I have known?

"Fits Liberator 3" has no problem reading the file with the first naming.

2. The S Sex file has the same processing of RA/DEC info as successful others, but VPhot complained about "WCS" info not available -  but it is there...   I loaded that S Sex file into "astrometry.net" for additional plate-solving and uploaded that file to VPhot: This time no complaint, the coordinates were recognized.

Please don't misunderstand me: I really do love VPhot for its photometry capabilities (think transformation coefficients using M67...), but the proper file naming and WCS processing could use some improvements(?). It is very frustrating  if you have to wait at busy times for your files to get processed, and then they disappear to somewhere... and you do not understand why.

So, my - I think simple - request for improvement: Could you tell us (maybe on the log-in page with examples and in the manual) how to properly name the files or what to avoid? I am mostly concerned about new users and the cases when you want to change your own processing pipeline...

Thanks,

Helmar (AHM)

 

MZK
MZK's picture
File Naming

Hi Helmar:

Thanks for trying this "experiment".  Yes, I like "experiments" since I'm a chemist.  wink

They are interesting observations. I'm going to talk with Geir off-line, but will place appropriate posts as well!

As a user of VPhot and an obvious proponent, I feel your frustration. I (we) certainly would like to understand and correct this recurrent issue. It has not been understood yet?!  angry

Thanks for your help. I always enjoy your efforts to help resolve issues like this.

Ken

MZK
MZK's picture
Parentheses

<<1. My experiment with "S Sex (VSX34975)_V_AHM_2015-03-20_00001.fts" passed the wizard, got into the queue without complaint and disappeared after being processed. It is not the "parental filter": The same file named "S Sex_V_AHM_2015-03-20_00001.fts" went through completely and showed up in the images list. My conclusion: VPhot does not like the brackets (even if the brackets show up in the fits header and later in the images list) - should I have known?>.

Just to confirm - You uploaded the single (or 3?) file with the parentheses in the name FIRST with the wizard and THEN you uploaded the same file without the parentheses in the name? You only "renamed" the image file in windows between uploads and that was the order of upload, not the reverse? And it was the second upload which made it to your image list?

I'm trying my own similar test with some old images long expunged from my image list. Clarifying because this simple renaming would be something VPhot's QC check should be able to catch to prevent the "same/identical" image from being processed? Forget the name, it is the same image which could then be submitted to AID as a duplicate otherwise! Also, part of the uncertainty is which image (earlier or later) would show up in the image list. It may not be the later one which you would expect to be there that actually shows up!!!???

Ken

 

MZK
MZK's picture
Parentheses

Helmar:

I think (?) I have confirmed your observation about parentheses in the image name?

My experiment involved 8 images of the same target at different times (time series / not identical), 4 in V and 4 in I filter. These were all uploaded and reduced successfully about a year ago. I used both a proprietary upload tool and quick upload.

I first tried the proprietary upload tool with no parentheses in the name first and subsequently those with parentheses. In this case the first set of 8 images w/o parentheses in the name went to the queue, through the queue and to my image list. The following upload of 8 images with parentheses did not make it to my image list. Due to final naming of the zip images by the proprietary tool I cannot confirm which set made it to the list but I'm quite certain it was the images w/o parentheses. They may have been considered duplicates so this step is not conclusive.

I then tried the quick upload after deleting the above images from my image list. I first uploaded the 8 images with parentheses. They all made it to the queue and through the queue but did not appear in my image list. I then uploaded the 8 images w/o parentheses. They made it to the queue and through the queue and then appeared on my image list. If images in the image list are there even if they do not show, they may again be considered duplicates and not be shown. However, this seems unlikely to me?

SO, Geir: Could you try this out to see if you agree? I have shared 8 images with you to use but I'm sure you have some to try also. If this turns out to be true, it is disconcerting that we haven't noticed this before. There may also be other similar naming issues to deal with?  Perhaps Helmar is the only observant one among us!frown

Ken

 

hgeagle
Re: Parentheses

I uploaded three files with parentheses -> did not show up in images list

I deleted parantheses (and the text inside) in Win Explorer, then uploaded one file -> showed up in images list

Also I repeated this with a whole bunch of other files; after deleting the paranthesis parts in the file names -> all showed up in images list.

>"It may not be the later one which you would expect to be there that actually shows up!!!???"<

I cannot guarantee that I had not tried to upload any of those files earlier (yesterday), so this is an interesting note...

Hth,

Helmar (AHM)

P.S.: I have now successfully uploaded all my recent files without parantheses yes

SGEO
SGEO's picture
VPhot queue processing

I notice that the queue right now is backed up because one user has uploaded 1000 megabyte images to process. That's going to take a long time and doesn't seem to be a fair or considerate use of a shared resource.

Have there been discussions of how to manage the queue? Maybe limiting users to a time budget and then pushing their outstanding work to the end of the queue. .

George

MZK
MZK's picture
Number of Images

Hi George:

Yes, that issue has been discussed before. The current images (actually 1300) are not mine but I often upload up to about 700 images at once after an observing night. The current images are only about 1MB each. (Note that the kB at the end of the line is actually B, a know issue). At that size they should take a couple of hours or less if nothing is problematic with their pinpointing or headers. They seem to be moving at a reasonable clip.

Unless we have a problem processing the images (which has never happened before, I'm sure!  devil), occasional large numbers of images by an individual has not led to a mutiny! Yes, it has been noticed before. So I'm obviously not in favor of a "quota". At least not until it is a BIG problem. Patience is sometimes (always?) necessary. Something which many users may not have? I usually upload my images before I go to work and reduce them when I get home. On the weekend, I sometimes get anxious! I have come to expect a certain speed so I'm used to it and can cope with it.

So it's a matter of waiting, or making one person happy and another unhappy. Not sure we are at the tipping point yet? The queue does run empty during the day.

Ken

SGEO
SGEO's picture
VPhot queue

Looking at the queue display, I don't believe it is in processing order. Sure looks alphabetic.

I'm thinking that it is getting to be a problem.It's the tragedy of the commons: a free resource over used. There should be a level of queue management so no one user can monopolize the resource for half a day.

George

 

Lew Cook
Lew Cook's picture
Yeah, me too

I noted that the other day, too. It processed mine before the first on the list, but I was happy it did!

MZK
MZK's picture
Processing Order

George and Lew:

Yes, the list does not process in the order listed, it is alpahabetical and then by time. It is time in that determines order of processing. Geir knows about this. I'll add to more observant individuals to the list!

George:

If we came up with a plan to allocate usage it might be considered?

Ken

Tonisee
VPhot "stuck"

At the moment, it seems that approximately two 1 MB images are processed in one minute. That seems to me  very slow (compared what standalone programs could do with plate solving etc etc). And as it was noted, those individual images are fairly small according to typical full-frame images nowadays. Is there any room for perfomance improvements?

I'm not familiar with the "kitchen side" of VPhot, except that it should be running in a cloud. Wouldn't it be possible to review current service terms and ask dynamically allocated CPUs and RAM? Often there is no images in the queue, sometimes situations like the current one, appear. Can VPhot run on multiple cores? What about multiple instances/threads of VPhot running in parallel?

Best wishes,
Tõnis
 

MZK
MZK's picture
VPhot Speed

Hi Tonis:

Your observations are spot on. The comments about amazon cloud hardware options are known and have been discussed. $$ generally defines what options are feasible. Note that this is neither unreasonable nor unexpected. indecision I believe the system does use more than one instance to operate. Just do not remember exactly what? That was an upgrade about a year ago.

As the number of users increases and file size increases, any increasing frustration may ultimately help make any necessary or useful changes?

Your measurement of speed of processing is good. My observations of the variation in image processing speed drive me nuts! There continues to be disagreement among different parties about whether the change in processing speed observed on the VPhot system is only 3x (?) or 10x (me). The reason is not well understood?

So, keep the faith! 

Ken

PVEA
PVEA's picture
VPHOT upload engine stuck

Since yesterday VPHOT upload engine have not been processed any images but surprisingly none of the uploader had raised the alarm yet.

kge
kge's picture
VPHOT upload engine stuck

The processing engine had crashed, and needed a restart. Images are now being processed.

Geir

lmk
lmk's picture
VPhot code needs to be more robust.

[quote=kge]

The processing engine had crashed, and needed a restart. Images are now being processed.

[/quote]

Given the heavy usage, importance of this application, I repeat my prior recommendation - It needs to be looked at in more detail to improve the proper error handling in the low level code. "Try...catch..", if C++ or Java?

Mike

 

TCB168
TCB168's picture
Crashed VPhot

Seems to be stuck again.

Images appear in the queue but arn't being processed.

 

kqr
kqr's picture
VPhot is now processing

VPhot is now processing images again!
---
Doc Kinne

Lew Cook
Lew Cook's picture
Stuck again

Been "working" on this image for over 6 hours:

Calibrated-T18-chyneuze-M101-20150512-060550-Green-BIN1-W-300-002.fit

Of course, mine have not been processed - otherwise I wouldn't care ...

  wink

Lew

Log in to post comments
AAVSO 49 Bay State Rd. Cambridge, MA 02138 aavso@aavso.org 617-354-0484