# Stacking bug

Affiliation
Astronomical Society of South Australia (ASSAU)
Thu, 11/08/2018 - 22:04

Look at the date on this (average) stacked image.

The source images were 11:??:?? but the stacked image is 23:27:09. Odd no?

David

File Upload
Affiliation
American Association of Variable Star Observers (AAVSO)
Stacking Time

David:

My first question is why you are stacking 6 images from two different days, and also separated by 7 days? Actually, the stacked time is about right - the algorithm sets the stack time as the mid time of the stacked images. In this case with about 7.0 days between, the mid point would be 3.5 days, thus the 23:27?

It is logical in terms of the arithmetic of the common mid-time algorithm.  I would have stacked the images from each day separately, which makes more sense to me?

Ken

Affiliation
Astronomical Society of South Australia (ASSAU)
Sigh

Please ignore me Ken.

Can't believe I did that and wasted your time looking at this post.

That's what happens when I've done such a thing late at night and looked at it quickly early in the morning.

Apologies. How embarrassing.

David

small follow up question

Hi,

So if you stack images in VPHOT it sets the time of the stacked images to be the midtime of the image set?

Does it take exposure time into account? I.e if you stacked two images with different exposure times, would it weight the average of the times towards the image with the longer exposure?

Thank you,

Peter

Affiliation
American Association of Variable Star Observers (AAVSO)
stacking time

Peter,

I think for stacking to work the exposures must all be the same, but I'm not sure exactly how VPhot reports the time for a stacked image.

My guess would be that it reports the mid point between the start of the first exposure and the start of the last exposure.

Phil

Affiliation
American Association of Variable Star Observers (AAVSO)
stack time

Hi Peter:

Interesting question. I'm in the process of checking this out. I have short images (10s) and longer images (90s) of multiple LPVs that I collect so I do not have to worry about whether an LPV is faint or bright during imaging.

I just tabulated the VPhot displayed image time in the image list AND fits header times and image durations for these images. This will tell me what happens when I stack two images of different durations. I will report soon?

HOWEVER, there is NO reason why you cannot do the same thing. ;-)  It is not that difficult! You need to look at both the displayed image time (image list), and the image times and image duration time in the image headers. It should just require a little arithmetic? There is a little wrinkle that you should discover. Try even vs. odd number of stacked images. What did you discover? There is a reason for what Geir Klingenberg decided to do with respect to reported time!

In fact, I would prefer that you do this. It is the type of thing I have the participants do during the VPhot course. Are you willing to take a stab at this effort? It will show you what VPhot does to calculate these times, certainly for equal duration images!

BTW, in general I'm not sure why you would want to do this often? Typically, you want to improve SNR by stacking replicate images of equal duration OR, in my case above, I would use the long image if the target was faint and the short image if the target were bright/saturated. Do you have a particular routine where this is not done? Also keep in mind that you are only stacking images that you do not expect to vary over the total imaging duration. In such a circumstance, the exact time of constant magnitude is not particularly important BUT let's just deal with your question first rather than debate that comment.

Ken

Affiliation
American Association of Variable Star Observers (AAVSO)
VPhot stacking methodology

These are interesting questions which should be answerable without experimentation! If there are folks out there who would like to read the code, I would be happy to share access to the repo. It's VBNet code. :(

The VPhot team has been marking time for the past year and more with other priorities keeping us tied. up. The team is currently made up of myself and Ken and Diane Menzies. I have fond hopes of getting back to work on VPhot this year to answer questions like this stacking issue and address the large pile of suggestions we have accumulated for improving VPhot.

George

Affiliation
American Association of Variable Star Observers (AAVSO)
Experimentation

Hi George:

This is ALWAYS where we will disagree! As a chemist, experimentation is ingrained in my approach to photometry and to teaching.

Often (I'm not saying it is true in this case or always), members ask questions that require only a little "observation" to answer. I feel that is the BEST way to learn.

Ken

PS: Learning is most effective with a combination of listening, reading, teaching, AND practicing/doing, not just one of these.

i 'll give it a go

Ken,

I will try what you sugested re experimenting with various combinations.

I just asked about the different exposure times because I had some images I collected on a night where I ws having some difficulty with my guiding and I was experimenting so see what kind of exposure time I could get without too much elongation and I was hoping to use them all if possible.

Seems like it is not a great idea.

Peter

Affiliation
American Association of Variable Star Observers (AAVSO)
Stack Issues

Hi Peter:

Stacking is clearly a technique to improve signal to noise ratio by adding up all adu values in individual pixels in multiple images, after which VPhot allows averaging or median selection of signal and then divides by noise which only increases by the sqrt of signal. Signal/noise ratio (SNR) should improve with more stacked images by about sqrt N.

Except, I would suspect that if you add adu values as I did for a 10 sec image and a 90 sec image and then average (cut in half), the average signal would be less than (not same as) that in the longer image. I suspect that is not what you want to do?  Of course, the noise is reduced by sqrt.

When I did my stack test, the stack image SNR was clearly less than that of the longer image and more than that of the shorter image. What happened when you stacked your different duration images? BTW, I suspect your image durations were not as extreme as mine? If you got a similar result, it confirms that stacking images of different duration will not give you what you were looking for. It would be best just to use the longer image. Of course, you had image problems and you tried something else. That’s good, but it didn’t give you the result you expected/desired?

For the stack image time stamp question, I tabulated the different times used by VPhot. VPhot displays a date/time in the image list. You can also find the image start time and image duration in the fits headers.

You will notice that the time displayed on the image list is usually the midpoint time calculated from the fits header start time plus ½ the image duration. That is, normal for the displayed time of each individual image BUT not for a stacked image. When VPhot was created, Geir Klingenberg tried to avoid displaying identical times in the image list. IF, an odd number of images were stacked (e.g., 3), the displayed time of the stacked image would be identical to the displayed time of the middle image. To avoid this “confusion”, he decided to display the start time of the stacked image. However, the reported time in the AAVSO report of this stacked image is still the midpoint of that stacked average image. This would normally be true if the stacked images were all of the same duration. In my case where the images had distinctly different durations, VPhot actually ended up assuming the stacked image duration was 90s rather than some average of 10s and 90s. I guess one might consider this a bug? It certainly is an anomaly? Again, not something which would occur for stacked images of equal duration. Oh well!  ;-(

Ken

Affiliation
American Association of Variable Star Observers (AAVSO)
Server Error in '/app'

Server Error in '/app' Application.

Unable to load DLL 'PhotLib.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.DllNotFoundException: Unable to load DLL 'PhotLib.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)

Source Error:

Line 34: method = 1
Line 35: End If
Line 36: PhotLib.doStack2(stack.NbrOfFiles, stack.Files, newfilepath, stack.OffsetX(0), stack.OffsetY(0), stack.Width, stack.Height, method)
Line 37:
Line 38: ' Write the updated headers, date and time

Source File: C:\inetpub\wwwroot\app\restricted\Stack.aspx.vb    Line: 36

Stack Trace:

[DllNotFoundException: Unable to load DLL 'PhotLib.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)]
PhotLib.doStack2(Int32 n, String files, String newfile, Int32& offsetX, Int32& offsetY, Int32 width, Int32 height, Int32 stacktype) +0
Restricted_Stack.btnStack_Click(Object sender, EventArgs e) in C:\inetpub\wwwroot\app\restricted\Stack.aspx.vb:36
System.Web.UI.WebControls.Button.OnClick(EventArgs e) +9816434
System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument) +204
System.Web.UI.WebControls.Button.System.Web.UI.IPostBackEventHandler.RaisePostBackEvent(String eventArgument) +12
System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument) +15
System.Web.UI.Page.RaisePostBackEvent(NameValueCollection postData) +35
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +1639

Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.7.2623.0

Affiliation
American Association of Variable Star Observers (AAVSO)
Stack Error??

Hi Ray:

Not sure if you were trying to stack images, but if you were, the work around is to open one of the images first and then go back to the image list.

Stacking should work the second time. This workaround was noted earlier by others. Advise success or failure? Not sure why, but this seems to work.

HTH, Ken

Affiliation
American Association of Variable Star Observers (AAVSO)
work-around

Thanks Ken

I'll try that. Meanwhile, I have a work-around that I can't seem to quantify yet.

One thing; i did have one image wthout a name and no WCS. But I was not using that image to stack. If stacking looks at every image rather than just the ones being stacked, that could cause an error.

Second thing, I had a list solved images from both sides of the meridian, but was stacking just 10 on the same side of the meridian.

Again, if stacking looks at every image in the list rather than just the ones being stacked, that could possibly cause confusion.

RayTRE

Affiliation
American Association of Variable Star Observers (AAVSO)
nevermind

it began to work when 6 images were stacked rather than 10 or 7.

Also some judicious, superstitious, magic clicking around the stack window.

The problem has been ongoing for a couple days.

RayTRE

any insight into my problem?

Hi,

I have been taking sequences of images where I straddle my 240 sec V image with two 240 sec B images. The idea being to stack the B images and have an average image time for the stacked B image that more or less corresponds to the V image time.

I get the following from the FITS headers in VPHOT.

First B image

JD_UTC
2458683.576608796
Julian Date (UTC) at mid-exposure

V image -

JD_UTC
2458683.5794675923
Julian Date (UTC) at mid-exposure

Second B image

JD_UTC
2458683.582361111
Julian Date (UTC) at mid-exposure

Stacked B images

JD_UTC
2458683.582361111
Julian Date (UTC) at mid-exposure

I have double checked that I have been looking at the correct images and have re-stacked them to see if I goofed when I did that.

Anybody else have this experience? It seems to interpolate the airmass nicely.

Thank you,

Peter

Affiliation
American Association of Variable Star Observers (AAVSO)
Stack Time/JD?

Peter:

Please share these specific images to me at MZK.

Are you sure that the JD value of the stacked image said "at mid exposure" and not "at start of exposure"? VPhot stack times (UT and JD) have a few idiosyncracies reported in headers. Need to read them carefully.

BTW, were you transforming the BV pair or just trying to get times the same? I assume you stacked the B images to improve SNR for B?

Ken

shared

Hi Ken,

I have shared four images with you - or at least I tried to do so. Never done that so hopefully it worked.

three of the images are the unstacked images - B-V-B  - and the fourth is the stack of the two B images.

yes i was stacking to improve SNR for B. I have not transformed them yet.

Appreciate any insight into what I am doing wrong. Pretty sure I was looking at the mid expsoure times - the times listed in my earlier post are copied from the FITS header file in VPHOT.

Thank you,

Peter

Affiliation
American Association of Variable Star Observers (AAVSO)
Various JDs

Peter:

You shared them all correctly. Good!

You haven't done anything incorrectly. BUT, did you process these images with AIJ or other software before uploading?  All those extra JD headers with barycentric, etc, did not come from VPhot?

Note that the reported time of the stacked image on the image list is the average time of the stacked B images with 120 sec subtracted (1/2 of the total duration of 240 s). IOW, it is the start time of the stacked B image. Look in the header again but at the earlier JD not the different JD's listed later. I think this matches the mid time of the stacked image. Use a JD to UTC converter to confirm and report back?

HTH, Ken

thanks

Hi Ken,

I used AIJ to calibrate them - bias, dark and flat.

Here is my image list for the shared images

4

acadiaobs
V552Cas
2019-07-19 01:58:36
1.986
240 s
B

3

acadiaobs
V552Cas
2019-07-19 01:54:26
2.006
240 s
V

2

acadiaobs
V552Cas
2019-07-19 01:52:27
2.006
240 s
B

1

acadiaobs
V552Cas
2019-07-19 01:50:19
2.027
240 s
B

I am not 100% certain what these time represent - start times, or mid-image times. Examining the FITS header of image 1 it lists the exposure start time to be 01:48:19 - so it appears these are mid-exposure times. Is that correct?

Either way, if I take a 240 sec B image, then a 240 sec V image, and then a 240 B image, why is the stacked image time not more or less the same as the V image time?

For simplicity, suppose we start exposure 1 at t=0. It then runs to t=240

exposure 2 runs from 240-480

exposure 3 runs from 480-720

E1 midpoint is t=120

E2 midpoint is t=360

E3 midpoint is t=600

Stack of E1 and E3 should have a midpoint time of (600+120)/2 = 360, same as E2.

Looked at the headers again and examined the earlier JD as suggested. Get these results

b
2458683.5780092600

v
2458683.5808680600

b
2458683.5837615700

stacked b
2458683.5837615700

Stacked image has same JD as the final B image.

It appears to me that when VPHOT stacks it reports the image time as the start of the exposure in the image list, Unstacked images appear to me to have their time as the mid-image time.

JD in a stacked image appears to me to not be correct.

Or I am all wet.

Peter

Affiliation
American Association of Variable Star Observers (AAVSO)
Time/JD

Hi Peter:

UTC Times seem to be calculated based on mid-point of the two images that were stacked BUT perhaps not the JD?? I will check some more to confirm this or not. Try some other images if you get a chance.

Ken

my results agree with yours

Hi Ken,

Thank you very much for the follow up on this.  I checked some more images and found that DATE-OBS (UTC date/time of exposure start (averaged)) is the averaged exposure start time.

And that is the time that is displayed in the Date/Time column of the image list in VPHOT for the STACKED image. For the unstacked images, VPHOT displays the mid-exposure time in the Date/Time Column.

Thus, VPHOT displays exposure start time for stacked images, and mid-exposure time for unstacked images.

As for the JD's, the stacked images have the same JD (JULIAN DATE), JD_SOBS (Julian Date at start of exposure) and JD_UTC (Julian Date (UTC) at mid-exposure) as the second image. (I note that I have not experimented with more than two images for this analysis)

If I generate an AAVSO report for the single stacked image, it contains the average value of the JD_UTC of the two images it was stacked from - great news!!! Recall JD_UTC is the mid-exposure time.

However, if I do a time series analysis with the three images (two unstacked images and the stacked image created from them), the JD reported in the AAVSO report is for the start time for the stacked image but the mid-exposure time for unstacked images. This is consistent with why I started down this path in the first place - I noticed that my phase plot for the stacked images was shifted relative to that obtained with unstacked images.

Again, thanks for your attention to this matter.

Peter