Skip to main content

Time series issues

The membership database is currently undergoing upgrading/testing. Web functions that involve an interface with the member database (webobs, etc.) may experience intermittent problems. We hope to have this issue resolved within 24 hours (by 5pm Eastern Time on October 1st or 12:00 UT). Thank you for your patience.
lmk's picture
lmk
Offline
Joined: 2010-07-23

Its becoming apparent that the large increases we are seeing in observations totals is likely due mainly to the submission of long time series by CCD observers. Now, I am not implying anything is wrong with time series, in fact they are a great and useful feature of automated telescopes and software!

However, these long runs on a single object by a single observer can really screw up the traditional measures and behaviors of our systems here. For example, I had to cancel my Newsflash reports, because there was no method available to exclude all the time series. It results in lengthy multi-page Newsflash reports, which defeats their purpose. You just want a concise view of what your stars are up to, not endless rows of repetitive data.

It also affects the observations counts being submitted to the AID. A long time series is really not worthy of crediting hundreds of observations to a single observer, as would the same number of individual visual observations of different objects would!

What I am suggesting is the need for a way to somehow segregate time series from other individual observations. Maybe flag them as a single "block" or array of data which would be counted the same as a single visual observation, for example. Then, Newsflash would only output one representative observation from that long series, for example, making those reports become useful again. And, the observer totals would return to a more reasonable and realistic level as well.

Hopefully this issue can be addressed in a timely fashion.

Mahalo, Mike LMK

Filter out CCD obs?
HTY's picture
HTY
Offline
Joined: 2010-07-23

Hi Mike,

If I'm not mistaken, you can ask for just visual observations to be included on the Newsflash.  That's what I do and it works out for me.

...Tim

Visually speaking...
GTN's picture
GTN
Offline
Joined: 2010-07-08

Hi Tim,

I agree with Mike re the long time series provided by NewsFlash, but the problem may already have a user work-around since it looks like the visual observations are listed in the NewsFlashes after the CCD time series observations.  As an example, TT Ari has been getting a lot of well-deserved CCD attention lately so there are a lot of observations for it in my "NL" NewsFlash reports, and it's worked well for me to just scroll down to get to the visual stuff.

For stars not being covered very well, you might miss some event, a rapid fade of an RCB for example, that a CCD observer reported if you limit the reports to just visual observations.  I'd like a way to limit the time series information -  list the first, middle, and last observations by each observer maybe.  If the star is showing odd behavior, the appropriate comment code(s) should be set by the observers.

Cheers,

Thom

Somewhat of a quandry
HTY's picture
HTY
Offline
Joined: 2010-07-23

Hi Thom and Mike,

I hear what you are saying.  I'm thinking it would get messy because then you have to program in logic for Newsflash to recognize what a time series is (X number of observations where type = "YYY" and observer = "same" within a Z time period, etc.).  Then you would have to flag them as time series in order to perform what ever filtering logic (averaging, first, last, middle, etc.) you prefer.  Then of course everyone would want to treat a time series somewhat differently. 

I just wonder if it's a good use of programming resouces just to save someone some scrolling.  Maybe I'm wrong and it's easier than I think.

.....Tim (HTY)

One suggestion
lmk's picture
lmk
Offline
Joined: 2010-07-23

Putting on my IT thinking cap, one neat way to handle the problem of time series would be to define the basic observation unit in AID to be a variable length ARRAY, rather than a single DB row for each measurement.

Then, the user could simply select in the CCD submission form whether its a time series or individual observations.  The time series length, step size, beginning and end would be up to the observer. They would simply be creating a single observation, which is the ARRAY of measurements which constitute the time series.

Of course, visual observatins or single CCD observations would be simply a length=1 ARRAY. All existing observations would need to be redefined as ARRAY types, and appropriate changes made to the database, but this can be accomplished offline and just once and for all.

An alternate approach, if its not desirable to change the entire AID to ARRAY type for every observation (even though simpler in the long run), is to make 2 data types: the existing single row per observation for non-time series, and the ARRAY type for time series. Depending on the database and access methods employed, this may or may not be easier to do?

The legacy time series data as single observations will need to be handled in some way, though. But from this point forward the issue would be resolved.

Mike LMK

GROUP variable
Aaron Price's picture
Aaron Price
Offline
Joined: 2010-05-07

The extended format has a GROUP variable where the observer can group a set of observations based on their own criteria. It's currently optional and, in fact, the instructions state that time series people should put "na" as their GROUP value. Perhaps we should encourage time series submissions to use this variable? Then it would be easy to code MNF or any other tool to skip or collapse time series observations.

Time Series Issues
WGR's picture
WGR
Offline
Joined: 2010-08-17

Hello Michael

  

I agree that ccd time series measurements have greatly increased the numbers of observatons that we are seeing in the AID.  The critcism was that long time series runs by a single observer were screwing up the traditional measures and behaviors of our systems here.  While an example was not cited in your email, Thom mentioned TT Ari.  This is a questionable conclusion in this particular case.

While I am not a frequent user of the News Flash, I do look at various objects that I am observing via the use of the LCG.  It definitely can be set so that the Vis observations show up, and the BVRI and others are excluded.  I know that I get a lot more information from a plot than a table of numbers.  Perhaps this would be an option that would allow you to view only those observations that you prefer.

I did take a detailed look at TT Ari.  I can report that it clearly is not being messed up.  Its not a single observer with massive time series, its actually a lot of observers.  In fact looking back at the last 1000 days, it is clear that starting about 150 days ago, activity picked up a lot.  This co-incides with a campaign #469, which aavso ovservers are responding to a campaign, requested by a Professional Astronomer, Dr. Vogt, who was also granted time on the MOST satellite for 22 days, observing TT Ari.  MOST was going to observer TT Ari for 50 of every 100 minute orbit, for 22 days.  The Alert asked ccd observers to specifically observe at a 1 minute cadence for a minimum of 3.3 hours.  This was rationalized that the science goal of this project was to separate the superhumping behavior from the orbital behavior, which appeared to have different periods.  The secondary goal was to characterize the flickering in this object.  The alert asked ccd observers to observer starting in August and continuuing until December.  So this is what you see in the TT Ari light curve.  There apear to be about 50 ccd observers contributing data in the campaign during the past 100 days.  The beating of the orbital and superhumping periods can be very nicely seen.  Great work ccd observers.

Lets try to remember, that campaigns like this one, providing data to the Professional Community is really our Core Mission.  Our core mission is not focussed on acquiring as many observations as we can via technology.  We should "let the science rule".  The observation cadence needs to support the science goals.  In some cases, this should be a few observatons per night, and in some cases like this one, many.

 

Gary Walker

WGR

   

 

 

AAVSO 49 Bay State Rd. Cambridge, MA 02138 aavso@aavso.org 617-354-0484