Welcome to the Data Analysis forum!

Affiliation
Astronomical Society of South Australia (ASSAU)
Fri, 04/12/2013 - 14:09

Welcome to the AAVSO Data Analysis forum!

Thanks to Rebecca for setting it up and to everyone at HQ for their support of the idea.

This forum starts with the thought: "So now that I have a time series of observations from CCD/DSLR photometry, PEP, or visual estimation, what do I do now?" 

The purpose the forum is to:

  1. discuss methods and tools for variable star data (light curve) analysis;
  2. encourage, and in some cases be a home for, data analysis projects.

In scope is discussion about the theory and application of variable star data analysis methods as enabled by tools like Period04PeransoVStar, TS, WinWWZ, in particular:

  • statistical analysis
  • data search/filtering
  •  
  • period analysis including Fourier analysis, pre-whitening, models, residuals
  • time-frequency analysis

The motivation for the forum arose from the "Project LAVA" topic that recently started on the Visual Observing forum. That topic will be moved over to this forum soon.

If you have questions or suggestions for forum content, feel free to post here.

Regards,

David Benn (BDJB)

Affiliation
Madrid Astronomical Association M1 (Spain) (AAM)
It's a very good idea David.

It's a very good idea David. I hope to learn from experienced analysers and get skills to analyse data if some day it's needed for some project.

Regards

Affiliation
American Association of Variable Star Observers (AAVSO)
Data Analysis Forum

This is great!  I've been in recent contact with Dr. Hendon regarding a data mining/data analysis forum for AAVSO.  He indicated that there was some interest in this topic.

I've been involved recently with the Lowell Obseratory Lowell Amateur Research Initative (LARI) MGED project that does data mining and analysis for exoplanet transient research.  We use several analysis tools, including some homegrown tools as well as MPO Canopus (developed for asteroid analysis but still good for variable star/exoplanet analysis).  I've also used VStar and algorithms from the NASA Exoplanet Archive (Lomb-Scargle, Box-fitting Least Squares-BLS, and Plavchan PDM) for my LARI work.

For LARIMGED we use a Yahoo user group for day-to-day communications and a wiki to upload analysis results, post detailed technical information, etc.  It seems to work very well.

So, I would love to be an active participant in this forum as well as LAVA.  Please let me know how I can contribute to these forums.

 

Dave Hinzel (HDHA)

Affiliation
Astronomical Society of South Australia (ASSAU)
Hi Dave

Thanks for your

Hi Dave

Thanks for your post. Interesting!

I'm not sure whether you've seen the Data Mining links on the AAVSO Research Portal page. Apart from this forum, perhaps Arne was also thinking about that, among other things.

I'll post again soon to the LAVA thread re: access to documents created so far, much of which has been done by Pete Brock (see the LAVA topic) up to this point. There's plenty to be done. 

The practice of using a wiki makes a lot of sense to me. I have used them in various contexts for several years and they're hard to beat for rich collaboration.

David (BDJB)

Affiliation
American Association of Variable Star Observers (AAVSO)
Data Analysis Forum

David,

How do we begin to do data analysis work within this forum?  Is there a database of objects we can analyze with the various tools or do we need to access observational data from other forums?

I'm very interested in getting started with this effort.  Any information will be appreciated.

Thanks,

Dave

Affiliation
Astronomical Society of South Australia (ASSAU)
Hi Dave

If you are

Hi Dave

If you are interested in contributing to Project LAVA, the best place to start is to post to that topic. You can look at the docs there and request access to the Google Docs if you would like to.

No other particular analysis project has been proposed in this forum yet, but suggestions are welcome.

I'm currently preparing a new VStar release (within the next week I hope) and will devote more energy to this forum after that.

David

Affiliation
Astronomical Society of South Australia (ASSAU)
Using the Data Analysis wiki

Hi all

Apologies for my recent silence. My son sustained a knee injury recently. The hospital stay and home care have keep my family occupied for awhile. More recently, being interstate for work and getting a VStar release out have added to the pre-occupation.

Will, Matt and I corresponded about the data analysis wiki. The end result is that the wiki we started with is still considred to be OK. To use it, you need to create a login via: 

  http://www.aavso.org/data-analysis/wiki/HelpOnLogin?action=login

This will allow you to view the data analysis pages (e.g. LAVA):

  http://www.aavso.org/data-analysis/wiki/LAVA%3A%20Low%20Amplitude%20Visual%20Analysis

but to edit (modify, add) them you will need to send an email to Will McMain (will@aavso.org) saying that you want to do so; he will change your permissions. I suspect that myself and some others may also be able to do this in the near future, so feel free to CC me on the email.

David

Affiliation
None
Data Driven Astronomy course

I had signed up for the previous version of the course in August.  Per the requirements at the time, I migrated from Python 2.7 to 3.6, including the scipy and numpy libraries (among others) and installed Jupyter notebook.  Once the course began, I found that all the programs were to be entered into their hokey online version, and that the focus was really on SQL databases. 

Note also that nothing in the course deals with variable stars, and there is little if anything dealing with machine learning. 

Alghough I dropped the course the first week in disgust, I am not recommending against taking it.  Just read the syllabus carefully so you won't be disappointed.

Affiliation
Astronomical Society of South Australia (ASSAU)
Data Driven Astronomy course

Hi Cliff, Stephen

Coincidentally, before I saw these posts, I'd enrolled in the course too. I've completed the 3rd week as I imagine has Cliff.

The first two weeks are Python with a strong emphasis on numpy. The 3rd and 4th weeks are SQL and the last two weeks are Python with a machine learning emphasis.

Yes, the Grok Learning environment is a bit different but not too bad actually. They do encourage you to use an external Python environment as well. I use a Jupyter notebook for experimentation.

Is the astronomy at a high level? I suppose so, yes.

Is there any variable star content? No, but I didn't expect there to be any from the course outline.

Am I learning anything? Yes, some. 

I've been been writing Python code for about 20 years and not so long ago took a 3 day Python for Scientists and Engineers course. I've used SQL for about 20 years as well. A long time ago I developed an object-relational mapping framework (Java to MySQL) from scratch (although I argued at the time to use JDO or Hibernate). Yet, I'm still learning things about numpy, SQL etc that I didn't know. For example, I had never used the HAVING clause in conjunction with GROUP BY in SQL. Maybe I'm a slow learner or maybe I just have a healthy awareness of the scope of my ignorance. wink

David

Affiliation
American Association of Variable Star Observers (AAVSO)
Data Reduction Programs - 2nd order extinction and Ev

<p>Are there programs available on the aavso website for pep observers to enter the data collected for 2nd order extinction and transformation (Ev) observations? I have been sending the transformation data to the pep chairman for reduction, this year&nbsp;I&nbsp;have&nbsp;collected the&nbsp;2nd order extinction observations and&nbsp;sent the data&nbsp;to a member of the&nbsp;pep&nbsp;group for reduction.&nbsp;I &nbsp;will be&nbsp;collecting &nbsp;the Ev data over the next couple months (weather dependent) and send it for reduction as well. It would be helpful if I could reduce this information and pass the results to those of need.</p>

Affiliation
American Association of Variable Star Observers (AAVSO)
AAVSO software for PEP data reduction.

<p>David, i made an inquiry about data reduction for pep, the scope of the inquiry was about 2nd order extinction and Ev data reduction.</p>

Affiliation
Astronomical Society of South Australia (ASSAU)
AAVSO software for PEP data reduction

Hi Paul

I'm not a PEP observer (visual and DSLR) but a good place to start would be Tom Calderwood's The Hitch-hiker's Guide to Photoelectric Photometry (https://www.aavso.org/pep-observers-guide). There is discussion of first and second order extinction as well as a section on software. 

Other than that, I would suggest asking a question on the Photometry forum: https://www.aavso.org/forum/5010

David

Affiliation
American Association of Variable Star Observers (AAVSO)
AAVSO PEP data reduction software

<p>Thanks for the quick response David, Tom Calderwood emailed me yesterday informing me that he working on a reduction spreadsheet for BV observations. He believes that it is almost complete but has a few bugs to iron out.</p>

Affiliation
American Association of Variable Star Observers (AAVSO)
Helio seismology

Hi David, 

I am very interested in contributing to this community through research and data analysis particularly towards helio seismology and of other pulsating stars.  Can you please advise where I could begin?  

Regards,

Ananth

Affiliation
Variable Stars South (VSS)
Re: Helio seismology

Ananth,

I see no one has replied so I'll have a go. For asteroseismology of pulsating stars, there are a couple of graduate level textbooks available. Asteroseismology by C. Aerts, J. Christensen-Dalsgaard and D.W. Kurtz published by Springer and Asternoseismic Data Analysis: Foundations and Techniques by S. Basu and W.J. Chaplin published by Princeton University Press. As I said, these are graduate level texts so assume a lot of bakcground in stellar astrophysics, but as long as you have the necessary background they are good texts and will give good guidence about what you need for both making observations and doing data analysis.

Bill Rea

Affiliation
Astronomical Society of South Australia (ASSAU)
Data analysis of pulsating stars

In addition to Bill's reply, if I interpret your "...data analysis particularly towards helio seismology and of other pulsating stars..." more generally to mean light curve analysis (the bits in bold), Grant Foster's book: Analysing Light Curves: A practical guide is great.

John Percey's Understanding Variable Stars talks about helioseismology and asteroseismology. For the first he points to http://soi.stanford.edu/results/heliowhat.html

David