VStar Plugin Development

Affiliation
American Association of Variable Star Observers (AAVSO)
Sat, 08/04/2018 - 17:57

Let me ask a couple of questions on this topic.  If these now, or in the future become something best taken offline rather than answered in this forum, just let me know.

First is the plugin additional resources source code.  The link in this section at the bottom on this page does not seem to work.  If I look here instead is that the best place to go?

GAIA import is the plugin I am considering.  Photometry from GAIA comes in three passbands.  The project gives some guidelines on transforming the GAIA passbands into other ones which are supported by VStar.  Is there a way to import the GAIA photometry without transforming it into VStar and not have them all show up as unspecified?

thanks

Cliff

Affiliation
American Association of Variable Star Observers (AAVSO)
VStar Plugin "Hello World"

David,

I realize we live a half a world apart.  Not wanting to wait a day, I went ahead and progressed to compiling and testing the "hello world" type plugins.  I had mixed results and would welcome your suggestions.  Please see the attached description.

regards,

Cliff

File upload
Affiliation
Astronomical Society of South Australia (ASSAU)
VStar Plugin "Hello World"

Hi Cliff

Thanks for your interest in writing plugins and for your detailed log and analysis.

The daily timezone based cycle can make life interesting but we can work with it well enough.

Please try this:

  jar -cvf my.vstar.plugin.ExampleObSource.jar my

or

  jar -cvf my.vstar.plugin.ExampleObSource.jar my/vstar/plugin/ExampleObSource*.class

Both will have the same effect.

The jar file created with the command in https://www.aavso.org/sites/default/files/vstar-plugins/vstar-plugins-d… is okay when there is only one class file to be added to the jar file, which is the case for the ObservationCounter plugin but not for the observation source plugins you are working with.

Let me know how it goes.

It's been awhile since I updated the plugin dev doc. I should fix that.

David

ps - we'll sort out that broken link to the plugin code from the plugin library page too

Affiliation
Astronomical Society of South Australia (ASSAU)
GAIA plugin

Cliff

Happy to talk about the best approach to creating the GAIA plugin.

We can have the plugin assign bands of whatever name we desire for the GAIA observations.

David

Affiliation
American Association of Variable Star Observers (AAVSO)
Moving Along

David,

I have posted to the Photometry forum to try to get some feedback on the passbands.  So far there has been no interest expressed so I might just make the decision to suit myself.

The GAIA web service response is actually a VOTable in XML format.  Now I have searched around a  bit and do not see where VStar currently parses this format - is that right?  However, I note your post where you got so far as to write a sample.  However, I can not locate the file you refer to in that post.  I'm sure the last thing we want is two different ways to parse VOTables in VStar.  What do you recommend?

thanks,

Cliff

Affiliation
Astronomical Society of South Australia (ASSAU)
Moving Along: VOTable code

Hi Cliff

I saw your post to the photometry forum. Thanks.

The repository URL structure changed since that post apparently and that source file was removed from the latest codebase, however it's still available in the record of commits.

This was for the purpose of obtaining star information by name or AUID from VSX and at least shows how VOTable text could be handled.

Let me know if you have questions.

David

Affiliation
American Association of Variable Star Observers (AAVSO)
VOTable Package

Thanks David,

Unfortunately, the votable from GAIA is binary

<DATA>

<BINARY2>

<STREAM encoding='base64'>

It just seems to me that using a 3rd party package is the way to go with this.  I think this is particularly true for a "standard" format.  Since there is no other uses of VOTable in Vstar, it seems I should just include this in the plugin jar.  Make sense?  Any recommendations on the best java packages to consider?

Cliff

Affiliation
Astronomical Society of South Australia (ASSAU)
VOTable Package

Hi Cliff

It does make sense yes.

This Java library looks promising: Starlink Tables Infrastructure Library

Have to take a closer look but there's jar files and source code. We can go either way for a plugin.

There's also this documentation from IVOA that you have probably seen:

  http://www.ivoa.net/documents/VOTable/20130920/REC-VOTable-1.3-20130920.html#ToC39

Happy to talk more about all this.

David

Affiliation
American Association of Variable Star Observers (AAVSO)
Broken link fixed

Hi Cliff,

Thank you for letting us know about the broken link on the VStar Plugin Library page. It is now fixed.

Best regards,
Sara

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Progress

David,

Thanks for checking in.  I'll look at the ant tool.

I have made some progress.  I created a standalone python script to pull the GAIA photometry and work with it.  Python because it is hard to beat the astropy package functionality.  The VOTable access is pretty trivial with it.

Anyway, by goal was to implement the GAIA to VRI passband transformation and see how accurate it is.  I compared it to the Landolt North standard stars and, for V, to APASS.  Across those 300+ stars the transformed GAIA values are pretty good.  For V, the GAIA values overall beat APASS.  Still there are enough anomalies that I think it is better to import the native GAIA passbands into VStar.

As with all prototyping, I learned something important.  As I took a closer look at the GAIA documentation, I realized there is a CSV option that you can add to the URL.  So I will soon move on to building the CSV based plugin.  I suspect that the existing plugins will give me what I need to know.

So the help I need is on how to handle the GAIA passbands identifiers.  They are green:G, blue:Gbp and red:Grp.  So do we just decide ourselves and make this VStar specific, pick something that does not conflict with the filter options here?  Or should we take it to a wider group for discussion?

thanks very much,

Cliff

 

Affiliation
Astronomical Society of South Australia (ASSAU)
GAIA bands

Hi Cliff

Good news re: the CSV option. I know what you mean about Python + astropy.

There's red, green, and blue bands we could use. Have a look here starting around line 95:

  https://sourceforge.net/p/vstar/code/HEAD/tree/trunk/src/org/aavso/tools/vstar/data/SeriesType.java

Alternatively build and run the example GeneralTool plugin via the ant tool to see them listed in a dialog.

However, I think those bands are intended to be used for visual obs of some sort but they may also be legacy.

So, to your question: I think it makes sense to take it to a broader audience, but these can easily be changed during coding including creating new ones if necessary, as we do for ASAS and other ob src plugins.

I might ask Sara and Bert to look at this topic in case they haven't seen it.

David

Affiliation
American Association of Variable Star Observers (AAVSO)
Filter ID Choices

I will be the first to admit that I don't have enough experience with AAVSO to make a good judgement on what IDs to use.  However, I think an important consideration is that  no ground based observer is likely to be able to replicate the passbands of GAIA.  The blue and red are actually artifically generated by integrating the spectrum as I understand it.  And then, of course there is the space based aspect.

 

best regards,

Cliff

Affiliation
American Association of Variable Star Observers (AAVSO)
First Cut Working

David,

I have the first cut working.  I put aside the decision on filters and just use blue,green,red for now.

When I set it up for InputType.FILE_OR_URL it works as expected.  I can import directly from a URL like

http://geadata.esac.esa.int/data-server/data?RETRIEVAL_TYPE=epoch_photo…

or from a file saved from using that URL in a browser. 

 

Now I thought it would be useful to prepopulate the URL with everything through and including "ID=" so the user only needs to add (paste) the GAIA source_id.  I thought perhaps that was what the

ObservationSourcePluginBase.getURLs was for.  However, that method does not seem to get invoked unless the InputType.URL is used.  And then it appears it expects that to be a complete URL which of course fails if I just put in the first part.

 

Now the GAIA archive (https://gea.esac.esa.int/archive/) queries will supply the URL and anyone likely to use this plugin will be familiar with it.  However, they would need to modify the URL they get with the inclusion of "&FORMT=CSV" as I have dodged the VOTable work in this plugin.

 

Do you have any suggestions?

 

thanks,

Cliff

Affiliation
Astronomical Society of South Australia (ASSAU)
First Cut Working

Hi Cliff

It's great that you have a first cut working!

The way to do what you are thinking about is to request input via a dialog to get the GAIA source ID in getURLs() and return a list containing a single URL, somewhat like this:

import javax.swing.JOptionPane;

...

public List<URL> getURLs() throws Exception {

List<URL> urls = new ArrayList<URL>();

String id = JOptionPane.showInputDialog("Enter GAIA Source ID");

if (id != null && id().trim().length() != 0) {

urls.add(new URL(BASE_URL + id.trim());

}

return urls;

}

Make sense?

David

Affiliation
Astronomical Society of South Australia (ASSAU)
Feedback

Hi Cliff

This is looking good! Works fine for me.

Minor typo in dialog: the word transformation has some transposed letters.

The R, G, B band types probably should be SeriesType.Blue,Red,Green rather than TG, TR, TB.

David

Affiliation
American Association of Variable Star Observers (AAVSO)
Easily Changed

Thanks for pointing those out.  I'll consider it done unless I turn up something as I work with it.

Thanks for all the help.  I learned a lot about VStar.

Cliff

Affiliation
Astronomical Society of South Australia (ASSAU)
No worries

Hi Cliff

No worries.

When you think it's ready, we can include it in the standard set of plugins that are available with each release. Making the source available in GitHub or within the VStar Subversion repo will make that easier (especially the latter).

I usually write a short doc to show usage etc.

Don't be afraid to ask others via the forum to help (test, docs). There are people who have expressed interest in helping in recent posts.

David

Affiliation
American Association of Variable Star Observers (AAVSO)
Making Source Available

David,

So my suggestion is to conduct my own tests for a short time and then try to enlist others.  Of course I will provide documentation.

It seems this could happen in parallel with getting the documentation and source code to some public place.  I don't currently have any reason to use either GitHub or Subversion and so have no preference.  Again I would ask for your suggestion and a bit of help. 

Do you want the package name changed to org.aavso... or is the gaia.plugin package path fine as is?

Cliff

Affiliation
Astronomical Society of South Australia (ASSAU)
Making Source Available

That all sounds good Cliff.

It's not critical that the package name be changed but it may simplify things with respect to the bulk build process. Likewise, getting it into Sourceforge with Subversion. Happy to help.

I use git for pretty much everything else these days. I started using Subversion for VStar several years ago and have had little reason to change. That may change in future.

David

Affiliation
American Association of Variable Star Observers (AAVSO)
astroquery.vizier

FWIW Gais DR1 is accessible through the astroquery.vizier module.  The module provides pretty flexible output via astropy.table.

I haven't tried to use it.  Just found it when I was trying to find a way to access Pan-Starrs..

Jim

Affiliation
American Association of Variable Star Observers (AAVSO)
Note about Gaia magnitudes -

Note about Gaia magnitudes - G is like "Gaia" - it is broadband/white light magnitude from astrometric CCD-s. And Gaia's blue and red magnitudes should be just sums of respective, very wide in wavelength scale, blue and red spectra (see https://www.cosmos.esa.int/documents/29201/302420/IN_photometric_instrument-1.pdf/5c78c8d8-35a6-4258-be6e-f42a3d2b624c). So one should be moderately careful when comparing those magnitudes with other systems, even when using conversion equations. ;-)

Affiliation
American Association of Variable Star Observers (AAVSO)
GAIA Magnitudes

Thanks Tonis,

I attempted to get some discussion on this topic over in the photometry forum, but got none.

Over there I linked to a graphic that supports your more detailed link.  I was encouraged to look at the transformations by Appendix A of this paper.  Once I had the transformation working, I compared a few hundred Landolt standards with encouraging results.  I ended up implementing the transformation as optional.

So far my impression is that the photometry available on the 500,000 GAIA "variables" is more interesting to me than most others.  That's fine.

best regards,

Cliff

Affiliation
American Association of Variable Star Observers (AAVSO)
Enable Clipboard with Webstart

During the testing of my plugin I find the clipboard is not operating for my dialog box when I start VStar from webstart.  It does work if I start VStar locally.  I see that all the other plugin dialog boxes do work.  What's the trick?

thanks

Cliff

Affiliation
Astronomical Society of South Australia (ASSAU)
Enable Clipboard with Webstart

Hi Cliff

I'm wondering whether this is a security policy setting.

I have plans to move away from WebStart at some point as well.

Having said that, the fact that you don't see the same problem with other plugins is odd.

Once the code is in SF I'll build and test it too to see if I can reproduce what you see.

David

Affiliation
American Association of Variable Star Observers (AAVSO)
SeriesType

Hi David,

Is it possible to extend SeriesType adding user-defined series types from an ObservationSourcePlugin? Or it is a fixed list?

Best regards,

Max

Affiliation
Astronomical Society of South Australia (ASSAU)
SeriesType

Hi Max

Yep, SeriesType can be extended with new series. Indeed, most observation source plugins do exactly that.

Further, they appear in Preferences so that colour and point size can be changed.

David