Direct web query of VSX/VSP

Affiliation
Royal Astronomical Society of Canada (RASC)
Mon, 01/10/2011 - 14:10

I'm a planetarium software author (ECU) and I'd like to update it so users can do an aavso query of an object.

I used to use: http://www.aavso.org/cgi-bin/searchcharts3.pl?name=VARNAME

but it no longer works.

I see that VSX in the about page says:

"Querying the Database

Web-based tools for querying VSX in various ways are available to the public. The public data returned in a query transaction contains all the accumulated data for the most recent revision level for each star in the recordset, including details of all modifications made to the data, and references to support those changes. Previous revisions may also be viewed. The public interface does not include the means for modifying the data in any way, but a form for sending suggestions for modification to the Project Administrator is made available."

Is there documentation for this?

Also, since we keep being told: "VSP is not a planetarium program" :-), I'd like to better integrate it with one, as that is what many of us use to plan our observations. It would be really nice to click on a star in the planetarium and press a button and up comes a VSP chart.

Can I bring up a VSP chart from an appropriately formatted web link? If so, where can I get the documentation?

--- Dave

Affiliation
Groupe Europeen d'Observations Stellaires (GEOS)
VSP HTTP GET Specification

Hello Dave

I sent you the HTTP GET Specification for VSX.

And I do advocate to reintroduce again the old VSP interface as it was at the old
AAVSO web site and publish a specification on how to use it.
All old VSP links are broken including all the ones in older AAVSO alert notices
and special notices (using the URL parameter: pickname=) and
especially the ones from VSX.

CS
Wolfgang

Affiliation
Groupe Europeen d'Observations Stellaires (GEOS)
VSX HTTP GET spec

It originates from the longe gone Bugzilla page:
http://www.aavso.org/bugzilla/attachment.cgi?id=5&action=view
So parts might be outdated. The last time I checked all parameters, a few of the hi vs. low range values had to be reversed to get what I expected.

A recent example I stiched together is:
All vars and suspects in VSX that are brighter than 6.5 mag at max:
http://www.aavso.org/vsx/index.php?view=results.get&coords=180.0+0.0&format=d&size=420&unit=1&geom=b&order=5&maxhi=-30&maxlo=6.5

___________________________________________________________________________

 

 VSX HTTP GET Specification

 

This is the preliminary specification for HTTP GET queries at VSX. The IP address for the server is temporary and will eventually become vsx.aavso.org, but not right now. The common portion of the URL is:

http://www.aavso.org/vsx/index.php?view=results.get

Following this root URL, any workable combination of the below described name/value pairs may be appended. They do not need to appear in the URL in the order given here, and they do not all need to be present. Selective absence and presence of these values from the URL are what make for all the different search types. All non-numeric argument values may include the SQL wildcard character (%) in order to create wildcard searches, but the character must be URL-encoded as %25. Spaces must be URL-encoded with the whitespace replacement character (the plus sign). Plus (+) characters present in designations or coordinates must be URL-encoded as %2B.

&coords

The central J2000 RA/DEC coordinates for a radius search, expressed sexagesimally (by default), or in decimal degrees if format is set to d. Northern hemisphere coordinates must have the plus (+) sign URL-encoded as %2B. Space characters between all other figures must be replaced with the URL whitespace character (+). The order argument (which see) must also be included in the query string with its value set to 9 in order to prompt VSX to display distances from the central coordinates in the results listing. Default is empty string (no radius search).

&ident

Object identification for name searches. Space characters must be replaced with the URL whitespace character (+). Other special characters may also need to be URL-encoded. Default is empty string (no name search).

&constid

The numeric constellation ID for constellation-bound searches. See http://www.sai.msu.su/groups/cluster/gcvs/gcvs/iii/constel.dat for codes. Default is 0 (not bounded by constellation).

&format

Explicit specification for format of coords. For sexagesimal, this value should be s. For decimal degrees, this value should be d. Default is s (sexagesimal).

&geom

The geometry for central coordinate-based searches. For radius searches, this value should be r. For box searches, this value should be b. Default is r (radius search).

&size

For box searches (geom=b), the width of the box. For radius searches (geom=r), the radius of the circle. Expressed in the units specified by unit (see next). Default is 10.0.

&unit

The unit of measurement used for the value given in size (see above). For arc degrees, this value should be 1. For arc minutes, this value should be 2. For arc seconds, this value should be 3. Default is 2 (arc minutes).

&vtype

The variability type. Used to limit search to only stars of specified variability type. Typically, this can be set to a standard GCVS variable type abbreviation (see http://www.sai.msu.su/groups/cluster/gcvs/gcvs/iii/vartype.txt), but VSX holds in its database many different non-standard variable types taken from other catalogs. Default is empty string (no limitation to variability type).

&stype

The spectral type. Used to limit search to only stars of specified spectral type. Default is empty string (no limitation to spectral type).

Range values

The following pairs of query string arguments are used to perform searches on values greater then, less then, or between given values. Including one value from the pair establishes only an upper or lower limit, and including both established a complete range.

&maxhi
&maxlo

The upper and lower limits to a search within a range of maximum magnitudes. Defaults are empty strings.

&minhi
&minlo

The upper and lower limits to a search within a range of minimum magnitudes. Defaults are empty strings.

&perhi
&perlo

The upper and lower limits to a search within a range of periods. Defaults are empty strings.

&ephi
&eplo

The upper and lower limits to a search within a range of epochs. Defaults are empty strings.

&riselo
&risehi

The lower and upper limits to a search within a range of rise times. Defaults are empty strings.

&yrlo
&yrhi

The lower and upper limits to a search within a range of outburst years. Defaults are empty strings.

Additional Fields

&filter

A comma-delimited list of the values representing which class of objects should be included in the search results. This allows for filtering in/out proven variables, suspected variables, and/or non-variables. The value for proven variables is 0, for suspected variables is 1, and for non-variables is 2. For example, if you wanted to search for only proven variables, the value should be set to 0. If you wanted to search for all classes of objects, the value should be set to 0,1,2. Defaults to 0,1 (includes proven and suspected variables, but not non-variables).

&order

A value representing the desired sort order for the search results. The sort order values are as follows:

1 ? by GCVS name sequence

2 ? alphabetical by name

3 ? by R.A.

4 ? by Dec.

5 ? by maximum magnitude

6 ? by variable type

7 ? by spectral type

8 ? by constellation

9 ? by angular separation from central coordinates

Default is 1 (by GCVS name sequence).

Notes

There is a limit of 9999 records that can be returned from any HTTP GET query of VSX.

Affiliation
Astronomical Society of South Australia (ASSAU)
Thanks for the VSX http get

Thanks for the VSX http get spec Wolfgang. I'd like to remove (eventually) all direct database access to VSX, AID etc from VStar, using web services or http get.

Regards,

David

Affiliation
Vereniging Voor Sterrenkunde, Werkgroep Veranderlijke Sterren (Belgium) (VVS)
VOTable

Hi David,

I don't see much use for 1950 coordinates anymore.  You should probably calculate them yourself.

Patrick

Affiliation
Astronomical Society of South Australia (ASSAU)
J2000 vs B1950 coords for HJD conversion

Hi Patrick

Okay. The context is that I have code for conversion of JD to HJD using B1950 coordinates. I'm not otherwise interested in B1950. If I can find a suitable algorithm and validate it against test data for using J2000 coordinates instead for HJD conversion, I'll happily use J2000 instead.

One possible source is:

  http://articles.adsabs.harvard.edu//full/1972PASP...84..784L/0000784.000.html

Do you have any more suitable suggestions?

Thanks.

David

Affiliation
Vereniging Voor Sterrenkunde, Werkgroep Veranderlijke Sterren (Belgium) (VVS)
Heliocentric correction

Hi David,

There are a few online HJD calculators around which work with J2000 coordinates, most with the source code available.  Here are links to just a few: http://mira.aavso.org/pipermail/aavso-discussion/2009-July/033152.html, http://britastro.org/computing/applets_dt.html, http://search.cpan.org/~rcreager/Astro-Time-HJD-0.02/HJD.pm.

Patrick

Affiliation
American Association of Variable Star Observers (AAVSO)
Query for charts

The query mechanism works pretty well for charts. It is documented at https://www.aavso.org/vsp-api .

So you can build up a string, eg  https://www.aavso.org/apps/vsp/api/chart/?star=SS+Cyg&fov=60&maglimit=14.5   . This will return an xml or json object that will include a field defining the url of a chart image, eg   "image_uri": "https://www.aavso.org/apps/vsp/chart/X22562FZL.png",

I see from other posts you are working to port your CHARA app from VB6 to VB.Net. Big step but truly necessary as VB6 loses support. If you get good at vb.net, you can help out on VPhot which is up for revisions this year.

Cheers,
George

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Hi George,

Hi George,

Looking forward to porting to vb.net - just waiting on a couple (literally) of problems.

The string I am looking to build up is like the one you quote BUT it's my wish to include a string variable instead of a literal - i.e, not ...star=SS+Cyg... but star=$starname... (where $starname is a hypothetical PHP variable from the calling website)

Is VSX api still valid?

I followed the (2011) api info in this thread but I'm not getting back xml or json data, just the html content of the index page.

My test GET request is: 

http://www.aavso.org/vsx/index.php?ident=1SWASP+J022916.91-395901.4&vie…

This is constructed in Python from the code below. Can anyone guide me as to what I'm doing wrong (or even if I've got the right url):

def get_vsx_data():
    url = "http://www.aavso.org/vsx/index.php"
    star = "1SWASP J022916.91-395901.4"
    params = {}
    params['view']='results.get'
    params['ident']=star
    response = requests.get(url,params=params)
    if (response.status_code > 400):
        raise AAVSO_VSX_Misc_Error(entry.status_code, entry_reason)
    return(response.content)

Hi Patrick

Hi Patrick

Yes that works fine for a single star but I can't get it to work for a query that returns a list of stars. For example the following returns just an empty list. Is there something obvious I'm missing? Thanks. Nigel.

def get_vsx_data():
    url = "http://www.aavso.org/vsx/index.php"
    params = {}
    params['view']='api.object'
    params['format']='json'
    params['vtype']='ep'
    response = requests.get(url,params=params)
    return(response.json())

 

Affiliation
Vereniging Voor Sterrenkunde, Werkgroep Veranderlijke Sterren (Belgium) (VVS)
VOTable

Hi Nigel,

Yes, api.object is intended to get information on a single object by name only.  There is no specific API with JSON output to query the database in the same way as the HTTP GET query documented in comment #4, which apparently is what you want to do.  Replacing view=results.get with view=query.votable as mentioned in comment #8, will give you VOTable output instead of HTML.  Here is an example

Patrick

Thanks again Patrick. I've

Thanks again Patrick. I've not played with VOTable before (just Json) so will have to research a bit.

btw is this thread the only source of detailed info on AAVSO apis? Don't want to keep wasting your time if there's more detailed documentation elsewhere.

Many thanks

Nigel

 

Parsing VSX API VOTable results

Hi Patrick

Is there a recommended Python parser for the votable output from the VSX api? Astropy.io.votable parse and parse_single_table don't work (main problems being that the votable output doesn't specify arraysize for character fields so defaults to 1 which loses pretty much all the data).

If not I guess I can write my own.

Thanks

Nigel

Simple parser

For now I've constructed a simple parser that parses the VSX api results into a python list of dictionary objects. Code is provided below in case it is of interest (I don't claim it is Pythonic !).

 

# get_vsx_data
#
# Returns the results of a query to the AAVSO VSX API to retrieve target data.
#
# Input:    url         VSX API url. Defaults to http://www.aavso.org/vsx/index.php
#           params      VSX API query parameters. Defaults to query.votable view.
#                       Additional parameters defined here: https://www.aavso.org/direct-web-query-vsxvsp
#
# Output:   List of dictionary objects with one dictionary per target (star)
#           returned by the VSX API. Dictionaries keyed by the VSX field id. 
#
# This function uses the BeautifulSoup HTTP/XML parser:
#   from bs4 import BeautifulSoup as bs

def get_vsx_data(url = "http://www.aavso.org/vsx/index.php",
                 params = {'view': 'query.votable',
                           'vtype': 'ep'}   # ep for test purposes only to limit results
                          ):
    
    response = requests.get(url,params=params)
    if (response.status_code > 400):
        raise AAVSO_VSX_Misc_Error(response.status_code, response.reason)
    votableXML = StringIO.StringIO()        # Create in-memory file
    votableXML.write(response.content)      # Save response to in-memory file
    soup = bs(votableXML.getvalue(), "xml") # Parse using BeautifulSoup

    # Load field definitions, checking for missing or duplicate ids
    fields = soup.find_all("FIELD")
    for field in fields:
        if field["id"] is None:
            raise Exception("Field has no id.")
        elif len(soup.select('FIELD[id=%s]' % field["id"])) != 1:
            raise Exception("Duplicate field id.")

    # Build list of dictionaries, one dict per row (star), indexed by field id
    rows = soup.find_all("TR")
    results = []
    for row in rows:
        row_dict = {}
        row_fields = row.find_all("TD")
        if len(row_fields) != len(fields):
            # No. of TD elements in row should always == no. of FIELD elements
            raise Exception("Field and TD numbers differ.")
        fieldNum = 0
        for row_field in row_fields:
            row_dict[fields[fieldNum]["id"]] = unicode(row_field.string)
            fieldNum += 1
        results.append(row_dict)
        row_dict = None
    return(results)

VSX server error?

I'm getting the following response from the VSX api:

<br />\n<b>Parse error</b>: syntax error, unexpected '[' in <b>/media/disk/www/vsx/query/qry_get.php</b> on line <b>73</b><br />\n"

It seemed to be working fine a couple of days ago. Has there been a change in the server code or is it something peculiar about my query?

My params (as passed to requests.get) are as follows:

{'coords': '0.0 +41.0', 'format': 'd', 'geom': 'r', 'order': '9', 'size': '10', 'unit': '2', 'vtype': 'ep'}

Thanks

Nigel

 

Affiliation
American Association of Variable Star Observers (AAVSO)
What do I change the " &delimiter=" for tab separated file?

What do I change the " &delimiter=" for tab separated file?

Affiliation
American Association of Variable Star Observers (AAVSO)
Try %09

Doesn't delimiter=%09 work?

%09 is the standard encoding within URLs for the tab character.