Mon, 01/20/2020 - 08:20
Hello,
I'd like to download observations between dates but am running into a problem with the RESTful API and wonder if there is any documentation that might help.
Briefly, I'm following the example in this thread: https://www.aavso.org/api-retrieving-entire-history-star-not-given-period-api-working-ok
It works well, except that the comma-delimited results so not come back quoted. This means that commas showing up in any field will be interpreted as field separators and parsing will not work.
I'd be happy to use any other method to get observation data between two dates, but I'm banging my head trying to figure out the parameters needed to make this work. A document describing the API would be great.
Thanks for any help.
-Matt
I'm no expert in this particular API, but in similar circumstances I've simply changed the delimiter to something not found in the data fields themselves. Using a semi-colon delimiter has saved me a lot of grief in years past.
In recent code to call the VSX API, I actually used &delimiter=@@@ and it worked reliably, though it was a bit more work to parse the returned text.
So the URL that worked for me was in the form:
url = 'https://www.aavso.org/vsx/index.php?view=api.delim&ident=AW+Cyg&tojd=24…@@@'
and then I split each returned line with that same delimiter; e.g., in python:
byte_text = urllib.request.urlopen(url)
text = [line.decode('utf-8') for line in byte_text]
data = [line.split('@@@') for line in text]
Your mileage may vary, and I certainly don't represent this as AAVSO-approved code; it's just the code I rolled on my own to do the job. Good luck.
Hi Eric,
Much appreciated! I've tried that approach but found that no single character is immune from being used by submitters as subfield delimiters or annotations: all of [@ | ; ,] and others show up in some field or another. I think your multi-character delimiter is probably the only way to go (though as you point out, it complicates the parsing--the package I use expects single-character delimiters).
This is really not a problem, but part of me rebels at the thought of implementing a hack, no matter how rational and useful, when there may be a correct solution lurking in the documentation.
Thanks for your help.
-Matt
Hi Matt,
Fields that may contain a comma should normally be quoted in the output if you use api.csv (instead of api.delim). Do you have an example for which that is not the case?
You might also benefit by using e.g. https://www.aavso.org/vsx/index.php?view=api.object&ident=V371+Cyg&data&csv&minfields. The output you get is xml, but the observations themselves will be in csv format in the data section and limited to the most important fields.
There is no full documentation of the API's yet. Much will depend on the move to the new server. There the minfields option will also be available for the api.csv and api.delim flavours.
Patrick