I was wondering how hard it is to go combing through public data to search for new variable stars?
Since I already participate in an exoplanet search on Zooniverse in TESS data, I come across a TON of variable stars that aren't transits.
I am able to use the exofop search from Caltech and see specifics about the star.
Usually they are so new that they only have a small number of surveys that have looked at them, but a wide range of magnitudes if they are variable.
Would this be considered discovering if it's not listed as an already known variable? I have ten that aren't in VSX but need more followup in order to verify they are variable or not.
Any help is appreciated if I'm not on the right track here!
You’re exactly in the same situation like me 2.5 years ago! I was just after Exoplanet Explorers, the Zooniverse project about Kepler’s K2 mission. I was also intrigued with the massive amount of unreported variable stars, so I wrote a message to Sebastian Otero (VSX moderator) what can I do in this situation.
It was a little different that time, because to make a submission using space-based observations you had to make some your observations to confirm the variability. So, I downloaded the latest Campaign data (12) and I went through the light curves (with LcViewer) to find what’s the easiest object to work on. It turned out that highest chances to detect are for HD 5843 (0.03 mag DSCT of a 8 mag star). I made the same thing only for three other objects, because I discovered that with own observations I could make much more discoveries. For example, during HD 6121 observation (another DSCT star) in two nights I detected variability of two more stars that were in the background. And they were real, as confirmed by Kepler (BY+UV and GDOR). And there were much more coming from exoplanet transit observations from unobserved areas by Kepler. I countinued to submit like that after about 50 variables, before I completely went back to data mining just after two months. And it still continues!
I am telling that it was quite different, because currently people regulary submit TESS variables (and you can do so) without making own observations. Back in 2017, in such cases I would have to put “Kepler” in Discoverer field instead of my name and the use of high precision space-based measurements was something like a reward for obtaining some observations. I liked that difficulty, so that’s why I’m skeptical to treat current TESS as true discoveries. This is because these pre-generated light curves were already seen by a large amount of people (because there are just a few thousands objects). The reason why they are not in VSX yet is because there are too many to submit fastly and there are probably 1-3 teams working on papers like “TESS variable stars from Sector 15” and it takes a few months before they are published. Also, TESS has huge pixels that makes the source identification often uncertain in dense fields. In this case, follow-up observations are greatly encouraged, even more than for Kepler when the scale at start is good enough. But I talked with Sebastian that you can go this way and still submit with your name instead of putting “TESS” in Discoverer field (and your own star name as well). That’s why Polish Team never submit TESS variables as alone, but actually others do.
I prefer to work with ZTF, PanSTARRS-1 and ASAS-SN (all are ground-based) as a data mining variable star search, because often you’re the very first seeing the light curve of the variable and they are often nicely masked, which makes discovering more fun!
EDIT: Also remember to check VizieR if it's actually a known variable! Some KELT variables might be there, but those haven't been imported to the VSX yet.
This was an amazing writeup!
I just saw your post from last year about your massive haul of new objects!
Are there any specific links to the ZTF, PanSTARRS-1, and ASAS-SN that you can share so I can start to browse them?
Thank you again for all of your help and the goldmine of information!! :)
You can search for candidates for new star variables in APASS data. To do this, go to the APASS website, download small radius objects and build a Verr (Vmag) or Berr (Bmag) chart. Candidates for new variables will be above the bulk of stars-points in the chart. You will only have to check these candidates by VizieR, VSX, ASAS-SN database and if no one found the object, get ASAS-SN, ZTF, CRTS, SuperWASP, NSVS photometry. The second way to search is to download DSS (DSS Plate Finder) pictures, run them through the VAST or Muniwin search program and find candidates for new star variables. The third way is to search ZTF images, but you can make sql-queries on their site. Usually the variables of stars have high magRMS. Using a polygonal query and playing with MinMAG and MaxMAG and goodobs, you can find candidates for new star variables. The fourth way is just to look through the light curves in the TESS data. Even bright stars in their data are not illuminated and show excellent results. You can check undefined and questionable objects marked as dubios in ATLAS data.
DSS Plate Finder: https://archive.stsci.edu/cgi-bin/dss_plate_finder
APASS (DR10): https://www.aavso.org/apass-dr10-download
All the best, Ivan
In Excel, build a Vmag(Verr) graphs and you wi ll see everything. Verr=STD. If there are no Vmag data, APASS has data in the infrared photometric system (r,g). You can make graphs in r(rerr), g(gerr) and other filters.
For Verr greater than 0.4, everything is most likely already detected by automatic reviews. You should probably check candidates with Verr between 0.1 and 0.4. That is to hope to detect low-amplitude objects. You can also make sql-queries to the TAP Vizier system to select candidates for new variables. The scripting language is not very complicated there and you can learn it even for a beginner. As for TESS. I talked to the authors of the project. They said that they will NOT be able to create a CATALOG OF VARIABLE Stars! Because their mission is not variable stars but exoplanets. This is a chance for fans to contribute to the science of variable stars.
I started my search back in 2005 with an NSVS review https://skydot.lanl.gov/nsvs/nsvs.php . The criterion for the selection of candidates for new variable stars Magscat/Magerr >2, i.e. the average amplitude of brightness oscillations divided by the average brightness measurement error should not be less than two. The NSVS data most likely still (!) contain undetected new stars with small amplitudes of brightness oscillations. You can try to search. You can also search for new stars like RS CVn, select candidates from the 1RSX catalogue of X-ray sources (ROSAT observatory) via Vizier. You can search the PS1 data for new cataclysmic variables also by X-ray radiation. If you look at the IRAS data, you can find new red star variables (semi-regular, incorrect, myrids). Datamaining the variable stars is very fascinating but requires perseverance, skill, experience and enthusiasm. Go for it! The road will be overwhelmed by the coming!
All the best, Ivan Sergey (SIV)
So I must be wrong! I was pretty sure they're going to do the same with Kepler data, but probably it would be too problematic with such large pixels. Now I understand why there's so large interest in TESS variables nowdays!
Sorry for another question, but should I also search the same paramaters in a Berr chart?
Example: Search 0.1-0.4 candidates but ignore most over 0.4 and under 0.1?
Hi Chris, Ivan et al.,
I do not recommend using APASS for data-mining purposes. The scatter in the observations is large and there are just a few observations to analyze. There are plenty of surveys like ASAS-SN, TESS, ZTF, NSVS, ASAS-3, Kepler, CRTS, SuperWASP, that will be more productive in that regard.
As Bert said, each survey has its vagaries, you need to get used to their problems and advantages.
You have to keep in mind the survey's resolution (nearby stars will be blended in the case of surveys using large pixel sizes) and their useful range (saturation point at bright magnitudes and limiting magnitude).
There are some tips about that in the Data-mining section in the VSX FAQ page.
Thanks for the input. I'll keep that in mind and likely favor TESS data to start out despite the larger pixel sizes and potential for blending.
I'll take a look at the link you provided shortly.
Have a good day!
That's awesome news about TESS! It's a goldmine of data for variables that have a very low amplitude change among others.
I'll have to start going through that data as well!
Thanks again for your help!
Have you or anybody else actually thought about making a tutorial for how to query the data and search for new variables?
We at HQ have seen a definite increase in desire for data mining resources and information. I have personally done a lot of it and can testify to the fact that each survey is different and you have to learn a new process for each source you select. Because of this the bar for entry into data mining can be pretty high, but it is very worthwhile.
We have discussed a number of options for how to support this and one of the things that we are very strongly considering is having a data mining section in the same vain as the observing sections we have currently. How quickly this can happen will depend on the how strongly the AAVSO community advocates for it and is willing to help with its development. We definitely value your input and if these are the sort of resources you would like available, please let us know.
Staff Astronomer AAVSO
This is exactly the topic I want to bring up at the next AAVSO Ambassador meeting.
Since AAVSO has always been trying to reach out to a more diverse age range of users and contributors (younger members, etc.). I strongly believe that getting this sort of program off the ground combined with the efforts of the AAVSO Ambassador program has the potential to draw in younger members who desire to have a direct impact in discoveries.
My suggestion would be to create a data mining section that introduces data mining in a simple form. Such as going through TESS data rather than ZTF data that requires some knowledge of SQL programming.
As you previously mentioned the learning curve could be a little steep, but the people that could overcome that curve are the ones that we would want searching for and submitting new variables.
I am still very new at data mining (the strongly worded email I got back from a VSX moderator about a submission I made speaks to that...) but am eager to learn more and be able to submit useful data.
I would be more than willing to support this effort in anyway that I possibly could.
Again, in my opinion this sort of page and "Discovery" program could be a major draw to get new members to join AAVSO.
Please feel free to reach out on here or email me to discuss further!
Chris Colvin AAVSO Ambassador
Yes! The TESS results are amazing! I looked at some stars with my eyes wide open and there was no limit to my surprise, especially when I first started working with TESS data. In my opinion, working with TESS data is the most advanced and fashionable direction today. And yes - it's a gold mine today. I can name some pioneers: Nikolay Myshevsky, Vitaly Nevsky, Maxim Pyatnitsky.
In 2006, I created a topic to share my experience in finding new star variables: https://astronomy.ru/forum/index.php/topic,22265.0.html There are almost 190 pages but in Russian. If you throw away the offtop, extraneous debate, this topic more or less reflects most of the work of the datamainer of the search engine for new star variables.
The most important teacher and mentor in datamining, we all have the very esteemed Sebastian Otero. Thank him so much for his tremendous work! Also Professor Denis Denisenko (Russia) helps me a lot.
All the best, Ivan
Experienced dataminers working for many years with photometric data (open source) are very interested in the latest data, advanced ideas and search methods that exist today. Therefore, I wish you to continue to support the development of information this area on the site AAVSO. I am often asked by novice observers and how do I look for new variables? Do I need to search with a telescope or in the XXI century telescope for this purpose is not needed? It is a pity that there is no big and detailed manual for beginners datamainer. I have to tell the basics to start working with open sources.
All the best, I.Sergey
I think I'm onto something with the search.
After following your insstructions I am able to comb through APASS data and pick out already confirmed variables that show up in the ASAS-SN database when I search the coordinates of my "suspect."
Thanks for all your help!
Bert's comments about the bar to entry into data mining and a forum along with Ivan's comment:
The most important teacher and mentor in datamining, we all have the very esteemed Sebastian Otero.
along with others such as Chris re: a possible tutorial, leads me to wonder whether a data mining CHOICE course would make sense?
Hi, Sebastian, Chris!
Yes, indeed, there is erroneous photometry in the APASS data, which leads to abnormally high STD (Verr, Berr). However, despite this, I was able to detect new star variables. For example, an article in the German circular BAV: https://www.bav-astro.eu/images/Up_Journal/BAVJ020_R2_2018.pdf ,
All the best, Ivan
Your search method of APASS definitely works.
I went through a bunch of data points and found already confirmed variables just by viewing the Verr and Vmag alone. (Verr >01. and Verr <0.4) After finding the suspects I then searched for them in VSX and ASAS-SN. Found six confirmed variables in the data.
I even had my spreadsheet to do automated formatting to highlight the most likely and least likely data to present a variable. It worked!
Only one datapoint that I believe could be a variable out of over 32,000 stars I searched in APASS data.