Transcript of AAVSO chat, Thursday, 20:00ET Oct 18 (00:00UT Oct 19) Topic: "Amateurs in the Era of Surveys" [20:01] KevinPKV Small crowd... Hi all... [20:01] MatthewTMT Hi Kevin. [20:01] OSE Hi, Kevin, this is Sebastian [20:01] OSE Hi, Matthew [20:01] MatthewTMT Hi Sebastian. [20:02] KevinPKV Hola... [20:02] OSE How was the first session? I couldn't make it [20:02] MatthewTMT I have been following the binoc sequence work onthe side -- you people are busy! [20:02] OSE [20:02] OSE yeah, that's what kept me away from the earlier chat! [20:03] MatthewTMT Good reason! I missed the early session too -- I was stuck on the Red Line on my commute home at the time. [20:03] OSE [20:03] OSE So we need gossip from someone who was present. Arne?? [20:04] MatthewTMT Arne is meeting with our endowment managers this evening, but should be joining in by 8:30 or so. [20:04] OSE ok [20:05] MatthewTMT I told Arne I would try not to say anything wildly different from his opinions while he's not here. [20:05] |<-- AndrewP has left irc.aavso.org (Quit: http://www.mibbit.com ajax IRC Client) [20:05] OSE you know his opinions that well? [20:06] KevinPKV LOL... [20:06] OSE you won't say something like: "amateurs have nothing to do in the survey era" [20:06] MatthewTMT Well, he and I talk about surveys and visual observing often enough that I know we agree on something and disagree on others. [20:06] MatthewTMT Sebastian: no, I will not say that! [20:06] -->| AndrewP (7cb68717@rox-4FDDE039.mibbit.com) has joined #AAVSO [20:07] OSE well, I think the higher number of oportunities lie in the data-mining field [20:07] OSE there are **too many** surveys online already [20:08] OSE we're getting close to the fact that almost every star (at least down to 17 or so) will have data in some of them [20:08] MatthewTMT Does "too many" imply they're doing the same thing and serve the same purpose? [20:08] OSE No, fortunately not all [20:08] KevinPKV Well, the data mining section is nearly dead. Martin Nicholson, is he the only one doing this activity? [20:08] OSE but at times you have data from 4 differnt surveys [20:08] |<-- AndrewP has left irc.aavso.org (Quit: http://www.mibbit.com ajax IRC Client) [20:08] MatthewTMT Yes, and I think that's a good thing. [20:09] OSE some is better than the other [20:09] OSE etc [20:09] OSE you hve to make choices sometimes [20:09] -->| AndrewP (7cb68717@rox-4FDDE039.mibbit.com) has joined #AAVSO [20:09] MatthewTMT Sure. [20:09] OSE or you'll spend a lot of time downloading and analysing data. [20:12] MatthewTMT An "active group" requires an active group. If people are into it, there will be a data mining section. If not, there won't be. [20:12] OSE Currently the number of people data-mining databases has grown a lot [20:12] OSE at least in the photometry aspect [20:12] OSE we received 250-300 new stars a month submitted to VSX [20:13] OSE most of them come from data-mining [20:13] MatthewTMT What databases are providing the richest source material? [20:13] OSE NSVS, MACHO, OGLE, CRTS and ASAS being the most used DBs [20:13] OSE I was just writing the reply before reading the question! [20:14] MatthewTMT I haven't checked ASAS recently, but it will be great when Pojmanski has that fully online again. [20:14] OSE It is rather unstable... It is down often [20:15] OSE and we all think that could be the last time we saw those observations [20:15] OSE It is the easiest one to handle by far [20:15] MatthewTMT I traded a few emails with him awhile ago, and got the impression he's really trying. He was also somewhat horrified that ASAS was seen as discouraging some visual observers -- he said that wasn't the intention at all. [20:15] OSE HJD-2450000 plus magnitudes and errors [20:15] OSE yes [20:16] OSE he was very kind and didn't have any problem with credits or the like, e.g. from stars being found with ASAS data [20:16] MatthewTMT And he's right, it absolutely shouldn't. [20:16] MatthewTMT Even if the current availability issues weren't happening. [20:16] OSE but I lost contact a couple of years ago after another one of their system failures [20:16] MatthewTMT I think our last exchange was early 2012. [20:17] OSE he must have got tired of me then :)) [20:17] OSE It's sad that ASAS North data never got online [20:18] MatthewTMT I never dug into that issue, but yeah that'd be a fascinating resource. [20:18] KevinPKV Perhaps a data mining article would increase member interest... Perhaps a pilot like Harvard Annals... [20:18] OSE do you know if by early 2012 ASAS was still taking images? [20:19] MatthewTMT I wouldn't want to comment until I reread those emails -- it's been awhile. I can get back to you tomorrow. [20:19] OSE Kevin, it's true [20:19] OSE MatthewTMT ok [20:20] OSE Spaeking of the current data-mining activity, a lot come from non-AAVSO members [20:20] KevinPKV Perhaps a good data source with a valid data mining research objective [20:20] OSE and I'd dare to say tht 90% comes from Europe [20:20] MatthewTMT An article in what sense? How to, or an example of a successful project? [20:20] -->| MDAV (42d64b9b@rox-4FDDE039.mibbit.com) has joined #AAVSO [20:20] OSE I was thinking of an article on the different databases and how to deal with them [20:21] KevinPKV A pilot data mining exercise to publicize the activity... [20:21] -->| Pluto (4a3cd16d@rox-4FDDE039.mibbit.com) has joined #AAVSO [20:21] |<-- MDAV has left irc.aavso.org (Quit: http://www.mibbit.com ajax IRC Client) [20:22] OSE One can choose a star with data in different surveys and use it as an excuse to describe the pros and cons of each one [20:22] MatthewTMT I think both of those are potentially interesting, but is this something that you think should come from HQ or something that should be community led? [20:22] OSE good question, it would be nice if people can share their own experience [20:23] OSE but I note that usually people are focused in the use of one database and don't use other ones [20:23] OSE It requires time and effort [20:24] KevinPKV I know nothing about data mining... Busy archiving... [20:24] MatthewTMT Depending on the project and the database being used, just the act of searching a database can be challenging, let alone trying to get astrophysics out the other end. [20:24] OSE that's a kinf of data-miningthe data you archive will be used for data-mining in the future [20:25] MatthewTMT There are a couple of universities in the US (Michigan?) whose astronomy programs have launched graduate astronomy concentrations in data mining. [20:25] OSE Matthew, most of the databases are very difficult to handle [20:25] OSE with calibration issues, blended results, misleading error figures and so on and so forth [20:25] OSE you need to take all that into account before analysing or we could jump to wrong conclussions [20:26] OSE you could jump I meant [20:26] MatthewTMT Definitely. We have a few database projects going on at HQ, and we're seeing some issues requiring more hands-on treatment than an automated pipeline can deal with. [20:27] OSE Yes, automated classification nd analysis has led to half of the current elements in VSX to be potentially wrong [20:27] MatthewTMT Ephemerides you mean? I'd believe it. [20:28] OSE If people are willing to correct those, that would be another source for datamining [20:28] OSE Not only that, variability types, besides periods, epochs [20:28] OSE if you classify an eclipser as a pulsating variable, you'll be giving an epoch of maximum instead of minimum so... [20:29] OSE add that the periods are usually wrong [20:29] MatthewTMT That's been a perennial problem. [20:29] |<-- AndrewP has left irc.aavso.org (Quit: http://www.mibbit.com ajax IRC Client) [20:30] OSE Earlier this year I hace a CS webinar on the matter, calling out for people to help [20:30] OSE I got a couple of replies [20:30] OSE what I was told was that [20:30] -->| AndrewP (7cb68717@rox-4FDDE039.mibbit.com) has joined #AAVSO [20:30] OSE correcting a wrong star gives you no credit, people want to discover something [20:32] MatthewTMT Discovering something gets more column-space in Sky & Telescope. [20:32] OSE that's the point [20:32] KevinPKV Only problem with data mining is lack of romance... Attracting new people to do that activity... [20:33] OSE Probably when you start getting results you eventually fall in love with the activity [20:33] MatthewTMT You have to cast it in a way that appeals to the right kind of person. I bet there are a lot of comp sci, math, or statistics-types who would be turned by to the challenges involved. [20:33] OSE a team from belarus has discovered hundreds of variables by data-mining the NSVS only [20:34] OSE Yes, see how many people have signed in to Planet Hunters [20:34] KevinPKV Data archiving had the hook of classic stars and well known old observers... [20:35] OSE Here the hook is probably that you are solving the elements and type of a star noone else in the world had seen before [20:35] ArneHQA back [20:36] MatthewTMT Yes, and it appealed to people who wanted to "protect" data or make it usable. The utility aspect appealed to several people -- some didn't know much at all about the AAVSO's history, they were just intellectually stimulated by the preservation and historical aspect of it. [20:36] OSE Ok, delete everything we've said till now ;)) [20:36] MatthewTMT So, anyway, those surveys, whoo... nightmares. [20:37] KevinPKV Data Mining proper has less of a hook IMHO... [20:37] OSE there's akind of people to every activity [20:37] MatthewTMT Absolutely, including observing. [20:38] |<-- AndrewP has left irc.aavso.org (Quit: http://www.mibbit.com ajax IRC Client) [20:38] OSE the photometry analysis aspect of the data-mining activities is fascinating in my opinion, it's all a matter of personality. I am more keen on stars than humans, history has never been appleaing to me (excepting that redhead teacher that was so hot in highschool!!) [20:39] MatthewTMT *laugh* [20:39] OSE Her hair had a B-V of 1.5 [20:40] MatthewTMT Totally going in the logs. [20:40] OSE ... (..) [20:40] ArneHQA much better than the earlier session [20:40] MatthewTMT So, Arne, what happened during the earlier chat? [20:41] OSE Has any of the people connected now bene involved in any kind of datamining (excepting HQ people)? [20:41] OSE yes, Arne, told us what it was about [20:41] NeilBIW Not here [20:41] OSE tell [20:41] KevinPKV Not me... [20:42] OSE So we need to find the hook for you! [20:43] ArneHQA I mentioned some of the current surveys, and mentioned that they had little impact on amateur activities except to give more objects to study [20:43] KevinPKV LOL... I have more archiving to do... My target is now 100,000... [20:43] -->| AndrewP (7cb68717@rox-7B41FB14.mibbit.com) has joined #AAVSO [20:44] OSE yep, we are finding more exotic stars through data-mining [20:44] OSE soon the exotic ones won't be exotic anymore! [20:44] ArneHQA I think the hard part are the current/future surveys, in that they will provide too many transients/new objects [20:45] OSE but there will be more weird things to fill that category [20:45] MatthewTMT For me that's the great thing about surveys -- you really start to get a big enough sample that you can understand the "exotic" stars in context of their underlying population. [20:45] ArneHQA if you can isolate the weird ones - the hard part [20:45] OSE yes, the CRTS/CSS is an example of what is coming... nowadays there is a couple of new CVs a day [20:46] ArneHQA Master is the same. But maybe CVs are not worth studying when you have thousands of them [20:46] MatthewTMT WZ Sge stars used to be these phenomenal things. I suspect LSST will find one per day. [20:46] OSE yes, some of them, being all-sky are bot biased [20:46] OSE are not biased so they help statistics [20:46] OSE yes [20:47] ArneHQA LSST's WZ Sge stars will not be helpful [20:48] ArneHQA not many people can do time series of a 20th magnitude object [20:48] OSE Data-mining also helps you clean up databases to improve statistics, there were a large number of "short period RRAB" stars (0.2-0.4 d) that were actually not RRAB stars, and that could be corrected through data-mining [20:48] MatthewTMT They won't be useful in the sense of doing time-series and measuring superhumps, but they will point to the underlying statistics of how many there are. [20:49] ArneHQA I agree, Matt [20:49] OSE so there will be a lot of unstudied WZ Sge stars [20:49] ArneHQA yup [20:49] MatthewTMT Until we get observers with bigger telescopes. [20:50] ArneHQA but there are thousands of eclipsing binaries, only studied statistically [20:50] |<-- AndrewP has left irc.aavso.org (Quit: http://www.mibbit.com ajax IRC Client) [20:50] KevinPKV Well, it comes back to coming up with to good hook to motivate new data miners... [20:50] ArneHQA so I see the majority of variables looked at in a statistical sense; fewer (but thousands) will be unusual enough to be studied individually [20:51] OSE yes, but to find the unusual you need to do the statistical part first, I think it has its merits [20:51] MatthewTMT Surveys are an efficient way of uncovering lots of weird ones quickly, *if* you can data-mine efficiently. [20:51] KevinPKV That pilot study or new data mining oriented article. [20:52] ArneHQA perhaps a good pilot study would be the upcoming BSM epoch photometry database [20:52] MatthewTMT [20:52] OSE We have ssome guidelines in the web, the VSX guielines and the web page on reporting new variables give some tips [20:53] OSE But with BSM you'll probably have know stuff to study, people usually want to discover new things [20:53] OSE (know = known) [20:54] OSE the new usually hides at the faint end (not necessarily, but usually) [20:55] MatthewTMT Delta Sco. [20:55] ArneHQA the majority of the BSM fields tend to be in the 6 < V < 12 range [20:55] OSE ... but it has already been discovered, now it is more difficult [20:55] ArneHQA so there are many fainter than Tycho/Hipparcos, and north so not yet in ASAS [20:56] ArneHQA but - in NSVS and possibly superwasp [20:56] ArneHQA so I agree [20:56] KevinPKV Is data mining the only activity for the new digital era? [20:56] OSE well, those are non-standard so there will be a benefit [20:57] ArneHQA by no means, Kevin [20:57] MatthewTMT Kevin: emphatic "no". I think there will be a role for observers always. They will not be the only game in town, but their contributions will matter. [20:57] KevinPKV Or perhaps get more visual observers in DSLR or CCD? [20:58] -->| AndrewP (7cb68717@rox-4FDDE039.mibbit.com) has joined #AAVSO [20:58] ArneHQA I think people tend to migrate to their observing method of choice [20:58] OSE Digitalizing paper data is an example, so you have one of the answers to your own question [20:59] KevinPKV Yep... Our mission is educate people on more options on modes of contributing... [20:59] KevinPKV Is to [21:00] ArneHQA true. but not to force anyone to use a particular method [21:00] MatthewTMT More modes, yes, but also to become more sophisticated and effective in whatever mode they choose. [21:01] KevinPKV Yep. Suggest options and provide examples and education. [21:01] ArneHQA we're about ready to make CCD observers work harder; they may return to visual observing. [21:01] |<-- AndrewP has left irc.aavso.org (Quit: http://www.mibbit.com ajax IRC Client) [21:01] OSE Yes, if we specialize in something, then follow the methods, respect the rules that will lead you to the best results. One problem I see is that people take the easiest way but want the highest recognition [21:02] OSE I see it a lot in my Latin pals sadly.. [21:02] MatthewTMT Sebastian, I think that's universally true of many human endeavors... *shrug* [21:03] OSE probably... [21:04] KevinPKV Perhaps effort should be rewarded more that numbers, if that is possible... [21:06] OSE When I find a data-miner that takes care of eevery little detail and is very responsible with their procedures, it's really comforting. When you see they love the stars and the knowledge [more than] the credits [21:09] ArneHQA Aaron had a neat "CCD Points" system that he shared with me; we may be reviving something like that