Skip to main content

Equation of model

avon
avon's picture
Offline
Joined: 2012-12-11

Hi

I have been trying to use the equation given from Vstar to overplot the data with the model light-curve using a programming language (for research purposes).

However, I find that using the given equation, with the dates of the data, often does not match the fit in the Vstar plot itself, there is a offset in the phase, such that the model curve (despite being correct in period and amplitude) does not fit the data. I have experimented with this but can see no obvious reason.

Any suggestions?

Many regards

Avon

An example
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon

It may be worth looking at a particular example. To reproduce the problem, can I ask you to post a sample dataset, DCDFT parameters, periods and number-of-harmonics selected to created the model?

Also, the program (in python, ...?) you're using to work with the VStar model equation would be good to look at.

Regards,

David

equation of model
avon
avon's picture
Offline
Joined: 2012-12-11

 Hi David,

Many thanks for such a quick post.I have attached a sample data file and a PDF of the plot my IDL code gives. I know Python might be better, but after so many years.....:-)

The key bit of the IDL code is at the end of the post. The parameters of the fit are in the frequency and the three coefficients.

I did some more experimenting yesterday, and it does seem (by eye) that when the fits did not overplot on the data I can bring them in line if I add/subtract a factor to the MJD that is a fraction of the period, e.g. (MJD - period/2) or (MJD - period/4). I suspect that might hint at the issue.

Many regards

 

Avon

----------------------------------------------

   

file_css = '........./light_curves/'+ file_string+'.dat'

readcol,file_css,id,mag_css,mag_err_css,ra,dec,mjd_css,blend, format='A,F,F,F,F,F,I',comment='#',/silent

 

plot,mjd_css,mag_css,psym= blob,yrange=[max(mag_css)+0.3,min(mag_css)-0.3], xrange=[53300,56300], xstyle=1, ystyle=1, xtickformat='(i5)', xtitle='MJD'

oploterror,mjd_css,mag_css, mag_err_css,psym=3,color=black

 

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

; overplot Vstar model

frequency = 0.005082

period=1/frequency

print,'period = ',period

A = 13.096262

B = -0.382092

C = 0.112805

mjd_model = findgen(max(mjd_css) - min(mjd_css))

mag_model = A + B * cos(2 * !PI * frequency * mjd_model) +  C* sin(2 * !PI *  frequency * mjd_model)

 

oplot, mjd_model+min(mjd_css), mag_model,color=black

   

 

Hi Avon Thanks. I'll have a
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon

Thanks. I'll have a look at this data.

To clarify, does the IDL code use the frequency from DCDFT and the Fourier Series coefficients A, B, C from a model (# of harmonics=1) created from that frequency in VStar?

Also, as an aside, is the data file from a particular source that might make a good candidate for an observation source plugin, or a personal format?

Regards,

David

Hi David Yes, the frequency
avon
avon's picture
Offline
Joined: 2012-12-11

Hi David

Yes, the frequency and coefficients are for a simple harmonics=1 solution provided by Vstar.

The data format is actually the native data table from the Catalina Sky Survey, which is the database that I am using.

It is an wide field survey, so many of the objects are non-variable and/or too faint. But there are some excellent variable objects in it too.

Many regards

 

Avon

Hi Avon Apologies for the
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon

Apologies for the silence the last couple of days. I've been focussing on preparing a new VStar release before the Silly Season gets fully underway.

I just wanted to mention that:

  1. As part of looking at this problem I ended up writing a Catalina Sky Survey observation source plugin for VStar. That will be available on the plugin library page when the new release is out (look for release announcement on this forum).
  2. I haven't tracked down the cause of the offset issue you reported yet. It appears that you are able to compensate for this but we need to understand where the problem is introduced.

I will continue to look at this when I can. The next couple of weeks will be problematic in that regard however.

I have captured this problem in a SourceForge tracker item:

https://sourceforge.net/tracker/?func=detail&aid=3596907&group_id=263306&atid=1152052

Regards,

David

Hi David   Many thanks
avon
avon's picture
Offline
Joined: 2012-12-11

Hi David

 

Many thanks for writing the plug-in. That will be most useful, and I look forward to the next release.

And I do understand that the next few weeks will be difficult:-)

Have a good Xmas and New Year

Regards

Avon

URL for sample CSS data?
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon

Can you point me to the URL from which the data file CSS_J105923.9+394405.txt was obtained?

It's been pointed out to me that my CSS plugin needs to cater for CSV and does not currently; more details later. That plugin has had fairly minimal testing (on your sample data) so far. Any other pointers you can provide re: CSS data file format would also be appreciated.

I hope you (and all) have had a good break so far, assuming you had one of course. :)

Thanks!

Regards,

David 

Hi David Excuse slow
avon
avon's picture
Offline
Joined: 2012-12-11

Hi David

Excuse slow reply...I have indeed had a good break, only back to work yesterday :-)

There are few links to the data format. Maybe the best thing is if you went to

        http://nesssi.cacr.caltech.edu/cgi-bin/getcssconedb_release.cgi

and put in a random ra/dec to see the data for the stars near that location.

Many regards

Avon

CSS format and output options
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon

Thanks, and no problem at all. It's that time of year. :)

I should first say thanks to John Greaves for sending me an email just before Christmas pointing out the bug in the Catalina Sky Survey (CSS) plugin. As mentioned in an earlier post, I had an unusually short amount of time to test the CSS plugin, only having used the sample data file you supplied above. I normally do a lot more testing. So, that's my fault for not being thorough.

I've replaced the space-delimiter with comma-delimiter (CSV) handling in the plugin. I plan to make a new VStar release available before the end of the month, and will ask Sara to update the plugin library page at that time. 

Further to this though, the CSS format/output options seem to be numerous. This CSS usage page is fairly brief and non-specific (see Format section). The page you refer to above has an "Advanced Parameters" link which appears to be the primary reference. This has radio buttons to allow selection between output types:

  • HTML
  • ASCII
  • VOTable

and format types:

  • short
  • long

The default is ASCII + short. This results in CSV being output to the browser (View button at end of page) which can then be copied and pasted into a file or saved via the download link and right-mouse-click (or ctrl-mouse-click on Mac etc).

The changes I have made to the plugin now support this "ASCII + short" default, but nothing else.

I would appreciate guidance re: whether support for this short format is sufficient or whether there is merit in supporting other combinations. The short format column headers are:

MasterID,Mag,Magerr,RA,Dec,MJD,Blend

The long format column headers are:

MasterID,ID,Mag,Magerr,RA,Dec,FWHM,Var,FrameID,MJD,airmass,exposure,X,Y,Flux,Area,Flags,Theta,

Elong,MuMax,Blend

As an aside, John G suggested (in email) that providing VStar with a URL rather than requiring a file to be specified would be better, but this is a CGI based download page with no URL parameters, so that's not feasible.

Thanks.

Regards,

David

David I think the short
avon
avon's picture
Offline
Joined: 2012-12-11

David


I think the short form would be more than sufficient. I have used catalina data for some months now and never felt the need to use the full long form.
 (and yes, I too had noticed that the CSS documentation is a bit on the 'light' side)

 

BTW, excuse me for asking, but has there been any luck with tracking the issue of the model equation fit:-)

Cheers


Avon

Short form, fit bug
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon

I was thinking the same thing. From a data analysis viewpoint, it seems sufficient. Thanks for your help with this, and to John for pointing out the error. There will be a new release of VStar next week and a plugin update, including CSS.

Re: the model fit bug, no progress as yet, no. Sorry. All I've done so far is to capture it here:

  https://sourceforge.net/tracker/?func=detail&aid=3596907&group_id=263306&atid=1152052

and think about it a bit. If you'd like me to increase the priority, let me know. I don't want it to be a source of problems for you.

Regards,

David

Hi David I must admit I
avon
avon's picture
Offline
Joined: 2012-12-11

Hi David


I must admit I would like to know what the problem is that causes the phase  offset. Even though I am sure it is easy to fix for plotting, it is not a 'clean' solution as required by journal peer review.


But I also would not want to make things more difficult at your end.


Regards

 

Avon

Of course
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon

Of course, I completely understand.

I have a new VStar release to get out this week and will make it my highest priority bug after that.

Regards,

David

Reproduced your result
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon, all

Okay, so first: apologies for the long time since my last post to this thread. I was busy being a CHOICE course participant for the last few few weeks.

Second, I've reproduced your result by writing some R code, and I agree that the problem exists. The model equation generates a continuous curve that is offset (to the right) by a certain amount from both the observation data and the model data that the model equation purports to represent. So, this implies that the model function being generated by VStar is in error.

I've attached a bunch of files. They relate to your sample CSS data. Here's the raw and phase plots and model data points for a single period (196.783 days).

I also created a model (in VStar from DCDFT dialog) based upon that period and 4 harmonics (see 5f files below). Here's the phase plot with observations and model data points:

The attached files are:

  • data.csv: the observations saved from VStar as CSV.
  • model_f.csv: the single period model saved from VStar as CSV.
  • model_5f.csv: the 5 period model saved from VStar as CSV.
  • model_f.png: plot of observations (from data.csv), single period model data points, and single period model function.
  • model_5f.png: plot of observations (from data.csv), 5 period model data points, and 5 period model function.
  • model_plotter.R.txt (only has .txt suffix so I could attach it): R code that generates the last two plots.

Here are the two plots generated by the R code:

In both cases the offset of the function generated line plot from the observation and model data points is obvious.

Next step is to figure out what is apparently going wrong with model function generation.

David

I ran TS on the same data
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

I ran TS on the same data (converted to tab-separated form: data.tsv.txt, attached), generating a model for the 196.7829 day period.

The result is consistent with that from VStar. It would have been surprising if that weren't the case given that VStar's implementation is based upon TS.

The file ts_session.txt (attached) shows the TS run.

 

The following is from the end of the session output (data.ts.txt (attached)):

 

     Frequency     Period      Power  Amplitude      cos        sin       const

   0.005081741    196.7829     Fre01     0.3984    -0.3821     0.1128    13.0963

 

The file model_f_residuals.dat_.txt contains the residuals from the single-period model.

 

David

I should qualify the last
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

I should qualify the last post by saying that while VStar and TS agree upon Fourier coefficients for the model, TS is not taking the next step of generating an equation, so the problem may lie in how VStar does that.

I've asked Matthew Templeton about this problem and he has made some suggestions (thanks Matt) for things to look at and I'll post again when I have an update from that.

David

Right shift by a fraction of the period...
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon, all

Almost another month has elapsed since my last post on this topic!

Here's an update.

With reference to my previous post, the model (line fit) was plotted via the R code:

lines(jds, model(jds), col="blue")

giving:

In email correspondence, Matthew pointed out the need to use the same time-center of zero (or zero point) when plotting the model (blue line) as was used when the model was built. I looked at this and confirm that VStar (and TS) create a zero point and subtract it from JDs values in the range when building the Fourier series model. I'm looking further at the computation of the zero point.

Taking this into account, Avon's earlier comment about subtracting period/4 from the (zeroed) JDs gives the expected fit:

jds = jds-jd0

lines(jds+jd0-period/4, model(jds), col="blue")

where jd0 is a zero point constant (based upon, but not equal to, the first JD in the model range) subtracted from all JDs in the range before the model function is invoked on each JD (added back for the plot), and period is the fundamental period of the model, giving:

However, the same result can be obtained by subtracting period/8 without first subtracting a zero point from all JDs in the range:

lines(jds-period/8, model(jds), col="blue")

I'm continuing to look at the model creation code as I find the time.

Additional datasets and models of these datasets also need to be looked at to determine whether the findings based upon this sample dataset and model apply more generally.

David

VStar model equation bug fixed
David Benn
David Benn's picture
Offline
Joined: 2010-07-29

Hi Avon, all

This bug is now fixed!

Thanks to Matt and Avon for their help in resolving this! Their comments guided me toward a solution.

The zero-point extracted during model creation was not being added back into the equation shown in the Analysis -> Models... dialog. Finding the right bits of the model creation code took awhile. 

One of the things that will be included in SourceForge and probably elsewhere (e.g. docs) now is an R script that will plot observation and model data (saved from VStar as data.csv and model.csv) and the model function as a line plot. Here it is:

 https://sourceforge.net/p/vstar/code/HEAD/tree/trunk/script/plot_model.R

I've attached files showing another example, using LX Cyg. One plot is from VStar, the other from the R script.

This fix, along with others, will be in a new release sometime in the next week or two.

David

AAVSO 49 Bay State Rd. Cambridge, MA 02138 aavso@aavso.org 617-354-0484