Skip to main content

AVSpec fits header

14 posts / 0 new
Last post
ErnstPollmann
AVSpec fits header

Dear colleagues,
my VV Cep spectra, which have been processed with VSpec, have some typical settings in the fits-header, which are not in agreement with the fit-convention at AVSpec. If I want to upload several hundred of these spectra, a manual correction of each VSpec fits-header would be necessary.
Is here someone, who have experience with this fit-header issues?

Ernst Pollmann

Robin Leadbeater
BeSS compatibility

Hi Ernst,

I am familar with the BeSS standard and its implementation in BeSS and the BAA database. This is worrying as AAVSO claim their fits format to be compatible with BeSS (though it is not a full implementation) and VSpec produces spectra to BeSS specification. Are your spectra accepted by BeSS and the BAA database?  If so there would seem to be an issue with the AAVSO database which should be resolved rather than modifying your headers. Can you describe exactly what the problem is ?

If all else fails you can add them to the BAA database for similar long term security

Cheers

Robin

bpablo
Hello Robin,

Hello Robin,

Thanks for your response and your concern. I have been helping Ernst with this so let me provide a little more information. The fits header itself is fine in the sense that all the keywords are there and are compatible. However, the values of the keywords themselves, at least in Ernst case, have a couple issues which our cleaning process has an issue with.

1.) Most of the fits keywords have a bunch of extra spaces in them which confuses the system. When it tries to match for instance '                       VV Cep' with 'VV Cep' it fails to find the star.

2. The CRPIX1 value is '1.' instead of '1'. This may not seem significant, but since the reference pixel must by definition be an integer the system is insisting that there is a problem as that dot makes it a float.  

I am unsure why these things are happening as I have not used the VSpec software.  While it is easy enough to correct the headers, it doesn't answer the main problem which is why VSpec is adding it's header keyword values with all of these unnecessary characters.  I was hoping someone more familiar with VSpec could help Ernst (and by extension us at the AAVSO) figure out why this is happening.

Thanks,

Bert Pablo
Staff Astronomer, AAVSO

Robin Leadbeater
leading spaces in Visual Spec header

Hi Bert,

Yes It looks like Visual Spec pads out the header entries with leading spaces so that the length of the string matches the maximum allowable in the BeSS specfication. Not sure why but I suggest asking the Author Valerie Desnoux who is also a long standing contributor to and validator for spectra submitted to BeSS so will be able to answer your questions.  You can find her contact details on her Visual Spec website here

http://astrosurf.com/vdesnoux/download.html

However I think perhaps the AAVSO database should be able to manage this situation. For example I was able to search VSX, BeSS and SIMBAD successfully even with leading spaces added to the name 

Cheers

Robin

Robin Leadbeater
A work round

In the mean time if the spectra are opened in ISIS and then resaved, ISIS will remove the leading spaces

Robin

Robin Leadbeater
BAA specdb ignores leading spaces

I have just double checked and can confirm that the BAA database ignores leading spaces and identifies the object name correctly so spectra conforming to BeSS standard and produced using Visual Spec would be accepted there

Cheers

Robin

Robin Leadbeater
CRPIX1 is floating point in BeSS standard, not integer

"1."  is a valid  value for CRPIX1 in the BeSS standard. Extract from the standard:-

Reference pixel for the reference wavelength ● Keyword = CRPIX1 (mandatory). This keyword defines the pixel where reference wavelength (CRVAL1) is given. It is usually the first pixel of the image – in this case, value is 1. ● Format: floating point ● Units: none ● Valid Range: no limit ● Example: 1.0 

bpablo
Hello Robin,

Hello Robin,

 

Thanks for your input. I will look into ignoring leading spaces I wish I understood why that was done, though, because it seems unecessary. I can also change CRPIX1, however, in the definition you mention up above it states that it is the reference pixel. Pixel is a finite unit. You can't have a reference pixel of 1.5 or 2.7 can you? Your reference pixel should be an integer not a float simply by definition of what it is. Am I missing something? Why would this be defined as a floating point number?

Thanks,

Bert Pablo
Staff Astronomer, AAVSO

 

Tonisee
FITS standards

Hello Bert,
this spring I had to dig deep into the topic of FITS standards. It is quite well defined, and there exists a rather exhaustive document about that, the latest one is: https://fits.gsfc.nasa.gov/standard40/fits_standard40aa-le.pdf  In general https://fits.gsfc.nasa.gov/fits_home.html contains a lot of useful information about FITS-related things, including list of software and reference to a validator.

About coordinate systems in FITS, pages 29-32 are probably most useful. Standard defines that reference pixel values and other related keywords shall be given as floating point numbers (see p 29 below the figure, p 31 in general), with defaults to 1.0 or 0.0. There is also good short indication about spectral coordinate systems in paragraph 8.4 (p 34).

I'm a bit curious, how are photometric observations uploaded to VPHOT and spectra submitted into AAVSO spectra database validated - what tools are used?

About CRPIX# values - it is definitely possible to measure something with higher accuracy than pixel size and it is rather common to e.g. linearize a spectrum with smaller (or larger if needed) steps than one physical pixel. E.g. if I reduce long slit spectra from our telescope, average uncertainty of wavelength scale is in the order of 1/15 pixels. Now it depends on the software or just practicality if I rebin original data to linear scale (usually defined by pixels) or work in wavelength scale coordinate system with some linear or non-linear step.
CRPIX is representing a coordinate system reference pixel, that is not necessarily a physical one.

As it is shown in FITS home, in FITS keyword dictionary:

KEYWORD: CRPIXn
REFERENCE: FITS Standard
STATUS: reserved
HDU: image
VALUE: real
COMMENT: coordinate system reference pixel
DEFINITION: The value field shall contain a floating point number,
identifying the location of a reference point along axis n, in units of
the axis index. This value is based upon a counter that runs from 1 to
NAXISn with an increment of 1 per pixel. The reference point value
need not be that for the center of a pixel nor lie within the actual
data array. Use comments to indicate the location of the index point
relative to the pixel.

There is an interesting question that IMHO has been solved differently by e.g. NOAO and ESO - what are the coordinates of pixel corners ;-)

However, I think that when creating a new infosystem, it is safer to stick to FITS standards as much as possible, especially in the case of mandatory FITS keywords.

Best wishes,
Tõnis Eenmäe

Robin Leadbeater
Hi  Tõnis,

Hi  Tõnis,

Yes indeed. The BeSS standard complies with the universal definition of CRPIX where it can be non integer. There are fits formats for spectra where the coeficients describing the full non linear mapping between the original pixels and wavelength (not just CRPIX1) are included in the fits header. In the BeSS standard the spectrum is linearised using equally spaced bins and the mapping between pixels in the image and the final spectrum is lost. The final spectrum bins are therefore not the same as the original pixels so the "pixels" in the BeSS standard here become the central wavelengths for each wavelength bin.  CRPIX1 is therefore always integer in this case.

Cheers

Robin 

bpablo
Hey Tõnis,

Hey Tõnis,

Thanks for your comments. They are very useful. It is an interesting idea that the reference pixel could be at a corner or center of a pixel. I had not really considered this. As to your question:

"I'm a bit curious, how are photometric observations uploaded to VPHOT and spectra submitted into AAVSO spectra database validated - what tools are used?"

I'm not really sure what answer you are looking for. VPHOT and AVSpec are very different in how they handle things. I'm not an expert on VPHOT so I cannot tell you that off the top of my head. AVSpec however, I can give you a general overview. It is a django tool which is python based. It uses astropy for a lot of the opening and checking of fits files. There is a cleaning routine that checks that all the necessary keywords are included, that their values match the correct type, and a few other things along the way. There is also post upload validation, where each spectra is checked individually to make sure that the spectra themselves have no obvious issues (e.g. wavelength calibration, right star etc.). Does this answer your question?

Thanks,

Bert Pablo
Staff Astronomer, AAVSO

Tonisee
Thank you, Bert for very good

Thank you, Bert for very good answer!

Tõnis

Robin Leadbeater
standards

I can't help you with that as I didn't write Visual Spec or define the BeSS standard (That was done by professional astronomers at Observatoire Meudon France).  The contacts are there on the BeSS database website though and most in the amateur community doing research grade spectroscopy  (Including Ernst),  will know Valerie Desnoux.  AAVSO is not working in a vaccuum here. There has been a well established community doing this sort of work for well over a decade.  

Cheers

Robin

bpablo
Thanks for your response

Thanks for your response Robin. I do not wish to change the community standards. I was just trying to understand why things are the way they are, because I had not run across any issues like this with other spectroscopy software. Thank you very much for all your helpful comments. I will do my best to address these issues as quickly as possible.

Thanks,

Bert Pablo
Staff Astronomer, AAVSO

Log in to post comments
AAVSO 49 Bay State Rd. Cambridge, MA 02138 aavso@aavso.org 617-354-0484