I'd like to start a discussion about the best practices in archiving spectra. There are a number of separate groups that have taken different approaches. I'm interested in why we're doing that and getting a discussion going about if there is any one standard we can settle on that works best for the widest number of applications available. To kick it off, I can think of three factors that there are a wide variety of practices on: wavelength calibration, flat-fielding, and 1D vs 2D spectra.
I think that it might work best to take each of these in one forum thread at a time. Let's start with wavelength calibration. If anyone is anxious to start a discussion about one of the other sub topics, be my guest, but I suggest that it might work best to do that on separate threads.
Lets acknowledge that spectra are in many ways more complicated than photometry. There is not really even a professionally agreed on standard. And this makes it harder for Pro-Am collaboration and crowd-sourcing spectroscopy.
As a professional who does spectroscopy, I know that the most useful way to archive spectra can depend on what you want to do with them. If you are doing identification of a transient, then you might not even need a wavelength calibration. You know what a Type II vs a Type Ia supernova spectrum looks like as long as you generally know this is optical or this is IR or UV. But if you want to do radial velocity work or catalog how an H-alpha emission profile evolves overtime, then you want a good wavelength calibration. A good relative (pixel to pixel) wavelength calibration is also needed to measure equivalent widths.
We should also consider that there are different ways to wavelength calibrate spectra. An end user that is doing radial velocity work will probably do an artificial wavelength standard like an arc lamp for good relative calibration but then on top of it observe radial velocity standards to improve their absolute calibration. Whereas someone doing abundance analysis is satisfied with a good relative wavelength scale from the arc lamps but not as concerned about a very well defined absolute scale.
We would all do wavelength calibration all the time to the highest degree possible but we don't because it takes extra steps and it can be hard with some setups. As a result we do it if it is easy or we really need it and otherwise we ignore it. Different archives have approached this different ways. Here are the approaches I can think of:
I'll say that option 4 is kind of what I do for myself for my own data. But as an archive end-user we all love #3 where the data falls in your lap with no additional work to do. And the observer in each of us has a guilty affinity for the idea of just dumping our data with little additional effort on our part.
In each of these we see a balancing (or conflict depending on how you look at it) of the expectations and requirements of end-users versus observers providing spectra. So many of you on this forum have experience running archives, or as an end-user, observer, or both. What do you think is the best balance and practice for a spectra archive when it comes to wavelength calibration? What have you personally found the best compromise for an archive to make? And I'd also be interested in hearing why.