Re: Calculation of mag errors

[droege@snapmail.us]

This is an old discussion here.  It turns out that because the darks and
flats are mostly noise, and pretty large noise at that, they do not
compress very well.  There is at least one tech note that collects a lot
of the discussion on this.

This is not really the problem.  I can keep darks and flats and plan to do
so for the coming runs.  The problem is the raw images.  If one keeps the
basic raw image, then one has to apply darks and flats evey time one looks
at one.  To do this efficiently, then we need a tool set that does this
when looking through the images that does this automatically.  Possibly
somone will volunteer to set such a thing up.

I have (lately) been keeping the dark and flat corrected images because
they are reasonable for one to look at to see if there is a satellite
trail or some such artifacti affecting one particular image.

Tom Droege


> On Mar 5 2005, droege@snapmail.us wrote:
>
>> Andrew Bennett writes:
>>
>> > Another reason for keeping raw data ...
>>
>> I hope to be more organized this year.  Rob's code allows me to do darks
>> every night.  I think I don't want to make flats every night.  I will
>> figure out a storage scheme to keep the current flat set with the data.
>>
>> Thus I might as well keep the raw data.  But note that it is not very
>> nice
>> to look at, so perhaps we should keep the processed data and the darks
>> and
>> flats and have a way to reverse back to the raw image.
>>
>> I really don't want to keep both.
>>
> First, I think it is necessary that I declare that astronomy is not my
> field of specialisation - it's just a sideline of mine so feel free to
> ignore me.
>
> But it would strike me that darks and flats have low information content
> and should compress losslessly very well. I would suspect that the darks
> and flats from different nights would be rather similar and if that
> feature
> were used somehow, it could lead to even better compression. So it may be
> possible that these fields could be retained very cheaply indeed.
>
> OTOH, the raw images are information-rich and contain also a lot of
> information useful for modelling the noise distribution. It would seem a
> pity to discard it or handle it in a manner that will not reconstruct it
> losslessly. Is the current background-subtraction and flattening pipeline
> algorithm fully reversible?
>
> Regards,
> David Huen
>
>