Table of Contents ------------------ A. What do I want to do? Look for variable stars. B. How will I interact with the database? C. Looking for novae and supernovae D. Looking for asteroids/comets E. Possible data structures for the database F. Random comments Several people have made the very good point that, before one designs a database, one should think carefully about how it will be used. Joe Strout made a very nice set of outlines for a possible database, and I'd like to suggest small changes. But first, I'll give examples of the queries _I_ would like to make (I can't speak for anyone else!), and we can see if the database will support them. I will enter my own vote on the issue of "how much data do we save?" I plan to turn images into lists of detections, keep all the detections, and throw out the raw images. A. What do I want to do? Look for variable stars. --------------------------------------------------- My main interest (at the moment :-) is to observe the same set of stars near the equator, over and over, and a) monitor known variable stars b) discover new variable stars For the most part, I mean "stars which can always be detected by TASS, but which vary in brightness". I am therefore ignoring novae, supernovae, and variables which disappear at minimum light. This will be a good start, anyway. B. How will I interact with the database? ------------------------------------------- I imagine the process being a 2-step process. First, I have to run a camera for many nights to build up a database of detections; then, days -- weeks -- months later, I search for objects which vary in brightness from night to night. Step 1: observing, and STUFFING the database 1. a camera runs for a night 2. the images are reduced (flatfielding, etc.) 3. the corrected images are searched for objects; all detected objects have positions and brightness measured 4. positions are calibrated using GSC stars in the field (this can be done on a frame-by-frame basis with relatively small frames, only 800x768 TASS pixels) 5. instrumental magnitudes are converted to standard ones (this _probably_ requires examining data from the entire night, identifying the _few_ photometric standards observed, and making some sort of time-varying solution for the entire night at once. This might be done after stuffing into the database, but one night's worth can probably be done simply by sifting through ASCII files of detections on the "local" disk) 6. the final outputs, position and magnitude for all detected objects, are stuffed into a database. We must also record the observer, camera, date, time, etc., of course. Step 2: QUERYING the database I imagine that the _position_ of an object will be used to identify it as "the same object" from one night to the next. As others have pointed out, if we go down to 14'th magnitude, our big 13.8-arcsec pixels should separate stars nicely, except near the galactic equator, perhaps. Therefore, I imagine making the following queries: "Seaching over objects detected in the past month (or year), find all occurences of >= 10 detections of objects at the same position (where 'same position' means at within a circle of diameter about 15 arcseconds). Call this the 'Star Set'." This step serves to weed out spurious detections: noise peaks, satellites, asteroids, planes, etc. We should be left with a sample of "pure" stars. "Consider the 'Star Set' only. Calculate the mean value of each object's magnitude, and the standard deviation from the mean." We can then plot a histogram of standard deviation vs. magnitude. We should find a locus of points that curves upwards as we go to fainter magnitudes. This gives us a general idea of the accuracy of our measurements as a function of magnitude. For example, we might find that stars with 10 < mag < 11 have typical standard deviations of 0.1 magnitudes. "Consider the 'Star Set' only. Give me a list of all objects with mean magnitude 10 < mag < 11, and which have a standard deviation of > 0.3 mag. Call this the 'Variable Set'." Following the earlier example, this will isolate stars which appear to vary more than others of the same average brightness. "Consider the 'Variable Set'. For each object, give me a chronologically-ordered list of time and magnitude, for every detection of this star." And -- voila -- I can plot this list to make a light curve. C. Looking for novae and supernovae ------------------------------------- If the database has been built, as above, then one can search for novae and supernovae in the following fashion: Ask the database "find all objects which appear in the SAME place on >= 2 images during night July 4, but which do NOT appear in that place in April-June." (most asteroids will move in the hour between images, and will certainly move over a period of 5 hours or so, if we can get observations from several cameras at different sites on the same night) (most source of "junk", like noise peaks and satellites and so forth, will not appear in the same place; one can require an object to appear in >= 3 images on the same night to cut down the spurious detections even more) In practice, to find a nova in a timely fashion, this requires that users at different sites - reduce their data promptly - stuff their results into the master database promptly - are able to make this query and get a quick return One can, of course, make a more robust query by asking for all objects which appear at the same place for two weeks, and then don't appear any more .... but by then, the object has faded quite a bit. So this application really does require a database ... not merely a comparison of two or three fields taken on the same night, or on consecutive nights. One must have confidence that objects haven't been present for a long time. D. Looking for asteroids/comets ----------------------------------- One can look for asteroids in several ways -- some of them do not require a "real" database, but can be done with a pair or triplet of object lists from the same night. One method requires only the object lists from a single triplet. As others have pointed out, one can do this without astrometric or photometric calibration (although one will certainly want astrometric calibration for any asteroids which ARE detected). - match up objects detected in images A, B, C (one can use the matching code I provided to match A-B, then B-C, for example) - consider the set of objects which are _unmatched_ in all three images - after transforming all of these objects to a common coordinate system (say, that of frame B), loop over the objects and try to find a set of 3 detections, one each from A,B,C, which are colinear and within X arcseconds of each other (where X is some reasonable value -- around 10-20 arcsec) Another method, appropriate for finding Near Earth Objects only, involves measuring the SHAPES of all detected objects (instead of assuming that they are all round, like stationary stars). Simply look for objects which are elongated beyond some limit, and prepare to find a lot of planes and satellites and so forth :-) To find _slowly_ moving objects, one might use information after it has all been entered into the master database. One might search for objects which are detected over a week or more, but which have positions which change slightly. Actually, this sounds vague to me, and it's probably tougher than it sounds. A lot depends on exactly HOW objects are matched with master objects in the database, as they are added in. E. Possible data structures for the database ----------------------------------------------- I'll follow Joe Strout's propsals, making only minor modifications here. I end up with 4 tables: Observations: one entry per detection in one image CelestialObjects: one entry per _star_ or _galaxy_ in the sky ImageInfos: one entry per image (I imagine "images" as being roughly squares) Cameras: one entry per camera (i.e. per CCD chip) person, location, filter, etc. table: Observations desc: An Observation is a single point of light imaged at a particular time. index: Hash table by rounded coordinates (RA, Dec); within a coordinate cell, sorted by time with most recent observations first. fields: - calibrated coordinates (RA, Dec) - uncertainty in coordinates? I'm not sure ... - calibrated magnitude (V or I or ...) - ImageInfo (reference to an entry in ImageInfos table) - raw image coordinates (row, col) - raw instrumental magnitude - uncertainty in magnitude? I'm not sure ... table: CelestialObjects desc: Each entry contains information on a distinct astronomical object; almost always, a star. One might stuff the database by grabbing stars from an existing catalog (GSC), or one might build it by observing the same field many times and only selecting objects which always appear. index: Hash table by rounded coordinates; within a coordinate cell, sorted by time with most recent observations first. fields: - calibrated coordinates (RA, Dec) (is this a "mean" position? I don't know) - calibrated magnitude (V or I or ...) (is this a "mean" magnitude? I don't know) - pointers to all 'Observations' which appear at the same position - catalog label (if known): star, galaxy, comet, satellite, etc. table: ImageInfos desc: Each entry contains information about a particular image taken with a certain camera at a certain time. index: ImageID fields: - ImageID: unique identifier for each image - sky coverage in normalized coordinates - Camera (reference to an entry in Cameras table) - filter (note that a camera might change filters after a while) - exposure info (time of exposure, aquisition method?) - weather info? table: Cameras desc: Each entry describes a particular camera set-up, with which probably many images have been taken. (Note, new entry is made when camera is moved etc.) index: CameraID fields: - CameraID: unique camera set-up identifier - contact person - camera serial number - location (latitude, longitude, altitude) F. Random comments --------------------- I suspect that co-addition of TASS images will NOT be an easy thing. I suspect that the sky brightness and noise will tend to vary quite a bit from one hour to the next, and certainly from one night to the next; doing a properly-weighted combination won't be easy. At some point, if one co-adds a sufficient number of images, we'll reach so deep that the field starts to become crowded. I'm not sure that we can calculate an accurate value of the uncertainty in position and/or magnitude based on a single image. I suspect that the proper way to determine photometric accuracy is to compare detections of the same star on different nights. _Maybe_ the astrometry is better-behaved, but I'm not sure. Do we want to store _average_ values for magnitudes and positions in the "CelestialObjects" table, or just the value of the _first_ or some other detection? I don't know. I like very much the idea of using ASCII tables and a simple relational database. I hope that we can find a good, free or cheap package --- thanks to Paul Bartholdi for his notes on "/rdb". Michael Richmond