[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Databases, was Data Processing Switch
Rob,
Yes please post it. I was going to do something a lot
less complex but I think you're "doing it right" and putting
all the smarts into the DBMS. I'll take a look and see if
I can't contribute something.
Back when I did the Mk III database work performance was not
good. Adding referential integrity links just killed it. I
had to resort to doing much of the work in C and other tricks
just to get a data import to run in under 24 hours. Today
we have faster CPUs but more importantly cheaper RAM.
So, what kind of performance do you see, How long to import
1E7 observations?
I exchanged some e-mail with Michael Sallman and it looks like
he could without to much effort point his very nice looking
Java based web page at a replacement backend DBMS. What you
are doing could make a nice replacement. All that would be
required is to define a "view" that emulates his table.
Do you have this on a CVS server yet.
--- "Creager, Robert S" <CreagRS@LOUISVILLE.STORTEK.COM> wrote:
>
> Hey Chris,
>
> Well, I based it on your previous work, but have a lot of referential
> integrity links and other stuff added on top. So the table schema
> will look
> similar to what you've done, but then I use lots of Perl (DBI) and
> SQL to
> manage the rest.
>
> I import nearly the entire Tycho2 db, and then trim it down into
> another
> table using Arne's color transformations and Michael's selection
> criteria.
> This is how I drive matching reduced star lists to reality (using
> MATCH ala
> Michael R), and also how I generate color information.
>
> I'm storing the color terms for a nights observations, and zero
> points on a
> frame basis within the dB, but have not yet setup the trigger which
> will
> propagate that change through the dB. So I just run an UPDATE when I
> change
> any of those values for now.
>
> I've set up the tables so that, for instance, if I decide I want to
> remove
> an image from the data (clouds or something), I can delete an entry
> from a
> table, and that will cascade through the dB and end up adjusting the
> catalog
> table to remove the influence the stars in that file had on the
> catalog
> information (composite ra, dec, magnitudes for each band).
> Similarly, when
> I update an observation with a star id and magnitude (calibrated, not
> instrumental), the dB will handle adding that information to an
> existing
> catalog star (or creating a new one if appropriate). I'm using a
> PostgreSQL
> trigger for this.
>
> I'd be happy to post the schema, and supporting scripts if there is
> any
> interest. It's not done yet, but getting there.
>
> Cheers,
> Rob
>
> > -----Original Message-----
> > From: Chris Albertson [mailto:chrisalbertson90278@yahoo.com]
> > Sent: June 02, 2003 4:40 PM
> >
> > What are you doing re. a database? I was about to start a db
> > project myself.
> >
>
=====
Chris Albertson
Home: 310-376-1029 chrisalbertson90278@yahoo.com
Cell: 310-990-7550
Office: 310-336-5189 Christopher.J.Albertson@aero.org
KG6OMK
__________________________________
Do you Yahoo!?
Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
http://calendar.yahoo.com