Re: Databases, was Data Processing Switch

[Chris Albertson <chrisalbertson90278@yahoo.com>]

--- Tom Droege <tdroege2@earthlink.net> wrote:
> The disks I am sending out are uncompressed.  Sigh!  Just one more
> thing to 
> learn.  Someone give me a command line that will compress a directory
> that 
> is good for tabluar data:


Tom,

On a linux system you can use this assuming "foobar" is
the name of a directory inside your current directory

    tar cvzf foobar.tgz foobar

There are many ways to compress files but if you do it this
way people who have MS Windows PCs will be able to uncompact this
using "zip" winzipp or the like as modern "zips" know about
gziped tar files.  You can lookup the options I used above,
"cvzf" in the tar man page.  I'd expect about 3:1 or possibly
better compression as tabular text is very compressable.

To above does almost the exact same thing as the MS DOS "zip"
command only the file is called ".tgz" not ".zip"



The above command creats a file called "foobar.tgz" to
unpack it use "tar xvzf foobar.tgz" and to list it's
contents without unpacking "tar tvzf foobar.tgz"

If you want to compress every file in a directory and now
pack that into a large container file then all you need to
do is cd to that directory and type "gzip *"  or if the
directory is hugh you might try "nice gzip * &" to compress
in the background at reduced CPU priority so as not to tie
up the computer for hours.




> 
>  > compress *.foo > glop
> 
> and I will try it.
> 
> Tom Droege
> 
> At 11:05 PM 6/2/03 -0700, Chris Albertson wrote:
> 
> >How big are the files?  After compression.  Tabular
> >data should compress well.  Maybe someone who has a
> >good internet connection can put them on-line.
> >
> >My vision for how a database should work is that the person
> >who has a camera runs a little program that opens a
> >connection (posably over the Internet) and populates a database
> >as the last stage in the pipeline.  A databse can stay up to
> >date in near real time.
> >
> >
> >--- Robert Creager <Robert_Creager@LogicalChaos.org> wrote:
> > >
> > > On Mon, 02 Jun 2003 23:24:48 -0500
> > > Tom Droege <tdroege2@earthlink.net> said something like:
> > >
> > > > I just made 2 copies of the files to send to Michael S. (for
> the
> > > data
> > > > base) and Michael R.  (To look at the data quality).  It is a
> five
> > > > disk set so I don't really want to make a lot of copies.  Still
> if
> > > any
> > > > of you data base fans want a set I will make one for you. 
> There
> > > are
> > > > only about 3 full disks  15 million measurement pairs or so. 
> It is
> > > > convenient to separate the older and newer data and by TOM.
> > > >
> > >
> > > I'd be happy to take a second hand set of disks, and can pass
> them
> > > along
> > > to the next person if there are more takers.  Saves your time,
> and
> > > I'm
> > > in no hurry.
> > >
> > > I'd have to adjust my import/dB some, as when I import the data
> for
> > > my
> > > system, I import based on FITS files and add some of the FITS
> headers
> > > to
> > > the dB.
> > >
> > > Later,
> > > Rob
> > >
> > > --
> > > O_
> > >
> >
> > > ATTACHMENT part 2 application/pgp-signature
> >
> >
> >
> >=====
> >Chris Albertson
> >   Home:   310-376-1029  chrisalbertson90278@yahoo.com
> >   Cell:   310-990-7550
> >   Office: 310-336-5189  Christopher.J.Albertson@aero.org
> >   KG6OMK
> >
> >__________________________________
> >Do you Yahoo!?
> >Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
> >http://calendar.yahoo.com
> 
> 


=====
Chris Albertson
  Home:   310-376-1029  chrisalbertson90278@yahoo.com
  Cell:   310-990-7550
  Office: 310-336-5189  Christopher.J.Albertson@aero.org
  KG6OMK

__________________________________
Do you Yahoo!?
Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
http://calendar.yahoo.com