Feeding Monsters

[Tom Droege <tdroege2@earthlink.net>]

I now have 4 computers processing pipelines.  They are voracious eaters of 
CD's.  It takes a couple of hours of feeding disks and starting pipelines 
to get them all going.  Then I just have to come back every hour or so to 
check for completed runs.

I have processed roughly 6 million star pairs.  I have about 5 million to 
go to catch up with the current running.  This will be all new data.  Then 
I will start to reprocess the old data.  This will take another couple of 
months using just one machine.  I want to keep three machines going 
processing the day to day data.

I looks like it takes 1.5 to 2 times as long to process the data as to take 
it.  This is with a 3 sigma limit for star detection instead of Michael's 
setting of 5 sigma.  This picks up a few more real stars at the faint end, 
I think.

There are a few problems that take intervention, but mostly the pipelines 
just run.

I now have several machines sharing one monitor.  There were a few times I 
did something like rm -fr when I was on a machine that I did not think I 
was on.  Now I am very careful to read the machine name.  I have also done 
a rm -f * 234.fits  (that is a mistaken space between the * and the 
234.fits).  They warned me about that in the class I took.  I think you 
have to do it once to really learn.

Tom Droege