ClearCase Support: Automate Checking of ClearCase Shipping Bays on VOB Servers

I also _strongly_ recommend automatically checking all of your ClearCase Shipping Bays on each of your ClearCase VOB Servers. I wrote the Perl script below to check each Shipping Bay automatically. This script does several basic checks, but here are the three primary features:

Checks for files that are stuck in any shipping bay for over a single day.

Checks for invalid synchronization files.

Checks if the Shipping Bay Partitions are almost running out of disk space.

You can change the limits of each requirement to your environment's needs. From my experience, I recommend running this script at least once a day via cron.

Here is how to execute this script.

$ check_multisite.pl

You can download this Perl script here, or you can copy-n-paste from here:

#!/usr/local/bin/perl
 
 
 # This script was written for www.philforhumanity.com.
 # If you have any questions, issues, or enhancement requests, please contact us via the website. 
 # Use this script at your own risk.
 
 
 # Define variables
 my $AWK = "/bin/awk";
 my $bay;
 my $DF = "/bin/df";
 my @files;
 my $file;
 my $file_age;
 my $file_stat;
 my $LS = "/bin/ls";
 my $MAX_AGE =  86400;              # 1 day = 86400 seconds
 my $output;
 my @SHIPPING_BAYS = ("/var/adm/atria/shipping/ms_ship/incoming", "/var/adm/atria/shipping/ms_ship/outgoing");
 my $SHIPPING_ERROR_LIMIT = 75;     # Shipping bay partition full error limit
 my $SHIPPING_WARNING_LIMIT = 50;   # Shipping bay partition full warning limit
 my $TAIL = "/bin/tail";
 
 
 # For each shipping bay, check each file
 foreach $bay (@SHIPPING_BAYS)
 {
 	# Check if shipping bay exists
 	if (! -d $bay)
 	{
 		print "ERROR: Invalid Shipping Bay Directory\n Location: $bay\n";
 		next;
 	}
 	
   # Change to the shipping bay directory
   chdir ($bay);
 
 
   # Get each filename in shipping bay
   @files = `$LS -1`;
 
 
   # For each file in shipping, check the file
   foreach $file (@files)
   {
     # Remove the EOLN character at the end of each filename
     chomp $file;
 
 
     # If the file is a directory, then skip to the next file
     next if -d $file;
 
 
     # Get the age of the file in seconds
     $file_stat = (stat(_))[9];
     # If the file's stat is -1, then there could be a corrupt file in the shipping bay, so print a warning
     if ($file_stat == -1)
     {
       print "WARNING: If this file still exists in the shipping bay, then it may be corrupt.\n Location: $bay/$file\n";
       next;
     }
     $file_age = time - (stat(_))[9];
 
 
     # Check if the file starts with the word "email"
     if ($file =~ m!^email!)
     {
       # Print warning message
       print "WARNING: There is an \"email\" oplog status file in a shipping bay.\n Location: $bay/$file\n";
       next;
     }
 
 
     # Check if the file age is over the limit
     if ($file_age > $MAX_AGE)
     {
       print "ERROR: A ClearCase MultiSite packet is stuck in a shipping bay!\n Location: $bay/$file\n";
     }
   }
 }
 
 
 # Check each shippng partition if the disk space used is too high
 foreach $bay (@SHIPPING_BAYS)
 {
   # Check if shipping bay exists
 	if (! -d $bay)
 	{
 		next;
 	}
 	
 	
   # Run disk free
   $output = eval `$DF -k $bay | $TAIL -1 | $AWK '{print \$1}' `;
   
   
   # Check percentage
   if ($output >= $SHIPPING_ERROR_LIMIT)
   {
     # Print results
     print "ERROR: Shipping Bay Partition $bay is over $SHIPPING_ERROR_LIMIT% full!\n";
   }
   elsif ($output >= $SHIPPING_WARNING_LIMIT)
   {
     # Print results 
     print  "WARNING: Shipping Bay Partition $bay is over $SHIPPING_WARNING_LIMIT% full.\n";
   }
 }
  
 
 # Exit
 exit 0;

This script was written for www.philforhumanity.com.

by Phil for Humanity
on 12/08/2008

ClearCase Support: Automate Checking of ClearCase Shipping Bays on VOB Servers

Related Articles » ClearCase Support: Automate Checking of ClearCase Server Log Files » ClearCase Support: Automate Checking of ClearCase VOBs » ClearCase Support: Automate Checking of ClearCase Views

Related Articles
» ClearCase Support: Automate Checking of ClearCase Server Log Files
» ClearCase Support: Automate Checking of ClearCase VOBs
» ClearCase Support: Automate Checking of ClearCase Views