Title: Automate performance monitoring via telnet and ftp in a production environment
Submitter: Victor Yang (other recipes)Victor Yang (other recipes)
Last Updated: 2006/10/24
Version no: 1.1
Category: Network

Description:

This code snippet shows how to kich off a performance data gathering shell script with telnetlib and download the data back to a local workstation with ftplib.

Source: Text Source

#automate a log grabber script with telnetlib and ftplib
# LOG_GRABBER is a shell script which will grab the logs from the production logs. 
# It may invoke other shell script, perl, shell, python etc to get its job done.
# A generic example: getAccountBalance, 500 
# this means getAccountBalance function takes about 500 ms to finish. 
# its end result is saved in LOG_OUT in any format you may import later for analysis.  

import telnetlib
from ftplib import FTP

# full path to them
LOG_GRABBER='/users/perfmon/grabLogs.sh'
LOG_OUT='logstats.txt'

prdLogBox='142.178.1.3'
uid = 'uid'
pwd = 'yourpassword'

# kick off the log grabber via telnet

tn = telnetlib.Telnet(prdLogBox)

tn.read_until("login: ")
tn.write(uid + "\n")

tn.read_until("Password:")
tn.write(pwd + "\n")

tn.write(LOG_GRABBER+"\n")

tn.write("exit\n")

tn.close()


# download the timing statistics to local via FTP 

ftp=FTP(prdLogBox)
ftp.login(uid,pwd)
#ftp.set_debuglevel(2)
logOut=open(LOG_OUT,'wb+')
ftp.retrbinary('RETR '+LOG_OUT, logOut.write)
ftp.quit()
logOut.close()

Discussion:

This recipe is used for performance monitoring in real procution environment. It is non-intrusive as the timing log (LOG4J actually) has been produced daily and copy to a differnt server during the non-rush hour.


Gather timing statistics in log daily from a production environment during the non rush hour, then download the results, put the results into a RDBMS, calculate the statistics such as min,max,avg,median, deviation.