Wednesday, January 9, 2013

Listener logs - Rotation and Truncation without bouncing

Note:  Proofread any scripts before using. Always try scripts on a test instance first. This Blog is not responsible for any damage.
Listener Log file audits trail information which enables us to analyze network statistics, client connection requests, service registration events or other stuff. So it is obvious that listener log can contain very useful information for troubleshooting network related problems.

This file is usually small and easy to diagnose but in cases of very active databases it could grew very fast to several gigabytes. We have listener.log file which is growing very fast because of very active OLTP database. After few months that file grew up to 4 GB and there stopped updating new logs.

There’s a basic problem with the listener log in Unix: while you can move/rename it, it won’t start a new log file until the listener is bounced. This is because the listener data is written to the inode, and doesn’t take your change into effect until the listener is bounced.

There are multiple methods we can use to rotate and truncate the logs without bouncing the listener.

First method:

# Check the current size of listener log
$ ls -l
-rw-r----- 1 oracle dba 5341923 Jan  9 04:12 listener.log

# Make backup of the file
$ zip listener.log.zip listener.log
adding: listener.log (deflated 93%)

# Truncate file 
$ cat /dev/null > listener.log

# Check the size
$ ls -l
-rw-r----- 1 oracle dba           93 Jan  9 04:12 listener.log
-rw-r--r-- 1 oracle dba   612614 Jan  9 04:09 listener.log.zip


You could also use [ $echo "" > listener.log ] to truncate file.

Second method - using LSNRCTL utility:

### MY CURRENT LISTENER LOG FILE

$ ls -l
-rw-r----- 1 oracle dba 5341923 Jan  9 04:12 listener.log


### STARTING LSNRCTL UTILITY
$lsnrctl
LSNRCTL for Solaris: Version 10.2.0.5.0 - Production on 09-JAN-2013 04:09:01
Copyright (c) 1991, 2010, Oracle.  All rights reserved.
Welcome to LSNRCTL, type "help" for information.
LSNRCTL>

### SETTING MY CURRENT LISTENER
LSNRCTL> set current_listener LISTENER
Current Listener is LISTENER

### CHANGING CURRENT LISTENER LOG LOCATION TO TEMPORARY
LSNRCTL> set log_file listener_temp
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oracle10g)(PORT=1521)))
LISTENER parameter "log_file" set to listener_temp.log
The command completed successfully

### EXIT
LSNRCTL> exit

### WE CREATED NEW LISTENER LOG FILE listener_temp.log (TEMPORARY)
$ ls -l
-rw-r----- 1 oracle dba 5341923 Jan  9 04:12 listener.log
-rw-r----- 1 oracle dba       1136 Jan  9 04:13 listener_temp.log

### BACKUP OLD LISTENER LOGS
$ mv listener.log listener.log.bak

### START LSNRCTL UTILITY
$ lsnrctl
LSNRCTL for Solaris: Version 10.2.0.5.0 - Production on 09-JAN-2013 04:09:01
Copyright (c) 1991, 2010, Oracle.  All rights reserved.
Welcome to LSNRCTL, type "help" for information.
LSNRCTL>

### RENAME LISTENER LOG BACK TO listener.log
LSNRCTL> set log_file listener
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oracle10g)(PORT=1521)))
LISTENER parameter "log_file" set to listener.log
The command completed successfully
LSNRCTL> exit

### WE NOW HAVE NEW TRUNCATED listener.log AND OLD LOGS BACKUPED
$ ls -l
-rw-r----- 1 oracle dba       1959 Jan  9 04:15 listener.log
-rw-r----- 1 oracle dba 5341923 Jan  9 04:12 listener.log.bak
-rw-r----- 1 oracle dba       1136 Jan  9 04:13 listener_temp.log

Oracle 11g introduced new diagnostically system - "Automatic Diagnostic Repository (ADR)" and now we have listener log written to xml file (besides listener.log file).

Trying to change log file using LSNRCTL utility will fail with:
TNS-01251: Cannot set trace/log directory under ADR.

LSRNCTL>set current_listener
LSNRCTL>set log_directory fails:
TNS-01251: Cannot set trace/log directory under ADR

For solution find more information on Metalink - note 762787.1.

Third method - using LOGROTATE command:

We can use logrotate command in cron to rotate listener logs in some time intervals. Logrotate is designed to help administrators with managing large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files.

Simple logrotate configuration:

Add file:
/etc/logrotate.d/oracle

-- write this lines into that file:
/oracle/network/log/listener.log {
missingok
weekly
rotate 4
compress
create 644 oracle dba
}

Logs will rotate every weak and old ones will be compressed. We will have 4 latest backups.

Fourth method - using LSNRCTL utility #2

Oracle TNS listener holds an open handle to listener.log file so I can't just rename file because Oracle won't create new log file for logging purposes. To recreate log file I must restart TNS listener process. Better option you can use lsnrctl to set log_status off, which allows you to manipulate the underlying log file. You can then set it back to on, and it will continue happily logging connections

rotate_listener_log.sh:

There’s a few assumptions which you may need to tweak for your environment:
First, the location of ORACLE_BASE, ORACLE_HOME and listener logs (/oracle/network/log) which varies as per your environment from mentioned location.
Secondly, that you have the zip utility somewhere in your base path.

In case you were wondering, why the %s in the datestamp?  It basically makes the filename it creates reasonably unique, so that in case you accidentally (or intentionally, while testing) run this script multiple times in one day, you don’t lose your logs.

Parameters: The first parameter is listener name, represented as ${1} which is used to turn off/on the logging of the listener by the same name. The second parameter is the full path to the current listener log.

# rotate_listener_log.sh

#!/bin/bash

ORACLE_BASE=/oracle; export ORACLE_BASE
ORACLE_HOME=${ORACLE_BASE}/product/10.2.0.5; export ORACLE_HOME

LSNRNAME=${1}
LOGFILE=${2}
DATE_STAMP=`date +%m%d%y%s`

lsnrctl <<END
set current_listener ${1}
set log_status off
exit
END

mv $LOGFILE $LOGFILE.${DATE_STAMP}

lsnrctl <<END
set current_listener ${1}
set log_status on
exit
END

zip $LOGFILE.${DATE_STAMP}.zip $LOGFILE.${DATE_STAMP}

rm $LOGFILE.${DATE_STAMP}

Place it in cron, and you’re done!

For those not too good with cron:

# Minute   Hour   Day of Month       Month          Day of Week        Command    
# (0-59)  (0-23)     (1-31)    (1-12 or Jan-Dec)  (0-6 or Sun-Sat)                
    0        2          12             *               0,6           /usr/bin/find
• The first entry says at 00:15 on the first day of the month, remove anything from the trace directory over a year old.
• The second entry runs on every Sunday at 00:30 to rotate the listener log of LISTENER and effects the shown log.

crontab -l

# cleanup listener logs
15 0 1 * * /usr/bin/find /oracle/network/log -mtime +365 -exec rm -rf {} \;

# rotate listener logs
30 0 * * 0 /oracle/dba/scripts/rotate_listener_log.sh LISTENER /oracle/network/log/listener.log

Test this script in a non-production environment before you use it!

OR

#!/bin/bash

# Set Oracle environment
ORACLE_BASE=/oracle; export ORACLE_BASE
ORACLE_HOME=${ORACLE_BASE}/product/10.2.0.5; export ORACLE_HOME

# Set name of the listener
LSNR=LISTENER

lsnrctl<< EOF > /tmp/lsnr_log.temp
set current_listener ${LSNR}
show log_directory
show log_file
exit
EOF

LSNR_DIR=`cat /tmp/lsnr_log.temp| grep log_directory|awk '{print $6}'`;
LSNR_FILE=`cat /tmp/lsnr_log.temp| grep log_file|awk '{print $6}'`;
LOGFILE=${LSNR_DIR}${LSNR_FILE}

if [ -f ${LOGFILE} ]
then

cp ${LOGFILE} ${LOGFILE}.bak #Coment this line if you don't want backup of old file
cat /dev/null > ${LOGFILE}
fi

rm -f /tmp/lsnr_log.temp

Don’t need to touch the listener. There is however a chance of losing some log entries between the cp and cat.

1 comment:

Oracle ASM Concepts

Note: Proofread any scripts before using. Always try scripts on a test instance first. This Blog is not responsible for any damage. O...