...
1. As an admin, I should be able to set PBS logging via syslog so that all daemons log into the same file.
2. As an admin, I should be able to control log events level of the PBS logging.
3. As an admin, I should be able to set the logging via syslog for all nodes in the cluster
4. As an admin, I should be able to set the logging via syslog for all servers in a failover environment.
5. As a admin I should be able to have logging into both local daemons and Syslog enabled together
6. As an admin/user, I should be able to run tracejob for a specific job with Syslog.
Note- This functionality is not currently given by PBS. This is added here because it is nice enhancement for future use.
7. As an admin/user, I should be able to differentiate log messages from each daemon.
8. As an admin/user, I should be able to set logging via syslog for only a particular daemon (Server, Mom, Sched or Comm).
Design
For adding syslog support in PTL we will have to make changes in two files- pbs_logutils.py and pbs_testlib.py. We should make sure that the user running the tests will have permissions to read syslog file and decompress it. For eg - in test bed machine pbsroot shoud be able to read from syslog file.
A new class is to be added for reading the date and syslog messages file path.
———————syslog_config.py——————
import platform
// This dictionary will hold the values for the sysconf files for various linux distributions
syslog_conf_path = {'centos7': '/etc/rsyslog’.conf, 'RHEL7': '/etc/rsyslog'}
class syslog_config(object):
def _init_(self):
// Get OS version of the system
os = platform.dist()[0] + platform.dist()[1][0]
path = syslog_conf_path[os]
def path_to_syslog(self):
// Read the path of the syslog messages file
return path_to_messages
def time_syslog(self):
// Getting the date format logged in the syslog file
return time_format
def log_config_values(syslog=0)
// logic for which file to read will come from the above table
file_to_check=0
// read from PBS.conf PBS_SYSLOG & PBS_LOCALLOG
// Logic of which file to read log messages from- syslog / local
// file_to_check =1 for syslog, file_to_check=2 for local, file_to_check=3 for both
return file_to_check
————pbs_testlib.py——————————
To add the functionality on which log logs to check in log_match(), syslog or local logs we will follow this table -
PBS_SYSLOG and PBS_LOCALLOG are values from the pbs.conf file. Syslog_attribute is the attribute to be passed to the log_match() function. Return value is the return value of log_match()
| |||||||||||||||||||||||||||||||||||||||||||||||||
|
...
In row 7 where we have to match in both syslog and local log, we would have to call _log_match() twice.
For adding syslog support in PTL we will have to make changes in two files- pbs_logutils.py and pbs_testlib.py. We should make sure that the user running the tests will have permissions to read syslog file and decompress it. For eg - in test bed machine pbsroot shoud be able to read from syslog file.
Changes in pbs_logutils.pyFollowing changes are done to the current log_match() function
1) Adding extra attribute 'readSyslogsyslog' to _log_match(readSyslogsyslog=False)
By default this attribute should be set to false
2) Adding extra attribute 'readSyslogsyslog' to log_match(readSyslogsyslog=False)
By default this attribute should be set to false
...
In case both have to be read we should call the _log_match() twice.
Changes in this file are related to the format of the recorded datetime and getting the path to the syslog file. Syslog records the datetime in a different format than local logs. According to the format we will have to do changes to reading the datetime.
Some syslog files are compressed. Even if we can derectly read a compressed file (eg .bz file) from python, but I would prefer if we decompress it so that rest of the code in PTL for reading files will work without any changes.
1) Read file from Syslog config file
Currently we are only supporting for the current syslog file and not for previous days file. That would be covered in PP-969
2) Date format.
Change regular expression ‘tm_re ’ to match the date format of the syslog file.
if PBS_SYSLOG=1 (in PBS.conf)
tm_re = "(?P<datetime>\w{3}\s\d\d\s\d\d:\d\d:\d\d)"
def convert_date_time()
if PBS_SYSLOG=1 (in PBS.conf)
Fmt of date = "%b %d %H:%M:%S"
t = time.strptime(datetime, fmt)
Changes in pbs_testlib.py
1) Change log_file path from local logs to syslog. The local log file are in 4 locations- server, mom, sched, comm. For syslog since there is only one file, we only have to point it to that file. Path to the log file can be set through a environment variable before starting the PTL tests or through the test script. If the path is not set the default path "/var/log/messages" will be taken
sys_log_file_path= sys_log_path
def init_logfile_path
if PBS_SYSLOG=1 and Local_log =0
class PBSService(PBSObject)
def log_match(syslog=1)
x= syslog_config.log_config_values(syslog=1)
if x=1
return _log_match(syslog=1)
elif x=2
return _log_match(syslog=0)
elif x=3
_log_match(syslog=1)
_log_match(syslog=0)
def _log_match(syslog=1)
if syslog=1
// Setting an environment variable because whether to read from syslog file will be considered in other files too.
// Setting an environment variable will be easier than passing the value.
os.environ["SYSLOG_VALUE"] = "1"
def init_logfile_path()
syslog_value = int(os.environ.get("SYSLOG_VALUE"))
if syslog_value = 1
path_log_file = syslog_
...
config.path_
...
to_syslog()
elif locally
...
path_log_file= __local_path__
Note- 'def log_lines' returns the last n lines of the log file.
def log_lines
...
()
syslog_value = int(os.environ.get("SYSLOG_VALUE"))
if syslog_value = 1
path_log_file = syslog_
...
config.path_
...
to_syslog()
elif locally
path_log_file= __local_path__
——————————In pbs_logutils.py—————————————
Currently we are only supporting for the current syslog file and not for previous days file. That would be covered in PP-969
Change regular expression ‘tm_re ’ to match the date format of the syslog file.
self.syslog_value = int(os.environ.get("SYSLOG_VALUE"))
if self.syslog_value = 1
tm_re = syslog_config.time_syslog()
def convert_date_time()
if syslog_value = 1
Fmt of date = syslog_config.time_syslog()
t = time.strptime(datetime, fmt)
Test Scenarios
1) Test that when PBS_SYSLOG=1 is enabled in pbs.conf, PBS logs messages via syslog.
...