SourceFiles.org - Use the Source, Luke
Home | Register | News | Forums | Guide | MyLinks | Bookmark

Sponsored Links

Latest News
  General News
  Reviews
  Press Releases
  Software
  Hardware
  Security
  Tutorials
  Off Topic


Back to files

Log Scanner Documentation

Changelog

v1.0
-No longer requires "tail -f" (thanks to Paul Sand for info on this) -If a log file shrinks, logscanner now starts reading from the beginning again (to deal with log rotators) -Wrote a nifty installation program.

About

The latest version of Log Scanner can be retrieved via ftp or http at logscanner.tradeservices.com

The Log Scanner is written in Perl and designed to work in conjunction with the "TCP wrappers" package. There is no longer a dependency upon "tail -f" behaving in a "standard" (our definition of standard, anyway) manner. It was originally designed to run on a Redhat 4.2 Linux distribution. Keep in mind that we the original design was meant for our own proprietary use so some of it may not be of any use, and may in fact not function at all. We accept no responsibility for anything this program may do to your system. To quote the web site, "Any warrantee, explicit or implied, is a figment of your imagination." This software is being released under the GPL.

The purpose behind the log scanner is to enable a system administrator to set up a log parser that will contact them (or others) when predefined anomalies are discovered in a log file.

Configurability

The Log Scanner is designed to be configurable to the point that anybody could add extra checking ability, though some knowledge of perl is necessary. The conf files are set up as text files where the fields in each line are colon delimited. Comments in each file are accomplished by putting the first character as a `#'. A newline is the termination for each configuration line.

Rules.cfg:

This file (which resides in /etc/lscan by default) is the main configuration file to control what you want Log Scanner to do. This section will go over how to write this configuration file (there should be a default one already there). The general format of the file is that each line begins with a single character "tag" and then goes on to include some colon delimited parameters for each tag. Let's look at the specifics.

c:name:method:contact

The `c' tag denotes that you are setting up a contact. The name field allows you to choose a reference name for this contact. The method field is one of: email, log, or audiofile. Email will send email to an address, log will put an entry in a log file, and audiofile will cat an audiofile to your /dev/audio. Eventually, there will be support for externalprogram, beeper, and phone as well. The contact field allows you to put in an email address, log filename, or audio filename, as appropriate.

g:name:contact,contact,contact...

The `g' tag denotes that you are setting up a contact group. The name field allows you to choose a reference name for this contact. The contact list is just a comma separated list of (already defined!) contacts.

o:name:ouptput format

The `o' tag denotes that you are setting up an output style. The name field allows you to choose a reference name for this style. The output format must always begin and end with a dboule quote. The rest of it is up to you, whatever text you wish to put in it. You have access to several variables you may use as well. References to variables must always begin with a `^'. Here is a brief list and description of the variables you have available.

        month: month for that log line in Mmm format
        day: day of the month
        time: in hh:mm:ss format
        host: hostname (not domain name) of the machine running the log
        process: the process from the log line
        pid: the process id that triggered the rule
        logline: the message generated on the log line
        remotehost: the remote host related to the log line
        user: the user related to the log line
  • The availability of these fields is contingent upon the test function returning this information. More on this later.

r:name: function:xhymzs:c|g[,c|g]..:[rule]...

The r denotes that you are setting up a rule. The name field allows you to choose a reference name for this rule. The function field signifies the function to be called to test the condition you are specifying. More on this later. The next field is a "timeout" field. The purpose of this field is that when a rule is triggered, Log Scanner sets a time flag. The timeout will tell Log Scanner not to check this rule again until this timeout period has been reached. This will hopefully help to keep redundant contacting to a minimum. The next field lists the contact(s) and/or contact group(s) that you want to notify should this rule be activated. The final field is for any other rules that you want to use before verification has occurred. Keep in mind that ALL of these functions must return a true condition in order for the contact(s) to be notified.

f:name:logfile[,logfile],...:rule[,rule]...:format[,format]

The f denotes that you are setting up files to be scanned. The name is to allow you to choose a reference for the files. The logfile is where you specify the file (most frequently, something from /var/log) or files to be scanned. Each file must be referenced by full pathname in this field. If you want to scan against more than one file, enter them as a comma delimited list. The rule field is where you put in the "l" rules that you want to use to scan the file. Note that each rule can be activated separately. If you want to specify more than one rule, just reference them in a comma delimited list. The final field is where you specify the output format for each rule. There should be one format listed for each rule.

Function Modules:

Here's where your knowledge of perl will (likely have to) kick in. This is where the actual checking begins. The perl function you write will be passed one argument, an index into an array that will give you the line to test. The "array" is actually an array of hashes that have two elements. It looks like this:

$logs[$_[0]]{`logline'} - this is the actual line from the log that you should

test against

$logs[$_[0]]{`filename'} - this is the log file that the line came from ... in

                           case you need to know that for any checking you may 
                           perform

There are also some functions you have access to in order to make life a little easier. Take a look at Appendix A for a list and description of these functions.

Return values: Your function must return a 0 or non-0 value. A value of zero will tell the calling function that it failed and the contact should NOT be notified. The only other significant return values are either a 1, or a specially formatted scalar array. The first entry in this scalar array should be the source (remote host, etc) of the logline (you will have to parse this from the line yourself) and then the user that is related to the log line (if applicable). You must return them in that order and you must return both values (unless you are return a 1), even if the value is just a "".

Appendix A: Available Functions

head_parse: To aid in the parsing of the header of each line (which we are assuming to be in the format of: month, day, time, host, process, and pid) you have available to you the subroutine `head_parse'. This will return a hash with the fields listed above.

ourtime_to_seconds: Dealing with time translations can be a pain and so you also have available `ourtime_to_seconds'. This will translate a string in the format `xhymzs' to an integer representing the number of seconds in x hours, y minutes, and z seconds.

to_epoch: To aid in checking previous lines (for instance, to scan for a "strobe"), you have available the "to_epoch" function. Just pass it a log line and it will hand you the epoch time (number of seconds since January 1, 1970) that it occurred. You can then translate a time (see ourtime_to_seconds) and see if the time the current event occurred and the previously indexed event occurred is less than that amount of time.


Sponsored Links

Discussion Groups
  Beginners
  Distributions
  Networking / Security
  Software
  PDAs

About | FAQ | Privacy | Awards | Contact
Comments to the webmaster are welcome.
Copyright 2006 Sourcefiles.org All rights reserved.