Archive and statistics¶

• Table of contents
• Archive and statistics
  • 1 Basic concept
    • 1.1 Hardware
    • 1.2 Syslog message format
    • 1.3 Development steps
  • 2 Installing and configuring syslog-ng daemon on proxy
    • 2.1 syslog-ng daemon on proxy
    • 2.2 syslogd configuration on FRI-SMS
  • Statistical Data Analysis Software
  • References

TBD - to be discussed
TODO - to do :)
Tole sem pustil zaenkrat not, da lahko hitro najdem stvari, ki jih moram se dopolnit oziroma jih moramo se predebatirat. V koncni verziji se bom teh kratic znebil in tudi tega komentarja v slovenscini :)

1 Basic concept¶

TODO Why syslog

Eneraptor is designed in a way that it could communicate with any device regardless of its operating system. Syslog, system for logging program messages offers a quick and easy solutions for such communications, because it allows separation of the software that generates messages from the system that stores them and the software that reports and analyzes them.

TODO malo nerodno zapisano?

1.1 Hardware¶

We will be using syslog-ng (syslog from now on), which has a few advantages over conventional syslog. Most basic setup , originator - collector, suits our needs just fine so far, but it can also be extended in the future. Syslog daemon (collector) will run on proxy server, syslog client (originator) will run on FRI-SMS system.

TBD collector will be listening to a certain port (5140 - most commonly used) for incoming messages from client(s).

Client mode operation:

In client mode, syslog-ng collects the local logs generated by the host and forwards them through a network connection to the central syslog-ng server. Clients can also log the messages locally into files.

TODO more details on syslog-ng advantages over conventional syslog?

1.2 Syslog message format¶

TODO Mogoce tega sploh ne bo treba vedet, program()

TBD TCP/IP vs UDP
TBD How do we plan to send messages to our daemon, via client (not always possible) or do we generate them ourselves?
TBD Using logfiles vs logging into database

1.3 Development steps¶

1. Installing and configuring Syslog-ng daemon on proxy
2. Installing and configuring Syslog-ng client on FRI-SMS
3. TBD Designing Statistical Data Analysis Software to work with data collected by syslog daemon
4. TBD Implementing SDA software
5. TBD Testing?

2 Installing and configuring syslog-ng daemon on proxy¶

Syslog-ng supports wide variety of Operating Systems so we aren't really restricted here. Since our proxy and FRI-SMS both run Linux 2.6 our obvious choce is syslog-ng for Linux.

2.1 syslog-ng daemon on proxy¶

Configuration file location:
/opt/syslog-ng/etc/syslog-ng.conf

Basic syslog configuration:

@version: 3.2
#Default configuration file for syslog-ng.
#
# For a description of syslog-ng configuration file directives, please read
# the syslog-ng Administrator's guide at:
#
# http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html
#

options {
        time_reap(30);
        mark_freq(10);
        keep_hostname(yes);
    };

######
# sources
source s_local {
    # message generated by Syslog-NG
    internal();
    # standard Linux log source (this is the default place for the syslog()
    # function to send logs to)
    unix-stream("/dev/log");
    # messages from the kernel
    file("/proc/kmsg" program_override("kernel"));
};

# source s_syslog { syslog(ip(127.0.0.1) port(1999) transport("tcp")); };

########################
# Filters
########################
# Here's come the filter options. With this rules, we can set which 
# message go where.

# messages for eneraptor should have priority level LOG_MAIL and contain keyword "eneraptor" 
filter f_mail_eneraptor { facility(mail) and match("eneraptor"); };

######
# destinations
destination d_messages        { file("/var/log/messages"); };
destination filtered_messages { file("/var/log/messages_filtered"); };

#
# SQL logging support
#

destination d_pgsql {
  sql(type(pgsql)
  host("127.0.0.1") port(5432) username("eneraptor") password("eneraptor")
  database("eneraptordb")
  table("logs")
  columns("datetime varchar(16)", "host varchar(32)", "program varchar(80)", "pid varchar(80)", "message varchar(200)")
  values("$R_DATE", "$HOST", "$PROGRAM", "$PID", "$MSG")
  indexes("datetime", "host", "program", "pid", "message"));
};

log {
source(s_local);
destination(d_messages);
# destination(d_pgsql);
};

log {
source(s_local);
filter(f_mail_eneraptor);
destination(filtered_messages);
destination(d_pgsql);
};

2.2 syslogd configuration on FRI-SMS¶

Edit

/etc/inittab

- change:

null::respawn:/sbin/syslogd -n -m 0

to

null::respawn:/sbin/syslogd -n -m 0 -R 192.168.10.1

where

192.168.10.1

is the IP of the server running syslog-ng. Finally,
have init reload it's configuration by running:

kill -1 1

syslog-ng restart command:

/etc/init.d/syslog-ng restart

Statistical Data Analysis Software¶

TODO

References¶

Syslog protocol, RFC 5424
Syslog-ng homepage
Syslog-ng administrator guide