EPSIAE » eneraptor

Archive and statistics » History » Revision 9

Revision 8 (Janez Barbic, 12.12.2010 16:19) → Revision 9/17 (Janez Barbic, 12.12.2010 16:31)

h1. Archive and statistics 

 WIP - trenutno se delam na dokumentu, ne se brat :) 

 {{toc}} 

 TBD - to be discussed 
 TODO - to do :) 
 Tole sem pustil zaenkrat not, da lahko hitro najdem stvari, ki jih moram se dopolnit oziroma jih moramo se predebatirat. V koncni verziji se bom teh kratic znebil in tudi tega komentarja v slovenscini :) 

 h2. 1 Basic concept 

 TODO Zakaj syslog 

 Eneraptor is designed in a way that it could communicate with any device regardless of its operating system. Syslog, system for logging program messages offers a quick and easy solutions for such communications, because it allows separation of the software that generates messages from the system that stores them and the software that reports and analyzes them. It could refer to syslog protocol (TODO described below),  

 h3. 1.1 Hardware 

 p. We will be using syslog-ng (I will be referring to it as syslog from now on), which has a few advantages over conventional syslog. Syslog daemon will run on proxy server. TBD it will be listening on a certain port for incoming messages from client(s). Our main syslog client will run on FRI-SMS system. 

 Client mode operation: 

 !https://lusy.fri.uni-lj.si/redmine/attachments/11/client_server_syslog.png! 

 p. In client mode, syslog-ng collects the local logs generated by the host and forwards them through a network connection to the central syslog-ng server. Clients can also log the messages locally into files. 


 TODO more details on syslog-ng advantages 

 h3. 1.2 Syslog protocol overview 

 TBD TCP/IP vs UDP 
 Message structure 
 TBD How do we plan to send messages to our daemon, via client (not always possible) or do we generate them ourselves? 
 TBD Using logfiles vs logging into database 

 h3. 1.3 Development steps 

 # Installing and configuring Syslog-ng daemon on proxy 
 # Installing and configuring Syslog-ng client on FRI-SMS 
 # TBD Designing Statistical Data Analysis Software to work with data collected by syslog daemon 
 # TBD Implementing SDA software 
 # TBD Testing? 


 h2. 2 Installing and configuring syslog-ng daemon on proxy 

 p. Syslog-ng supports wide variety of Operating Systems so we aren't really restricted here. Since our proxy and FRI-SMS both run Linux 2.6 our obvious choce is syslog-ng for Linux. 

 
 h3. 2.1 syslog-ng daemon on proxy 

 
 Configuration file location: 
 /opt/syslog-ng/etc/syslog-ng.conf 

 Basic syslog configuration: 
 TODO add filters 
 <pre><code class="ruby"> 
 @version: 3.2 
 #Default configuration file for syslog-ng. 
 # 
 # For a description of syslog-ng configuration file directives, please read 
 # the syslog-ng Administrator's guide at: 
 # 
 # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html 
 # 

 options { 
    time_reap(30); 
    mark_freq(10); 
    keep_hostname(yes); 
 }; 

 ###### 
 # sources 
 source s_local { 
 }; 

 source s_net { 
    tcp(ip("SERVER_ADDRESS") port(5140) keep-alive(yes));  
 }; 

 ###### 
 # destinations 
 # should be able to point syslog to a database here 
 destination net_messages { file("/var/log/net_messages"); }; 

 log { 
 source(s_net); 
 destination(net_messages); 
 }; 
 </code></pre> 

 h3. 2.2 syslog-ng client on FRI-SMS 

 Configuration file location: 
 /opt/syslog-ng/etc/syslog-ng.conf 

 Basic syslog client configuration: 
 TODO add filters 

 <pre><code class="ruby"> 
 @version: 3.2 
 #Default configuration file for syslog-ng. 
 # 
 # For a description of syslog-ng configuration file directives, please read 
 # the syslog-ng Administrator's guide at: 
 # 
 # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html 
 # 

 options { 
 }; 

 ###### 
 # sources 
 source s_local { 
    # TBD what triggers our messages 
    # message generated by Syslog-NG 
    internal(); 
    # standard Linux log source (this is the default place for the syslog() 
    # function to send logs to) 
    unix-stream("/dev/log"); 
    # messages from the kernel 
    file("/proc/kmsg" program_override("kernel")); 
 }; 


 ###### 
 # destinations 
 # local 
 # destination d_messages { file("/var/log/messages"); }; 
 # net 
 destination host { tcp("SERVER_ADDRESS" port(5140)); }; 

 # logging locally 
 # log { 
 #     source(s_local); 
 #     destination(d_messages); 
 #}; 

 # sending log messages 
 log { 
    source(s_local); 
    destination(host); 
 }; 
 </code></pre>