Archive and statistics » History » Version 7
Janez Barbic, 12.12.2010 15:07
1 | 1 | Aleksander Bešir | h1. Archive and statistics |
---|---|---|---|
2 | 1 | Aleksander Bešir | |
3 | 6 | Janez Barbic | WIP - trenutno se delam na dokumentu, ne se brat :) |
4 | 6 | Janez Barbic | |
5 | 2 | Aleksander Bešir | {{toc}} |
6 | 2 | Aleksander Bešir | |
7 | 5 | Janez Barbic | TBD - to be discussed |
8 | 5 | Janez Barbic | TODO - to do :) |
9 | 5 | Janez Barbic | Tole sem pustil zaenkrat not, da lahko hitro najdem stvari, ki jih moram se dopolnit oziroma jih moramo se predebatirat. V koncni verziji se bom teh kratic znebil in tudi tega komentarja v slovenscini :) |
10 | 1 | Aleksander Bešir | |
11 | 6 | Janez Barbic | h2. 1 Basic concept |
12 | 3 | Janez Barbic | |
13 | 3 | Janez Barbic | TODO Zakaj syslog |
14 | 3 | Janez Barbic | |
15 | 3 | Janez Barbic | Eneraptor is designed in a way that it could communicate with any device regardless of its operating system. Syslog, system for logging program messages offers a quick and easy solutions for such communications, because it allows separation of the software that generates messages from the system that stores them and the software that reports and analyzes them. It could refer to syslog protocol (TODO described below), |
16 | 3 | Janez Barbic | |
17 | 3 | Janez Barbic | h3. 1.1 Hardware |
18 | 1 | Aleksander Bešir | |
19 | 5 | Janez Barbic | p. We will be using syslog-ng (I will be referring to it as syslog from now on), which has a few advantages over conventional syslog. Syslog daemon will run on proxy server. TBD it will be listening on a certain port for incoming messages from client(s). Our main syslog client will run on FRI-SMS system. |
20 | 1 | Aleksander Bešir | |
21 | 5 | Janez Barbic | Client mode operation: |
22 | 5 | Janez Barbic | |
23 | 1 | Aleksander Bešir | !https://lusy.fri.uni-lj.si/redmine/attachments/11/client_server_syslog.png! |
24 | 1 | Aleksander Bešir | |
25 | 5 | Janez Barbic | p. In client mode, syslog-ng collects the local logs generated by the host and forwards them through a network connection to the central syslog-ng server. Clients can also log the messages locally into files. |
26 | 5 | Janez Barbic | |
27 | 5 | Janez Barbic | |
28 | 5 | Janez Barbic | TODO more details on syslog-ng advantages |
29 | 5 | Janez Barbic | |
30 | 5 | Janez Barbic | h3. 1.2 Syslog protocol overview |
31 | 5 | Janez Barbic | |
32 | 5 | Janez Barbic | TBD TCP/IP vs UDP |
33 | 5 | Janez Barbic | Message structure |
34 | 5 | Janez Barbic | TBD How do we plan to send messages to our daemon, via client (not always possible) or do we generate them ourselves? |
35 | 5 | Janez Barbic | TBD Using logfiles vs logging into database |
36 | 5 | Janez Barbic | |
37 | 5 | Janez Barbic | h3. 1.3 Development steps |
38 | 5 | Janez Barbic | |
39 | 5 | Janez Barbic | # Installing and configuring Syslog-ng daemon on proxy |
40 | 5 | Janez Barbic | # Installing and configuring Syslog-ng client on FRI-SMS |
41 | 5 | Janez Barbic | # TBD Designing Statistical Data Analysis Software to work with data collected by syslog daemon |
42 | 5 | Janez Barbic | # TBD Implementing SDA software |
43 | 1 | Aleksander Bešir | # TBD Testing? |
44 | 6 | Janez Barbic | |
45 | 6 | Janez Barbic | |
46 | 6 | Janez Barbic | h2. 2 Installing and configuring syslog-ng daemon on proxy |
47 | 7 | Janez Barbic | |
48 | 7 | Janez Barbic | p. Syslog-ng supports wide variety of Operating Systems so we aren't really restricted here. Since our proxy and FRI-SMS both run Linux 2.6 our obvious choce is syslog-ng for Linux. |
49 | 1 | Aleksander Bešir | h3. 2.1 syslog-ng daemon on proxy |
50 | 7 | Janez Barbic | Configuration file location: |
51 | 7 | Janez Barbic | /opt/syslog-ng/etc/syslog-ng.conf |
52 | 7 | Janez Barbic | |
53 | 7 | Janez Barbic | <pre><code class="ruby"> |
54 | 7 | Janez Barbic | @version: 3.2 |
55 | 7 | Janez Barbic | #Default configuration file for syslog-ng. |
56 | 7 | Janez Barbic | # |
57 | 7 | Janez Barbic | # For a description of syslog-ng configuration file directives, please read |
58 | 7 | Janez Barbic | # the syslog-ng Administrator's guide at: |
59 | 7 | Janez Barbic | # |
60 | 7 | Janez Barbic | # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html |
61 | 7 | Janez Barbic | # |
62 | 7 | Janez Barbic | |
63 | 7 | Janez Barbic | options { |
64 | 7 | Janez Barbic | time_reap(30); |
65 | 7 | Janez Barbic | mark_freq(10); |
66 | 7 | Janez Barbic | keep_hostname(yes); |
67 | 7 | Janez Barbic | }; |
68 | 7 | Janez Barbic | |
69 | 7 | Janez Barbic | ###### |
70 | 7 | Janez Barbic | # sources |
71 | 7 | Janez Barbic | source s_local { |
72 | 7 | Janez Barbic | # message generated by Syslog-NG |
73 | 7 | Janez Barbic | internal(); |
74 | 7 | Janez Barbic | # standard Linux log source (this is the default place for the syslog() |
75 | 7 | Janez Barbic | # function to send logs to) |
76 | 7 | Janez Barbic | unix-stream("/dev/log"); |
77 | 7 | Janez Barbic | # messages from the kernel |
78 | 7 | Janez Barbic | file("/proc/kmsg" program_override("kernel")); |
79 | 7 | Janez Barbic | |
80 | 7 | Janez Barbic | |
81 | 7 | Janez Barbic | |
82 | 7 | Janez Barbic | |
83 | 7 | Janez Barbic | |
84 | 7 | Janez Barbic | |
85 | 7 | Janez Barbic | |
86 | 7 | Janez Barbic | }; |
87 | 7 | Janez Barbic | |
88 | 7 | Janez Barbic | source s_net { |
89 | 7 | Janez Barbic | udp(); |
90 | 7 | Janez Barbic | tcp(); |
91 | 7 | Janez Barbic | tcp(ip("192.168.1.5") port(5140) keep-alive(yes)); |
92 | 7 | Janez Barbic | }; |
93 | 7 | Janez Barbic | |
94 | 7 | Janez Barbic | |
95 | 7 | Janez Barbic | ###### |
96 | 7 | Janez Barbic | # destinations |
97 | 7 | Janez Barbic | destination d_messages { file("/var/log/messages"); }; |
98 | 7 | Janez Barbic | destination net_messages { file("/var/log/net_messages"); }; |
99 | 7 | Janez Barbic | |
100 | 7 | Janez Barbic | log { |
101 | 7 | Janez Barbic | source(s_local); |
102 | 7 | Janez Barbic | destination(d_messages); |
103 | 7 | Janez Barbic | }; |
104 | 7 | Janez Barbic | |
105 | 7 | Janez Barbic | log { |
106 | 7 | Janez Barbic | source(s_net); |
107 | 7 | Janez Barbic | destination(net_messages); |
108 | 7 | Janez Barbic | }; |
109 | 7 | Janez Barbic | |
110 | 7 | Janez Barbic | </code></pre> |
111 | 6 | Janez Barbic | |
112 | 6 | Janez Barbic | h3. 2.2 syslog-ng client on FRI-SMS |