Archive and statistics » History » Version 11
Janez Barbic, 13.12.2010 00:51
1 | 1 | Aleksander Bešir | h1. Archive and statistics |
---|---|---|---|
2 | 1 | Aleksander Bešir | |
3 | 6 | Janez Barbic | WIP - trenutno se delam na dokumentu, ne se brat :) |
4 | 6 | Janez Barbic | |
5 | 2 | Aleksander Bešir | {{toc}} |
6 | 2 | Aleksander Bešir | |
7 | 5 | Janez Barbic | TBD - to be discussed |
8 | 5 | Janez Barbic | TODO - to do :) |
9 | 5 | Janez Barbic | Tole sem pustil zaenkrat not, da lahko hitro najdem stvari, ki jih moram se dopolnit oziroma jih moramo se predebatirat. V koncni verziji se bom teh kratic znebil in tudi tega komentarja v slovenscini :) |
10 | 1 | Aleksander Bešir | |
11 | 6 | Janez Barbic | h2. 1 Basic concept |
12 | 3 | Janez Barbic | |
13 | 11 | Janez Barbic | TODO Why syslog |
14 | 3 | Janez Barbic | |
15 | 11 | Janez Barbic | p. Eneraptor is designed in a way that it could communicate with any device regardless of its operating system. Syslog, system for logging program messages offers a quick and easy solutions for such communications, because it allows separation of the software that generates messages from the system that stores them and the software that reports and analyzes them. |
16 | 1 | Aleksander Bešir | |
17 | 11 | Janez Barbic | TODO malo nerodno zapisano? |
18 | 11 | Janez Barbic | |
19 | 3 | Janez Barbic | h3. 1.1 Hardware |
20 | 1 | Aleksander Bešir | |
21 | 11 | Janez Barbic | p. We will be using syslog-ng (syslog from now on), which has a few advantages over conventional syslog. Most basic setup , originator - collector, suits our needs just fine so far, but it can also be extended in the future. Syslog daemon (collector) will run on proxy server, syslog client (originator) will run on FRI-SMS system. |
22 | 1 | Aleksander Bešir | |
23 | 11 | Janez Barbic | TBD collector will be listening to a certain port (5140 - most commonly used) for incoming messages from client(s). |
24 | 11 | Janez Barbic | |
25 | 5 | Janez Barbic | Client mode operation: |
26 | 1 | Aleksander Bešir | |
27 | 5 | Janez Barbic | !https://lusy.fri.uni-lj.si/redmine/attachments/11/client_server_syslog.png! |
28 | 1 | Aleksander Bešir | |
29 | 1 | Aleksander Bešir | p. In client mode, syslog-ng collects the local logs generated by the host and forwards them through a network connection to the central syslog-ng server. Clients can also log the messages locally into files. |
30 | 5 | Janez Barbic | |
31 | 1 | Aleksander Bešir | |
32 | 11 | Janez Barbic | TODO more details on syslog-ng advantages over conventional syslog? |
33 | 5 | Janez Barbic | |
34 | 11 | Janez Barbic | h3. 1.2 Syslog message format |
35 | 5 | Janez Barbic | |
36 | 11 | Janez Barbic | |
37 | 11 | Janez Barbic | |
38 | 5 | Janez Barbic | TBD TCP/IP vs UDP |
39 | 5 | Janez Barbic | TBD How do we plan to send messages to our daemon, via client (not always possible) or do we generate them ourselves? |
40 | 5 | Janez Barbic | TBD Using logfiles vs logging into database |
41 | 5 | Janez Barbic | |
42 | 5 | Janez Barbic | h3. 1.3 Development steps |
43 | 5 | Janez Barbic | |
44 | 5 | Janez Barbic | # Installing and configuring Syslog-ng daemon on proxy |
45 | 5 | Janez Barbic | # Installing and configuring Syslog-ng client on FRI-SMS |
46 | 5 | Janez Barbic | # TBD Designing Statistical Data Analysis Software to work with data collected by syslog daemon |
47 | 5 | Janez Barbic | # TBD Implementing SDA software |
48 | 1 | Aleksander Bešir | # TBD Testing? |
49 | 6 | Janez Barbic | |
50 | 6 | Janez Barbic | |
51 | 6 | Janez Barbic | h2. 2 Installing and configuring syslog-ng daemon on proxy |
52 | 7 | Janez Barbic | |
53 | 7 | Janez Barbic | p. Syslog-ng supports wide variety of Operating Systems so we aren't really restricted here. Since our proxy and FRI-SMS both run Linux 2.6 our obvious choce is syslog-ng for Linux. |
54 | 9 | Janez Barbic | |
55 | 1 | Aleksander Bešir | h3. 2.1 syslog-ng daemon on proxy |
56 | 9 | Janez Barbic | |
57 | 7 | Janez Barbic | Configuration file location: |
58 | 7 | Janez Barbic | /opt/syslog-ng/etc/syslog-ng.conf |
59 | 7 | Janez Barbic | |
60 | 8 | Janez Barbic | Basic syslog configuration: |
61 | 8 | Janez Barbic | TODO add filters |
62 | 7 | Janez Barbic | <pre><code class="ruby"> |
63 | 7 | Janez Barbic | @version: 3.2 |
64 | 7 | Janez Barbic | #Default configuration file for syslog-ng. |
65 | 7 | Janez Barbic | # |
66 | 7 | Janez Barbic | # For a description of syslog-ng configuration file directives, please read |
67 | 7 | Janez Barbic | # the syslog-ng Administrator's guide at: |
68 | 7 | Janez Barbic | # |
69 | 7 | Janez Barbic | # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html |
70 | 7 | Janez Barbic | # |
71 | 1 | Aleksander Bešir | |
72 | 1 | Aleksander Bešir | options { |
73 | 8 | Janez Barbic | time_reap(30); |
74 | 8 | Janez Barbic | mark_freq(10); |
75 | 8 | Janez Barbic | keep_hostname(yes); |
76 | 8 | Janez Barbic | }; |
77 | 7 | Janez Barbic | |
78 | 7 | Janez Barbic | ###### |
79 | 7 | Janez Barbic | # sources |
80 | 7 | Janez Barbic | source s_local { |
81 | 7 | Janez Barbic | }; |
82 | 7 | Janez Barbic | |
83 | 7 | Janez Barbic | source s_net { |
84 | 8 | Janez Barbic | tcp(ip("SERVER_ADDRESS") port(5140) keep-alive(yes)); |
85 | 7 | Janez Barbic | }; |
86 | 7 | Janez Barbic | |
87 | 7 | Janez Barbic | ###### |
88 | 7 | Janez Barbic | # destinations |
89 | 8 | Janez Barbic | # should be able to point syslog to a database here |
90 | 7 | Janez Barbic | destination net_messages { file("/var/log/net_messages"); }; |
91 | 7 | Janez Barbic | |
92 | 7 | Janez Barbic | log { |
93 | 7 | Janez Barbic | source(s_net); |
94 | 7 | Janez Barbic | destination(net_messages); |
95 | 7 | Janez Barbic | }; |
96 | 7 | Janez Barbic | </code></pre> |
97 | 6 | Janez Barbic | |
98 | 6 | Janez Barbic | h3. 2.2 syslog-ng client on FRI-SMS |
99 | 9 | Janez Barbic | |
100 | 9 | Janez Barbic | Configuration file location: |
101 | 9 | Janez Barbic | /opt/syslog-ng/etc/syslog-ng.conf |
102 | 9 | Janez Barbic | |
103 | 9 | Janez Barbic | Basic syslog client configuration: |
104 | 9 | Janez Barbic | TODO add filters |
105 | 9 | Janez Barbic | |
106 | 9 | Janez Barbic | <pre><code class="ruby"> |
107 | 9 | Janez Barbic | @version: 3.2 |
108 | 9 | Janez Barbic | #Default configuration file for syslog-ng. |
109 | 9 | Janez Barbic | # |
110 | 9 | Janez Barbic | # For a description of syslog-ng configuration file directives, please read |
111 | 9 | Janez Barbic | # the syslog-ng Administrator's guide at: |
112 | 9 | Janez Barbic | # |
113 | 9 | Janez Barbic | # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html |
114 | 9 | Janez Barbic | # |
115 | 9 | Janez Barbic | |
116 | 9 | Janez Barbic | options { |
117 | 9 | Janez Barbic | }; |
118 | 9 | Janez Barbic | |
119 | 9 | Janez Barbic | ###### |
120 | 9 | Janez Barbic | # sources |
121 | 9 | Janez Barbic | source s_local { |
122 | 9 | Janez Barbic | # TBD what triggers our messages |
123 | 9 | Janez Barbic | # message generated by Syslog-NG |
124 | 9 | Janez Barbic | internal(); |
125 | 9 | Janez Barbic | # standard Linux log source (this is the default place for the syslog() |
126 | 9 | Janez Barbic | # function to send logs to) |
127 | 9 | Janez Barbic | unix-stream("/dev/log"); |
128 | 9 | Janez Barbic | # messages from the kernel |
129 | 9 | Janez Barbic | file("/proc/kmsg" program_override("kernel")); |
130 | 9 | Janez Barbic | }; |
131 | 9 | Janez Barbic | |
132 | 9 | Janez Barbic | |
133 | 9 | Janez Barbic | ###### |
134 | 9 | Janez Barbic | # destinations |
135 | 9 | Janez Barbic | # local |
136 | 9 | Janez Barbic | # destination d_messages { file("/var/log/messages"); }; |
137 | 9 | Janez Barbic | # net |
138 | 9 | Janez Barbic | destination host { tcp("SERVER_ADDRESS" port(5140)); }; |
139 | 9 | Janez Barbic | |
140 | 9 | Janez Barbic | # logging locally |
141 | 9 | Janez Barbic | # log { |
142 | 9 | Janez Barbic | # source(s_local); |
143 | 9 | Janez Barbic | # destination(d_messages); |
144 | 9 | Janez Barbic | #}; |
145 | 9 | Janez Barbic | |
146 | 9 | Janez Barbic | # sending log messages |
147 | 9 | Janez Barbic | log { |
148 | 9 | Janez Barbic | source(s_local); |
149 | 9 | Janez Barbic | destination(host); |
150 | 9 | Janez Barbic | }; |
151 | 10 | Janez Barbic | </code></pre> |
152 | 1 | Aleksander Bešir | |
153 | 1 | Aleksander Bešir | h2. Statistical Data Analysis Software |
154 | 1 | Aleksander Bešir | |
155 | 10 | Janez Barbic | TODO |
156 | 10 | Janez Barbic | |
157 | 10 | Janez Barbic | h2. References |
158 | 10 | Janez Barbic | |
159 | 11 | Janez Barbic | "Syslog protocol, RFC 5424":http://tools.ietf.org/html/rfc5424 |
160 | 11 | Janez Barbic | "Syslog-ng homepage":http://www.balabit.com/network-security/syslog-ng |
161 | 11 | Janez Barbic | "Syslog-ng administrator guide":http://www.balabit.com/support/documentation/syslog-ng-ibm-agent-guide-admin-en.pdf |