{"id":1653,"date":"2017-03-10T13:40:40","date_gmt":"2017-03-10T13:40:40","guid":{"rendered":"http:\/\/cdblog.cdstealer.com\/?p=1653"},"modified":"2018-11-28T06:40:46","modified_gmt":"2018-11-28T06:40:46","slug":"systemd-syslog-ng","status":"publish","type":"post","link":"https:\/\/cdblog.cdstealer.com\/?p=1653","title":{"rendered":"Systemd & Syslog-ng"},"content":{"rendered":"\n\n\n
\"\"<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Systemd although very good, has caused problems for logging (for me). \u00a0Systemd does not play nice with logging applications such as Splunk. \u00a0The reason for this is that systemd takes over syslog and stores all log data in its journald system which uses tmpfs (RAM) until flushed to disk in its proprietary \u00a0format. \u00a0This is to make log data more secure.<\/p>\n

Here is how I got around that so that I could analyse my logs. \u00a0It's pretty straight forward and isn't too involved.<\/p>\n

Configure journald.conf<\/h3>\n

I have set the following options in my config and everything else is commented out.<\/p>\n

# cat \/etc\/systemd\/journald.conf \r\n\r\n[Journal]\r\nStorage=volatile\r\nForwardToSyslog=yes\r\nForwardToKMsg=no\r\nForwardToConsole=no\r\nForwardToWall=no<\/pre>\n
Configure Syslog-ng<\/h3>\n
The journald config above will now send everything to syslog which by default will store in \/var\/log\/messages. \u00a0To split out specific logs, you'll need to tell syslog what to do with them. \u00a0Below is a basic config to split a few logs that I'm interested in. \u00a0Some apps may use their own logger, so be aware of this.<\/p>\n
# cat \/etc\/syslog-ng\/syslog-ng.conf \r\n@version: 3.7\r\n# $Id$\r\n#\r\n# Syslog-ng default configuration file for Gentoo Linux\r\n\r\n# https:\/\/bugs.gentoo.org\/show_bug.cgi?id=426814\r\n@include \"scl.conf\"\r\n\r\noptions { \r\n threaded(yes);\r\n chain_hostnames(no); \r\n\r\n # The default action of syslog-ng is to log a STATS line\r\n # to the file every 10 minutes. That's pretty ugly after a while.\r\n # Change it to every 12 hours so you get a nice daily update of\r\n # how many messages syslog-ng missed (0).\r\n stats_freq(43200); \r\n # The default action of syslog-ng is to log a MARK line\r\n # to the file every 20 minutes. That's seems high for most\r\n # people so turn it down to once an hour. Set it to zero\r\n # if you don't want the functionality at all.\r\n mark_freq(3600); \r\n};\r\n\r\nsource src {\r\n system();\r\n internal();\r\n};\r\n\r\ndestination messages { file(\"\/var\/log\/messages\"); };\r\n\r\n# By default messages are logged to tty12...\r\ndestination console_all { file(\"\/dev\/tty12\"); };\r\n# ...if you intend to use \/dev\/console for programs like xconsole\r\n# you can comment out the destination line above that references \/dev\/tty12\r\n# and uncomment the line below.\r\n#destination console_all { file(\"\/dev\/console\"); };\r\n\r\n# iptables log\r\ndestination firewall { file(\"\/var\/log\/firewall.log\"); };\r\nfilter f_firewall { program(\"iptables\") or match(\"Dropped\" value(MESSAGE)); };\r\nlog { source(src); filter(f_firewall); destination(firewall); flags(final); };\r\n\r\n# ssh log\r\ndestination sshd { file(\"\/var\/log\/sshd.log\"); };\r\nfilter f_sshd { program(\"^sshd$\"); }; \r\nlog { source(src); filter(f_sshd); destination(sshd); flags(final); };\r\n\r\n# named log\r\ndestination named { file(\"\/var\/log\/named.log\" owner(named) group(named) perm(0600) dir_perm(0700)); };\r\nfilter f_named { program(\"^named$\"); }; \r\nlog { source(src); filter(f_named); destination(named); flags(final); };\r\n\r\n# dhcp log\r\ndestination dhcpd { file(\"\/var\/log\/dhcpd.log\"); };\r\nfilter f_dhcpd { program(\"^dhcpd$\"); }; \r\nlog { source(src); filter(f_dhcpd); destination(dhcpd); flags(final); };\r\n\r\n# spamd log\r\ndestination spamd { file(\"\/var\/log\/spamd.log\"); };\r\nfilter f_spamd { program(\"^spamassassin$\") or program(\"^\/usr\/sbin\/spamd$\"); }; \r\nlog { source(src); filter(f_spamd); destination(spamd); flags(final); };\r\n\r\n# ALWAYS AT THE END\r\nlog { source(src); destination(messages); };\r\nlog { source(src); destination(console_all); };<\/pre>\n","protected":false},"excerpt":{"rendered":"
Systemd although very good, has caused problems for logging (for me). \u00a0Systemd does not play nice with logging applications such as Splunk. \u00a0The reason for this is that systemd takes over syslog and stores all log data in its journald system which uses tmpfs (RAM) until flushed to disk in its proprietary \u00a0format. \u00a0This is … Continue reading Systemd & Syslog-ng<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[18,287,173],"class_list":["post-1653","post","type-post","status-publish","format-standard","hentry","category-gentoo","tag-linux","tag-syslog","tag-systemd"],"_links":{"self":[{"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/posts\/1653","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1653"}],"version-history":[{"count":15,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/posts\/1653\/revisions"}],"predecessor-version":[{"id":1673,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/posts\/1653\/revisions\/1673"}],"wp:attachment":[{"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1653"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1653"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1653"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}