MONITOR SYSLOG-NG WITH SPLUNK! [BETA]

Shoutout to Automine, l00py & the CAST Team.

Still under construction.

Syslog-NG config for stats collection

#global
stats_freq(300);
stats_level(2);

#source
};

source s_local {
internal(); # syslog-ng internal logs

};

#destination
destination d_logstats { file(“/var/log/sng_logs/$HOST/logstats.log” perm(0775) create_dirs(yes));};

#log
log { source(s_local); filter (f_logstats); destination(d_logstats); };

Output of Syslog-NG stats collection

2016-07-29T15:18:23+00:00 n00b-splkufwd-03 syslog-ng[9673]: Log statistics; processed=’src.host(n00b-splkufwd-03)=1204′, stamp=’src.host(n00b-splkufwd-03)=1469805421′, processed=’src.host(n00bfirewall)=229′, stamp=’src.host(n00bfirewall)=1469805499′, processed=’center(received)=19′, processed=’src.internal(s_local#0)=19′, stamp=’src.internal(s_local#0)=1469805203′, dropped=’dst.file(d_logstats#0,/var/log/sng_logs/n00b-splkufwd-03/logstats.log)=0′, processed=’dst.file(d_logstats#0,/var/log/sng_logs/n00b-splkufwd-03/logstats.log)=17′, stored=’dst.file(d_logstats#0,/var/log/sng_logs/n00b-splkufwd-03/logstats.log)=0′, dropped=’dst.file(d_catchall#0,/var/log/sng_catchall/n00bfirewall/n00bfirewall.log)=0′, processed=’dst.file(d_catchall#0,/var/log/sng_catchall/n00bfirewall/n00bfirewall.log)=229′, stored=’dst.file(d_catchall#0,/var/log/sng_catchall/n00bfirewall/n00bfirewall.log)=0′, processed=’destination(d_logstats)=17′, processed=’source(s_catchall)=0′, processed=’center(queued)=1450′, processed=’src.none()=0′, stamp=’src.none()=0′, dropped=’dst.file(d_file#0,/var/log/messages)=0′,

inputs.conf

monitor:///var/log/sng_logs/.../logstats.log] disabled = 0
followTail = 0
host_segment = 4
index = n00blab
sourcetype = syslog

transforms.conf

[syslogng_stats] REGEX = \s(?[a-z]+='[^’]+)=(?[^’]+)

props.conf

[syslog] KVMODE = NONE
REPORT-syslogng_stats = syslogng_stats

Search

index=n00blab sourcetype=syslog
| reverse
| streamstats window=2 current=t range(processed_*) as processed_*_delta, range(dropped_*) as dropped_*_delta, range(stored_*) AS stored_*_delta
| timechart span=5m avg(*_delta) as *_delta

Screen Shot 2016-07-29 at 5.25.37 PM

 

 

How To Set Up Raspberry Pi Without Monitor (DHCP, ya you know me!)


In this article we will take the SD card containing Raspbian Wheezy that we created in my last post, and we are going to boot the Pi blind, aka set up Raspberry Pi without monitor.

If you plan to use the Pi as a server rather than a desktop environment, you do your work from the command line, or you just want to save money on accessories, we can completely remove any need for a monitor, keyboard, mouse, etc by taking advantage of DHCP.

Continue reading How To Set Up Raspberry Pi Without Monitor (DHCP, ya you know me!)

How To Create Raspberry Pi SD card via Linux Command Line

This tutorial will walk you through how to create Raspberry Pi SD card via Linux command line (Ubuntu 12.04 LTS)

Instructions for creating the SD card on Windows or Mac are found here

Download desired image via torrent or direct download and place in your Downloads directory:

http://www.raspberrypi.org/downloads

Continue reading How To Create Raspberry Pi SD card via Linux Command Line

Cacti Pi – Optimized

After successfully installing Cacti on my RPi and letting it poll for a few cycles with the default local host device setup, it was apparent that it was a little sluggish. Nothing terribly bad, but obvious when compared to my other deployments, which is understandable considering RPi’s specs.

At first I considered perhaps I had made a mistake not going with lighttpd over apache2, however I quickly found another optimization option….

http://www.penguintutor.com/linux/raspberrypi-webserver

This tutorial includes some interesting info on reclaiming some RAM reserved for graphics processing…which RPi as a server really doesn’t require.

Continue reading Cacti Pi – Optimized

Raspberry Pi setup

My Raspberry Pi setup (so far)

2 Raspberry Pi set up as headless servers in beautiful Built-To-Spec cases.

RPi 1
16GB Sandisk SD
Raspbian Wheezy 2012-07-15
LAMP
Serving Cacti which is monitoring performance of both Pi

Rpi 2
16GB Kingston SD
Raspbian Wheezy 2012-08-16
Freshly installed and awaiting lighttpd install to compare against apache2

Future Projects:
Syslog server
Tftp server
Wordpress site host

Many others…

 

 

Cacti Pi – Correcting Cacti Timezone

Correcting Cacti Timezone

Battled a super annoying issue with what appeared to be Cacti not displaying graphs, which after half a day of messing around ended up being a timezone setting…..grrrrrrrrrrrr

After discovering Cacti was indeed graphing my expected values, just 4 hours ahead of my local time (EDT, America/Toronto) I realized cacti.log was showing UTC timestamps.

After confirming my Linux system time:

pi@raspberrypi ~ $ date
Fri Aug 17 04:39:13 EDT 2012

I dug around on cacti.net and found instructions on updating php.ini with timezone values:

http://docs.cacti.net/faq#time_zone_warning

Continue reading Cacti Pi – Correcting Cacti Timezone