rsyslog-kafkagelf-gateway

Example for kafka gelf gateway with rsyslog

graylog can poll kafka topics in gelf format whixh are generated by rsyslog.

Prequisits

a not to old linux distibution ( in this case CentOS 7 )

Install packages

In distributions rsyslog ist default. You find documentation on http://www.rsyslog.com

yum install rsyslog
yum install rsyslog-mmjsonparse
yum install rsyslog-kafka

rsyslog configuration

The template defines an json document. You can multiple this template with different "name" and "_type" to assign it to different topic.

/etc/rsyslog.conf example ( not tested )

$ModLoad imuxsock
$ModLoad imklog

$ModLoad omkafka

template(name="rsyslog" type="list") {
    constant(value="{\"version\":\"1.1\",")
    constant(value="\"host\":\"")
    property(name="hostname")
    constant(value="\",\"short_message\":\"")
    property(name="msg" format="json")
    constant(value="\",\"timestamp\": ")
    property(name="timegenerated" dateformat="unixtimestamp")
    constant(value=" ,\"_serverity\":\"")
    property(name="syslogseverity")
    constant(value="\",\"_serverity-text\":\"")
    property(name="syslogseverity-text")
    constant(value="\",\"_facility-text\":\"")
    property(name="syslogfacility-text")
    constant(value="\",\"_programname\":\"")
    property(name="programname")
    constant(value="\",\"_procid\":\"")
    property(name="procid")
    constant(value="\",\"_msgid\":\"")
    property(name="msgid")
    constant(value="\",\"_type\":\"rsyslog\"")
    constant(value="}")
}


action( type="omkafka" topic="rsyslog" broker="localhost:9092" template="rsyslog" )