简述
rsyslog 处理多服务器日志分散不容易管理，可以将多台服务器日志发送到指定的日志服务器上分析。

image.png

1. 准备环境

2台liunx机器,并且开通514端口
- 3.17.190.209 Server
- 3.21.75.99 Client
tomcat
- apache-tomcat-8.5.60

2.安装

1. 查看rsyslog 版本

   • 命令：rsyslogd -v
     rsyslog-version.png

注意版本不同配置格式也不同,但是高版本上面也可以使用低版本的配置
可以使用下面的命令升级成最新的
cd /etc/yum.repos.d
wget http://rpms.adiscon.com/v8-stable/rsyslog.repo
yum update rsyslog

2. 安装wget
   • 下载wget：yum install wget

wget 是liunx下载文件工具

3. 安装jdk

   • 下载jdk：wget https://download.oracle.com/otn/java/jdk/8u271-b09/61ae65e088624f5aaa0b1d2d801acb16/jdk-8u271-linux-x64.tar.gz
   • 配置jdk环境
     • vi /etc/profile
       export JAVA_HOME="/home/user/jdk1.8.0_271"
       export CLASSPATH=".:/home/user/jdk1.8.0_271/lib:/home/ec2-user/jdk1.8.0_271/jre/lib"
       pathmunge /home/user/jdk1.8.0_271/bin:/home/user/jdk1.8.0_271/jre/bin
     • source /etc/profile

     • java -version
       jdk.png

4. 安装tomcat

   • 下载tomcat： wget https://mirrors.bfsu.edu.cn/apache/tomcat/tomcat-8/v8.5.60/bin/apache-tomcat-8.5.60.tar.gz

   • 启动tomcat:
     tomcat.png

3. Client 配置Rsyslog

1. 配置rsyslog.conf

   • vi /etc/rsyslog.conf
     默认的路径：etc/rsyslog.conf

   #### MODULES ###
   module(load="imuxsock" SysSock.Use="off")             #imuxsock模块，支持本地系统日志的模块
   module(load="imjournal" StateFile="imjournal.state")  #imjournal模块，支持对系统日志的访问（此模块与上一模块默认启用
   module(load="imfile" PollingInterval="1")                #imfile模块，支持对文件进行操作
   module(load="imklog")                #imklog模块，支持内核日志的模块
   module(load="immark")                #immark模块，支持日志标记
   
   #提供远程rsyslog日志的udp协议的接收支持
   module(load="imudp")               #imudp模块，用于支持udp协议
   input(type="imudp" port="514")     #允许通过514端口接收使用udp协议的远程日志
   
   #提供远程rsyslog日志的tcp协议的接收支持
   module(load="imtcp")             #imtcp模块，用于支持tcp协议
   input(type="imtcp" port="514")   #允许通过514端口接收使用tcp协议的远程日志
   
   #### GLOBAL DIRECTIVES ####
   global(workDirectory="/var/lib/rsyslog")    #工作目录
   module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")    #定义日志格式默认模板（可以自行设定，参看template部分）
   include(file="/etc/rsyslog.d/*.conf" mode="optional")     #需要引入的自定义配置文件的路径
   
   #### RULES ####
   #内核消息，默认不启用
   #kern.*                                                 /dev/console
   
   # 记录所有日志类型的，信息等级大于等于info级别的信息到messages文件（mail邮件信息，authpriv验证信息和corn时间和任务信息除外）
   *.info;mail.none;authpriv.none;cron.none                /var/log/messages
   
   # authpriv验证相关的所有信息存放在/var/log/secure
   authpriv.*                                              /var/log/secure
   
   # 邮件的所有信息存在/var/log/maillog；这里有一个“-”符号表示是使用异步的方式记录
   mail.*                                                  -/var/log/maillog
   
   # 任务计划有关的信息存放在/var/log/cron
   cron.*                                                  /var/log/cron
   
   # 记录所有的≥emerg级别信息，发送给每个登录到系统的日志
   *.emerg                                                 :omusrmsg:*
   
   # 记录uucp，news.crit等
   uucp,news.crit                                          /var/log/spooler
   
   # 本地服务器的启动的所有日志
   local7.*                                                /var/log/boot.log
   
   #日志发送的配置，@表示传输协议（@表示udp，@@表示tcp），后面是ip和端口，格式可配置
   #*.* @@3.17.190.209:514
   
   # 定制规则
   ruleset(name="remote"){
       action(type="omfwd"               #omfwd输出远程模式
          target="3.17.190.209"          #远程服务器的ip地址
          port="514"                     #端口
          protocol="tcp"                 #使用协议
          queue.type="linkedList"        #使用异步处理 
          queue.spoolDirectory="/var/log/rsyslog"       #队列目录
          queue.fileName="remoteQueue_3_17_190_209"     #队列名称
          queue.maxDiskSpace="5g"                       #队列占最大磁盘空间
          queue.saveOnShutdown="on"                     #保存内存数据如果rsyslog关闭
          action.resumeRetryCount="-1"                  #无限重试插入失败
          )
    stop
   }
   
   ruleset(name="MyRuleSet") {
    action(type="omfile"        #输出文件模式
      File="/var/log/test.out")  #输出文件的位置
    stop
   }
   
   

2. 配置tomcat-log.conf

   • vi /etc/rsyslog.d/tomcat-log.conf
     如果没有就创建一个新的文件

    #工作目录
    $WorkDirectory /var/log/rsyslog
   
    #输入文件模式
    input(type="imfile"
         File="/var/log/apache-tomcat-8.5.60/logs/catalina.out"    #文件的位置
         Tag="tomcat_catalina"          #文件标识，服务器接收到消息可以使用这个tag分类
         Severity="info"                #日志级别
         Facility="local0"             
         PersistStateInterval="1"    #回写偏移量数据到文件间隔时间(秒)
         ruleset="remote")          #规则集，rsyslog.conf中定义的rule名称
    
    #测试从本机的test.in 写到本地的test.out文件
    input(type="imfile"
         File="/var/log/test.in"
         Tag="test1"
         Severity="info"
         Facility="local0"
         ruleset="MyRuleSet")
   
   

3. 保存文件，并且检查配置文件是否正确

   • rsyslogd -N 1
     检查文件.png

4. 重启rsyslog

   • systemctl restart rsyslog.service

5. 查看启动日志

   • tail -f /var/log/messages
     

     要认真查看确保没有错误
     rsyslog启动日志.png

4. Server 配置Rsyslog

1. 配置rsyslog.conf
   • vi /etc/rsyslog.conf
     默认的路径：etc/rsyslog.conf

   $FileGroup root         #文件所属组
   $FileOwner root        #文件的拥有者
   $FileCreateMode 0777    #生成文件权限
   $DirCreateMode 0777     #生成文件目录权限
   $Umask 0022
   $PrivDropToUser root       #可以删除日志的用户
   $PrivDropToGroup root    #可以删除日志的用户组
   
   #### MODULES ###
   module(load="imuxsock" SysSock.Use="off")             #imuxsock模块，支持本地系统日志的模块
   module(load="imjournal" StateFile="imjournal.state")  #imjournal模块，支持对系统日志的访问（此模块与上一模块默认启用
   module(load="imfile" PollingInterval="1")                #imfile模块，支持对文件进行操作
   module(load="imklog")                #imklog模块，支持内核日志的模块
   module(load="immark")                #immark模块，支持日志标记
     
   #提供远程rsyslog日志的udp协议的接收支持
   module(load="imudp")               #imudp模块，用于支持udp协议
   input(type="imudp" port="514")     #允许通过514端口接收使用udp协议的远程日志
   
   #提供远程rsyslog日志的tcp协议的接收支持
   module(load="imtcp")             #imtcp模块，用于支持tcp协议
   input(type="imtcp" port="514")   #允许通过514端口接收使用tcp协议的远程日志
     
   #### GLOBAL DIRECTIVES ####
   global(workDirectory="/var/lib/rsyslog")    #工作目录
   module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")    #定义日志格式默认模板（可以自行设定，参看template部分）
   include(file="/etc/rsyslog.d/*.conf" mode="optional")     #需要引入的自定义配置文件的路径
     
   #### RULES ####
   #内核消息，默认不启用
   #kern.*                                                 /dev/console
   
   # 记录所有日志类型的，信息等级大于等于info级别的信息到messages文件（mail邮件信息，authpriv验证信息和corn时间和任务信息除外）
   *.info;mail.none;authpriv.none;cron.none                /var/log/messages
   
   # authpriv验证相关的所有信息存放在/var/log/secure
   authpriv.*                                              /var/log/secure
    
   # 邮件的所有信息存在/var/log/maillog；这里有一个“-”符号表示是使用异步的方式记录
   mail.*                                                  -/var/log/maillog
     
   # 任务计划有关的信息存放在/var/log/cron
   cron.*                                                  /var/log/cron
   
   # 记录所有的≥emerg级别信息，发送给每个登录到系统的日志
   *.emerg                                                 :omusrmsg:*
   
   # 记录uucp，news.crit等
   uucp,news.crit                                          /var/log/spooler
   
   # 本地服务器的启动的所有日志
   local7.*                                                /var/log/boot.log
   #生成模板
   $template SpiceTmpl,"%msg:2:$%\n"
   $template CatalinaDynaFile,"/var/log/rsyslog/%fromhost-ip%/catalina_%$YEAR%-%$MONTH%-%$DAY%.log"
     
   #匹配规则，文章后面将分享其他配置类型
   :fromhost-ip,contains,"3.21.75.99" ?CatalinaDynaFile;SpiceTmpl
   
   

2. 保存文件，并且重新启动Rsyslog
   • systemctl restart rsyslog.conf
3. 查看启动日志
   • tail -f /var/log/message
4. 查看514端口是否开放
   如果没有开放,请关闭514防火墙
   http://tool.chinaz.com/port/
   scanport.png

5. 测试写入local文件

1. 读取client 文件/var/log/test.in, 写入client /var/log/test.out 文件
   这个规则在上面的client 配置里面已经配置了输入文件类型，位置和输出文件类型，位置。

2. 查看实时日志test.out

   • tail -f /var/log/test.out
     test.out.png

3. 客户端重新开一个窗口，添加日志到test.in

• echo yanglibo date: date +%Y-%m-%d:%H:%M:%S >> /var/log/test.in

4. 测试结果，test.out 应该同时接收到文件
   测试结果.png

6. 测试写入远程server

1. 读取tomat 日志文件 /var/log/apache-tomcat-8.5.60/logs/catalina.out, 写入server 3.17.190.209 /var/log/rsyslog/3.17.190.209/catalina-2020-12-03.out
   这个输出路径是上面server rsyslog.conf 里面 template，filter 已经配置好的可以查看上面的配置
2. 启动tomcat
   • ./var/log/apache-tomcat-8.5.60/bin/startup.sh
3. 查看服务器端生成的文件
   • tail -f /var/log/rsyslog/3.17.190.209/catalina-2020-12-03.out

4. 结果
   start tomcat.png

• echo lbyang date: date +%Y-%m-%d:%H:%M:%S >> /var/log/apache-tomcat-8.5.60/logs/catalina.out
  
  手动写入文件.png

附录

1. 登录服务器AWS问题
我使用的是AWS使用.pem 证书的方式登录，用户是来宾用户。很多权限都没有，切换到root 加自定义密码方式

• sudo passwd root #回车然后设置密码
• su root #切换用户
• vi /etc/ssh/sshd_config #打开文件，修改下面的内容为yes
  passwordAuthentication yes
• systemctl restart sshd.service #重启服务

2. 目录权限问题
没有办法自定义目录。比如/home/lb/tomcat/logs/Catalina.out,没有办法读取启动rsyslog报错，没有权限
如果把tomat 安装到 /var/log/下面就可以读取

3. 参考的网址

• https://blog.csdn.net/weixin_33896726/article/details/92900499?utm_medium=distribute.pc_relevant_t0.none-task-blog-searchFromBaidu-1.control&depth_1-utm_source=distribute.pc_relevant_t0.none-task-blog-searchFromBaidu-1.control
• https://www.cnblogs.com/paul8339/p/12463868.html
• https://blog.csdn.net/weixin_41432553/article/details/93098599
• https://blog.csdn.net/weixin_33704234/article/details/89042131?utm_medium=distribute.pc_relevant.none-task-blog-searchFromBaidu-3.control&depth_1-utm_source=distribute.pc_relevant.none-task-blog-searchFromBaidu-3.control
• https://blog.csdn.net/vantler/article/details/104676128

property ( in English)
Message Properties:
These are extracted by rsyslog parsers from the original message. All message properties start with a letter.

The following message properties exist：

msg
the MSG part of the message (aka “the message” ; ))

rawmsg
the message excactly as it was received from the socket. Should be useful for debugging.

hostname
hostname from the message

source
alias for HOSTNAME

fromhost
hostname of the system the message was received from (in a relay chain, this is the system immediately in front of us and not necessarily the original sender). This is a DNS-resolved name, except if that is not possible or DNS resolution has been disabled.

fromhost-ip
The same as fromhost, but alsways as an IP address. Local inputs (like imklog) use 127.0.0.1 in this property.

syslogtag
TAG from the message

programname
the “static” part of the tag, as defined by BSD syslogd. For example, when TAG is “named[12345]”, programname is “named”.

pri
PRI part of the message - undecoded (single value)

pri-text
the PRI part of the message in a textual form with the numerical PRI appended in brackes (e.g. “local0.err<133>”)

iut
the monitorware InfoUnitType - used when talking to a MonitorWare backend (also for Adiscon LogAnalyzer)

syslogfacility
the facility from the message - in numerical form

syslogfacility-text
the facility from the message - in text form

syslogseverity
severity from the message - in numerical form

syslogseverity-text
severity from the message - in text form

syslogpriority
an alias for syslogseverity - included for historical reasons (be careful: it still is the severity, not PRI!)

syslogpriority-text
an alias for syslogseverity-text

timegenerated
timestamp when the message was RECEIVED. Always in high resolution

timereported
timestamp from the message. Resolution depends on what was provided in the message (in most cases, only seconds)

timestamp
alias for timereported

protocol-version
The contents of the PROTCOL-VERSION field from IETF draft draft-ietf-syslog-protcol

structured-data
The contents of the STRUCTURED-DATA field from IETF draft draft-ietf-syslog-protocol

app-name
The contents of the APP-NAME field from IETF draft draft-ietf-syslog-protocol

procid
The contents of the PROCID field from IETF draft draft-ietf-syslog-protocol

msgid
The contents of the MSGID field from IETF draft draft-ietf-syslog-protocol

inputname
The name of the input module that generated the message (e.g. “imuxsock”, “imudp”). Note that not all modules necessarily provide this property. If not provided, it is an empty string. Also note that the input module may provide any value of its liking. Most importantly, it is not necessarily the module input name. Internal sources can also provide inputnames. Currently, “rsyslogd” is defined as inputname for messages internally generated by rsyslogd, for example startup and shutdown and error messages. This property is considered useful when trying to filter messages based on where they originated - e.g. locally generated messages (“rsyslogd”, “imuxsock”, “imklog”) should go to a different place than messages generated somewhere.

System Properties:
These properties are provided by the rsyslog core engine. They are not related to the message. All system properties start with a dollar-sign.
For example, timereported contains the timestamp from the message. Depending on how long the message was in the relay chain, this can be quite old. In contrast, $now is the system time when the message is being processed. Depending on your needs, you need one or the other. Usually, the message-based timestamp is the more important one, but that really depdends on the use case.

The following system properties exist:

$bom
The UTF-8 encoded Unicode byte-order mask (BOM). This may be useful in templates for RFC5424 support, when the character set is know to be Unicode.

$now
The current date stamp in the format YYYY-MM-DD

$year
The current year (4-digit)

$month
The current month (2-digit)

$day
The current day of the month (2-digit)

$hour
The current hour in military (24 hour) time (2-digit)

$hhour
The current half hour we are in. From minute 0 to 29, this is always 0 while from 30 to 59 it is always 1.

HHOUR, but values range from 0 to 3 (for the four quater hours that are in each hour)

$minute
The current minute (2-digit)

$myhostname
The name of the current host as it knows itself (probably useful for filtering in a generic way)