How to disable rsyslog rate-limiting

Redhat/CentOS 6 and other Linux distributions use rsyslog instead of syslog. In rsyslog v.5, rate-limiting of the log messages is enabled by default. Additionally, systemd-journald imposes rate limiting on logging. This causes problems for Aspera because we need all the logging information in analyzing performance and problems. Turn off rate limiting on logging to ensure that all log messages are recorded:

  1. Open /etc/systemd/journald.conf in a text editor.
  2. Locate the following lines: 
    #RateLimitInterval=30s
    #RateLimitBurst=1000
    
    Change them to the following:
    RateLimitInterval=0
    RateLimitBurst=0
  3. Save your changes and close the file.
  4. Restart systemd-journald:
    $ systemctl restart systemd-journald
  5. Open /etc/rsyslog.conf in a text editor.
  6. Locate the Global Directives section and the following lines under it:
    # Disable rate-limiting of log entries
    $SystemLogRateLimitInterval 0
    $SystemLogRateLimitBurst 0
  7. Save your changes and close the file.
  8. Restart rsyslogd: 
    service rsyslog restart (/etc/init.d/rsyslog restart)

Make sure you are logging asynchronously, as described in this Knowledge Base article.

2 Comments

  • Avatar
    Geoff Ethridge

     

     

     

    We are on RHEL 6 is the above syntax correct? see errors below:

    Apr 14 16:46:52 fpx3 kernel: Kernel logging (proc) stopped.

    Apr 14 16:46:52 fpx3 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="21820" x-info="http://www.rsyslog.com"] exiting on signal 15.

    Apr 14 16:46:53 fpx3 kernel: imklog 5.8.10, log source = /proc/kmsg started.

    Apr 14 16:46:53 fpx3 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="32238" x-info="http://www.rsyslog.com"] start

    Apr 14 16:46:53 fpx3 rsyslogd-3003: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ]

    Apr 14 16:46:53 fpx3 rsyslogd: the last error occured in /etc/rsyslog.conf, line 8:"$SystemLogRateLimitInterval 0"

    Apr 14 16:46:53 fpx3 rsyslogd-3003: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ]

    Apr 14 16:46:53 fpx3 rsyslogd: the last error occured in /etc/rsyslog.conf, line 9:"$SystemLogRateLimitBurst 0"

    Apr 14 16:46:53 fpx3 rsyslogd-2124: CONFIG ERROR: could not interpret master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

    Apr 14 16:47:01 fpx3 asperacentral[29321]: LOG eb/pe/he/fc=1000/0/20/0 rs/rf=22/1434 ql/qt=20/783

     

    [root@fpx3 ~]# cat /etc/rsyslog.conf

    rsyslog v5 configuration file

    For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html

    If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

    Disable rate-limiting of log entries

    $SystemLogRateLimitInterval 0

    $SystemLogRateLimitBurst 0

    MODULES

    $ModLoad imuxsock # provides support for local system logging (e.g. via logger command)

    $ModLoad imklog # provides kernel logging support (previously done by rklogd)

    #$ModLoad immark # provides --MARK-- message capability

    Provides UDP syslog reception

    #$ModLoad imudp

    #$UDPServerRun 514

    Provides TCP syslog reception

    #$ModLoad imtcp

    #$InputTCPServerRun 514

    GLOBAL DIRECTIVES

    Use default timestamp format

    $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

    File syncing capability is disabled by default. This feature is usually not required,

    not useful and an extreme performance hit

    #$ActionFileEnableSync on

    Include all config files in /etc/rsyslog.d/

    $IncludeConfig /etc/rsyslog.d/*.conf

    RULES

    Log all kernel messages to the console.

    Logging much else clutters up the screen.

    #kern.* /dev/console

    Log anything (except mail) of level info or higher.

    Don't log private authentication messages!

    *.info;mail.none;authpriv.none;cron.none /var/log/messages

    The authpriv file has restricted access.

    authpriv.* /var/log/secure

    Log all the mail messages in one place.

    mail.* -/var/log/maillog

    Log cron stuff

    cron.* /var/log/cron

    Everybody gets emergency messages

    *.emerg *

    Save news errors of level crit and higher in a special file.

    uucp,news.crit /var/log/spooler

    Save boot messages also to boot.log

    local7.* /var/log/boot.log

    local2.* /var/log/aspera.log

    ### begin forwarding rule ###

    The statement between the begin ... end define a SINGLE forwarding

    rule. They belong together, do NOT split them. If you create multiple

    forwarding rules, duplicate the whole block!

    Remote Logging (we use TCP for reliable delivery)

    An on-disk queue is created for this action. If the remote host is

    down, messages are spooled to disk and sent when it is up again.

    #$WorkDirectory /var/lib/rsyslog # where to place spool files

    #$ActionQueueFileName fwdRule1 # unique name prefix for spool files

    #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)

    #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown

    #$ActionQueueType LinkedList # run asynchronously

    #$ActionResumeRetryCount -1 # infinite retries if host is down

    remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional

    #*.* @@remote-host:514

    ### end of the forwarding rule ###

    [root@fpx3 ~]#

  • Avatar
    Geoff Ethridge

    This may have worked:

     

    Now we need to search the right spot for the entries. Find the following:

    $ModLoad imuxsock.so

    This entry will load the imuxsock module.

    Now insert two new lines under the ModLoad command and fill them as follows:

    $SystemLogRateLimitInterval 0
    $SystemLogRateLimitBurst 0

Article is closed for comments.
Powered by Zendesk