Linux Log Management: A Practical Guide

Table of Contents

Linux Log Management: A Practical Guide

Linux log management involves collecting, analyzing, rotating, and storing system-generated log files. Proper log management helps administrators monitor system health, troubleshoot problems, detect security incidents, and maintain operational stability.

Most Linux systems generate large volumes of logs from the kernel, services, applications, and security subsystems. Understanding how these logs are produced and managed is essential for effective system administration.

🧾 Overview of Linux Logging Services
#

Modern Linux systems typically rely on two main logging components.

rsyslog
#

rsyslog is the traditional and widely used logging daemon. It collects messages from the kernel and user-space applications and routes them to various destinations such as:

Local log files
Remote log servers
Databases
Message queues

It provides a highly flexible rule-based configuration system that allows administrators to filter, redirect, and format log messages.

systemd Journal
#

Systems that use systemd also include the systemd journal, which stores logs in a structured binary format.

Key features include:

Rich metadata (process IDs, timestamps, service units)
Indexed logs for fast searching
Built-in filtering using journalctl
Integration with systemd services

Logs from the journal can also be forwarded to rsyslog for traditional file-based storage.

📂 Common Linux Log Files
#

Most traditional log files are stored under the /var/log directory. Each file typically corresponds to a specific subsystem or service.

Log File	Description
`/var/log/messages`	General system activity including kernel and service messages.
`/var/log/secure`	Authentication and security events such as SSH logins and `sudo` usage.
`/var/log/maillog`	Logs from mail servers like Postfix or Sendmail.
`/var/log/cron`	Activity related to scheduled tasks executed by cron.
`/var/log/dmesg`	Kernel messages generated during system boot.
`/var/log/lastlog`	Records the most recent login of each user.
`/var/log/wtmp`	Historical record of user logins, logouts, and system reboots.
`/var/log/btmp`	Tracks failed login attempts.

Several of these files use binary formats and require special commands for viewing.

⚙️ Understanding rsyslog Configuration
#

The primary configuration file for rsyslog is:

/etc/rsyslog.conf

Additional configuration snippets are often located in:

/etc/rsyslog.d/

The configuration follows a simple rule structure:

selector (facility.priority)    action (destination)

This defines which messages are captured and where they are sent.

Facilities (Log Sources)
#

Facilities identify the subsystem that generated the message.

Common facilities include:

kern — kernel messages
authpriv — authentication and authorization events
mail — mail system logs
cron — scheduled job logs
daemon — background services
local0–local7 — custom application logging

Priorities (Severity Levels)
#

Severity levels indicate the importance of a log message.

From highest to lowest priority:

EMERG — system is unusable
ALERT — immediate action required
CRIT — critical condition
ERR — error condition
WARNING — warning event
NOTICE — significant but normal condition
INFO — informational message
DEBUG — debugging information

When a severity level is selected, it typically includes all higher-priority messages as well.

Example rule:

authpriv.*     /var/log/secure

This sends all authentication-related logs to /var/log/secure.

🔄 Log Rotation and Disk Management
#

Log files grow continuously and can eventually consume large amounts of disk space. Linux systems prevent this using the logrotate utility.

logrotate periodically:

Rotates log files
Compresses older logs
Removes outdated logs
Creates new log files

The main configuration file is:

/etc/logrotate.conf

Service-specific rules are often stored in:

/etc/logrotate.d/

Example Logrotate Configuration
#

/var/log/httpd/* {
    daily
    rotate 30
    compress
    delaycompress
    missingok
}

Explanation:

daily — rotate logs every day
rotate 30 — keep 30 archived log files
compress — compress old logs to save disk space
delaycompress — compress logs starting from the second rotation
missingok — continue even if the log file is missing

🔎 Essential Log Analysis Tools
#

Linux provides several command-line tools for reading and analyzing logs efficiently.

Command	Purpose
`tail -f /var/log/messages`	Monitor logs in real time.
`grep "error" /var/log/syslog`	Search for specific keywords in log files.
`journalctl -u nginx`	View logs generated by a specific service.
`lastlog`	Display the last login time for each user.
`dmesg \| less`	View kernel messages interactively.

Additional useful journalctl commands include:

journalctl -xe

Displays detailed logs for recent system errors.

journalctl --since "1 hour ago"

Shows logs generated during the last hour.

🧩 Anatomy of a Linux Log Entry
#

A typical log entry stored in /var/log/messages looks like this:

Sep 15 09:03:59 my-server systemd-logind[1234]: New session 314 of user root.

Each field provides specific information.

Field	Description
Timestamp	`Sep 15 09:03:59` indicates when the event occurred.
Hostname	`my-server` identifies the system generating the log.
Application	`systemd-logind` is the service that produced the message.
Process ID	`[1234]` identifies the running process instance.
Message	Describes the event that occurred.