Audit and Logging on Ubuntu Server

09. Audit and Logging on Ubuntu Server: A Comprehensive Guide

by

Introduction: Why Audit and Logging Matter

Audit and Logging on Ubuntu Server are essential for maintaining the security, reliability, and accountability of your Ubuntu server. They provide insights into system activities, help detect anomalies, and are critical for compliance with regulatory requirements such as GDPR, HIPAA, or PCI-DSS.

However, many systems do not enable auditing due to concerns about disk space consumption or lack of tools to analyze the data effectively. In this article, we’ll address these concerns and guide you through implementing auditing and logging in a practical and scalable way.

Topics covered include:

  • Real-world usage and challenges of auditing.
  • Using auditd to track and record system events.
  • Disk space planning for audit logs.
  • Tools to analyze audit logs and system logs.
  • Configuring log rotation with logrotate.
  • Advanced monitoring and auditing practices.

Previous Article:

01. Introduction to Ubuntu Server – SysOSX: AI & Cloud

02. How to Setup Your First Ubuntu Server: A Beginner’s Guide – SysOSX: AI & Cloud

03.Mastering the Linux Command Line for Ubuntu Server – SysOSX: AI & Cloud

04. Managing Users and Permissions on Ubuntu Server: A Comprehensive Guide – SysOSX: AI & Cloud

05. Networking Basics for Ubuntu Server: A Comprehensive Guide – SysOSX: AI & Cloud

06. Installing and Managing Software on Ubuntu Server: A Complete Guide – SysOSX: AI & Cloud

07.Patching and Updating Ubuntu Server: A Comprehensive Guide – SysOSX: AI & Cloud

08. Securing Your Ubuntu Server: Practical Steps for Hardening and Protection – SysOSX: AI & Cloud

Table of Contents

1. Real-World Usage of System Auditing

Why Many Systems Skip Auditing

While auditing is powerful, many administrators avoid enabling it due to the following reasons:

  • Disk Space Consumption: Audit logs can grow rapidly, especially on busy systems, consuming significant disk space.
  • Performance Overhead: Enabling detailed auditing can add slight overhead to system performance.
  • Complexity: Audit logs are verbose and require specialized tools for analysis.

How to Plan for Auditing

Disk Space Planning

To effectively plan for auditing:

  1. Estimate Log Growth: Monitor log growth rates by enabling auditing temporarily and measuring the size of logs over a week. sudo du -sh /var/log/audit/ Use this data to calculate daily, weekly, and monthly storage requirements.
  2. Allocate Dedicated Storage: Store audit logs on a separate partition or disk to prevent logs from filling up the root filesystem. For example:
    • Create a partition for logs: /var/log/audit.
    • Use tools like lvcreate (Logical Volume Manager) to dynamically expand storage if needed.
  3. Enable Compression: Use logrotate to compress old audit logs, significantly reducing storage requirements.
  4. Retention Policies: Define retention policies based on your organization’s needs. For example:
    • Keep logs for 30 days for troubleshooting.
    • Archive older logs to external storage for compliance purposes.

Performance Considerations

Auditd’s performance impact is minimal for most systems. However, for high-traffic servers (e.g., web servers or database servers), monitor CPU and I/O usage after enabling auditing. If necessary:

  • Limit the scope of audit rules to critical files or directories instead of monitoring the entire system.
  • Use sampling techniques to reduce the frequency of logs.

2. Using auditd to Track and Record System Events

The auditd tool remains one of the most robust solutions for system auditing. Below are additional considerations for real-world usage.

Advanced Audit Rules

Instead of auditing every file, focus on critical areas:

  • System Configuration Files:
 sudo auditctl -w /etc/ssh/sshd_config -p wa -k ssh_config_changes
  • Sensitive Directories:
sudo auditctl -w /var/www/html -p rwxa -k web_access
  • User Activity:
    Monitor specific users or groups: 
sudo auditctl -a always,exit -F uid=1001 -F arch=b64 -S execve -k user_commands

Tools to Analyze Audit Logs

Audit logs can be overwhelming, but several tools can help digest and analyze the data:

  1. Aureport: Generates summary reports from audit logs. sudo aureport --summary
  2. Auditbeat: A lightweight tool from Elastic for shipping audit logs to a centralized ELK stack for visualization and analysis.
    • Install Auditbeat: sudo apt install auditbeat
    • Configure it to send logs to Elasticsearch or Logstash.
  3. Splunk: A commercial tool for log analysis with advanced search and reporting capabilities.

3. Disk Space Planning for Audit Logs

Example Disk Space Plan

Let’s assume:

  • Daily audit logs consume 500 MB.
  • Logs are rotated weekly and compressed to 20% of their original size.

Disk Space Calculation:

  • Uncompressed logs for 7 days: 500 MB×7=3.5 GB500MB×7=3.5GB
  • Compressed logs for 30 days: 500 MB×30×0.2=3 GB500MB×30×0.2=3GB
  • Total storage required: 3.5 GB+3 GB=6.5 GB3.5GB+3GB=6.5GB

Allocate at least 10 GB to account for unexpected spikes.

4. Configuring Log Rotation with logrotate

Log rotation is critical for managing the size of logs. Below are advanced configurations for audit logs and system logs.

Rotating Audit Logs

Audit logs are automatically managed by the auditd daemon. To customize rotation:

  • Edit the /etc/audit/auditd.conf file: 
max_log_file = 50   # Maximum size in MB for audit.log
max_log_file_action = rotate  # Rotate logs when size exceeds limit
  • Test the configuration: 
sudo service auditd restart

Rotating Other Logs

For non-audit logs, use logrotate:

  • Example for /var/log/syslog:
/var/log/syslog {
    weekly
    rotate 4
    compress
    missingok
    notifempty
    create 0640 root root
}

5. Advanced Monitoring and Auditing Practices

Real-Time Log Monitoring

Real-time monitoring is essential for detecting critical events. Here are tools and techniques:

  1. Logwatch: Summarizes daily logs and emails them to administrators. sudo apt install logwatch Configure it in /etc/logwatch/conf/logwatch.conf.
  2. Centralized Log Management:
    • Rsyslog: Forward logs to a central server. sudo apt install rsyslog Configure /etc/rsyslog.conf to send logs to a remote server.
    • Graylog: Provides a web-based interface for log analysis.
    • ELK Stack: Elasticsearch, Logstash, and Kibana for scalable log management.

Security-Focused Monitoring

  • Fail2ban: Monitors logs for failed login attempts and bans IPs. 
sudo apt install fail2ban
  • Tripwire: Detects file changes for intrusion detection. 
sudo apt install tripwire

Integrating Alerts

Set up alerts for critical events using tools like:

  • Nagios: Monitors server health and logs for anomalies.
  • Prometheus with Grafana: Tracks metrics and logs in real time.

Log files locations

There are many different log files that all serve different purposes. When trying to find a log about something, you should start by identifying the most relevant file. Below is a list of common log file locations.

System logs

System logs deal with exactly that – the Ubuntu system – as opposed to extra applications added by the user. These logs may contain information about authorizations, system daemons and system messages.

Authorization log

Location: /var/log/auth.log

Keeps track of authorization systems, such as password prompts, the sudo command and remote logins.

Daemon Log

Location: /var/log/daemon.log

Daemons are programs that run in the background, usually without user interaction. For example, display server, SSH sessions, printing services, bluetooth, and more.

Debug log

Location: /var/log/debug

Provides debugging information from the Ubuntu system and applications.

Kernel log

Location: /var/log/kern.log

Logs from the Linux kernel.

System log

Location: /var/log/syslog

Contains more information about your system. If you can’t find anything in the other logs, it’s probably here.

Application logs

Some applications also create logs in /var/log. Below are some examples.

Apache logs

Location: /var/log/apache2/ (subdirectory)

Apache creates several log files in the /var/log/apache2/ subdirectory. The access.log file records all requests made to the server to access files. error.log records all errors thrown by the server.

X11 server logs

Location: /var/log/Xorg.0.log

The X11 server creates a seperate log file for each of your displays. Display numbers start at zero, so your first display (display 0) will log to Xorg.0.log. The next display (display 1) would log to Xorg.1.log, and so on.

Non-human-readable logs

Not all log files are designed to be read by humans. Some were made to be parsed by applications. Below are some of examples.

Login failures log

Location: /var/log/faillog

Contains info about login failures. You can view it with the faillog command.

Last logins log

Location: /var/log/lastlog

Contains info about last logins. You can view it with the lastlog command.

Login records log

Location: /var/log/wtmp

Contains login info used by other utilities to find out who’s logged in. To view currently logged in users, use the who command.

Conclusion

Auditing and logging are essential for maintaining a secure and reliable Ubuntu server. By carefully planning disk space, using tools to analyze logs, and implementing advanced monitoring practices, you can overcome common challenges and gain better visibility into system activities.

Whether you’re managing compliance requirements or proactively securing your server, the techniques and tools discussed in this article will help you build a scalable and effective logging and auditing strategy.

Up Next: System Monitoring Tools for Ubuntu Server

In the next article, we’ll dive deeper into monitoring and performance management. We’ll explore powerful system monitoring tools for Ubuntu Server, including solutions like Nagios, Prometheus, Grafana, and htop, to help you track resource usage, detect bottlenecks, and optimize server performance.

Stay tuned to learn how to keep your server running efficiently and proactively address potential issues before they affect your operations!

3 thoughts on “09. Audit and Logging on Ubuntu Server: A Comprehensive Guide”

  1. Pingback: Ubuntu Server security
  2. Pingback: 13.Setting Up Your Ubuntu Server for Hosting and Web Applications: Ultimate Guide - SysOSX: AI & Cloud
  3. Pingback: 16. Setting Up File Sharing Services on Ubuntu Server: a complete guide - SysOSX: AI & Cloud

Leave a Comment