server administration

Navigating Email Spam Filters: Understanding and Adjusting Sieve Scripts on Plesk Servers

By in Blog Tags: , , , , , , ,

Sieve scripts — a tool used in Plesk servers outfitted with Dovecot and Warden Anti-Spam — which can lead to rules that filter emails in unexpected ways. If you’ve noticed emails mysteriously disappearing or ending up in spam folders when they shouldn’t, Sieve scripts could be the culprit worth investigating.

The Role of Sieve Scripts in Email Management

Sieve scripts offer a robust mechanism for filtering emails on the server side. They empower users to automate actions such as sorting emails into designated folders, setting up auto-replies, and filtering out spam. Implemented on the server, these scripts process emails before they even hit your email client, serving as a critical defence line against unwanted communications.

A Case Study: The Journey to INBOX.Spam

A recent examination revealed an interesting case where an email, marked as clean by the content filter Amavis, was nonetheless directed to the INBOX.Spam folder. The culprit? A specific fileinto action within a Sieve script, instructing Dovecot to categorise the email as spam. This instance highlights the significant impact Sieve scripts can have on email routing.

How to Access and Modify Sieve Scripts in Plesk

Plesk server users can manipulate Sieve scripts through the web interface or by shell access for those requiring a deeper level of customization. Within the Plesk interface, the “Mail” section offers a gateway to adjusting spam filter settings and managing rules at the individual email account level. For more advanced modifications, including direct Sieve script edits, shell access will be necessary. This approach demands a good grasp of Linux commands and familiarity with Dovecot’s structure.

Sieve Script File Paths and Patterns

Sieve scripts are typically located in the user’s mail directory, with paths varying based on the server’s configuration. Common locations include:

  • /var/qmail/mailnames/domain.com/username/.dovecot.sieve
  • /var/qmail/mailnames/domain.com/username/sieve/

When editing Sieve scripts, you might encounter patterns like:

if address :all :comparator "i;ascii-casemap" :is ["From", "Sender", "Resent-From"] ["example@domain.com"] {
    fileinto "INBOX.Spam";
    stop;
}

This rule directs emails from example@domain.com straight to spam, showcasing how specific patterns in Sieve scripts govern email sorting.

Adjusting Filtering Rules

The Sieve script shared earlier delineates various rules for email handling, from whitelisting certain addresses to blacklisting others, and employing spam filters based on the X-Spam-Level header. For instance, it includes conditions to file emails from specific senders into INBOX.Spam, underlining the necessity of precise rule configuration to avoid ensnaring legitimate emails.

Best Practices for Sieve Script Management

  1. Review Rules Periodically: Keep an eye on your Sieve script rules, especially after changes in email communication patterns.
  2. Proceed With Caution: Be careful when modifying Sieve scripts! Small syntax errors can have big consequences for email delivery.
  3. Back Up Before Modifying: Always back up your existing scripts before making adjustments:
    cp /path/to/username/.dovecot.sieve /path/to/username/.dovecot.sieve.backup
  4. Use Comments for Clarity: Add comments to remember why you created certain rules:
    # Rule to handle marketing emails
if header :contains "Subject" "Newsletter" {
    fileinto "INBOX.Marketing";
}
  5. Monitor After Changes: After tweaking rules, keep an eye on your email for a few days to make sure everything’s working as expected.

Troubleshooting Common Issues

If emails seem to go missing, first check your spam folder! It’s amazing how often this simple step resolves the mystery.

For syntax issues after editing, you can verify your script with:

sievec /path/to/script.sieve

If your changes don’t seem to take effect, check that the file ownership matches the mail user and consider restarting Dovecot to reload scripts.

Efficient email management hinges on a fine-tuned approach to spam filtering. By getting comfortable with Sieve scripts on your Plesk server, you can take control over which emails land in your inbox versus spam. Regular reviews and cautious customisation of these scripts will ensure your email system remains effective and secure, tailored to your specific needs.

Expanding the Root Partition on Debian with LVM: A Step-by-Step Guide

By in Blog Tags: , , , , , ,

Expanding your Debian server’s root partition without downtime is a critical skill for managing server resources efficiently, especially when you’re using Logical Volume Manager (LVM) for disk management. This guide provides a detailed walkthrough, including command outputs for clarity and copy-paste commands for ease of use.

Prerequisites

Before beginning the partition expansion process:

  • Ensure you have backed up all critical data
  • Verify you have root access or the ability to use sudo
  • This guide assumes familiarity with basic Linux terminal commands
  • Your system should be using LVM for volume management

Step 1: Verify Current Disk Layout

Identify your disk’s current layout to determine the partition you need to resize. Use lsblk to list all block devices and their mount points:

lsblk

Example output:

NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT 
sda             8:0    0  160G  0 disk 
├─sda1          8:1    0  487M  0 part /boot 
└─sda2          8:2    0   20G  0 part
  ├─vg00-lv01 254:1    0    8G  0 lvm  /
  └─vg00-lv00 254:0    0    2G  0 lvm  [SWAP]

This output shows:

  • A 160GB disk (sda)
  • A boot partition (sda1) of 487MB
  • An LVM partition (sda2) of 20GB, containing:
    • An 8GB root logical volume (vg00-lv01)
    • A 2GB swap logical volume (vg00-lv00)

Step 2: Resize the Partition with fdisk

Caution: Deleting and recreating the partition must be done carefully to avoid data loss.

  1. Launch fdisk on Your Disk: 
    bash
    sudo fdisk /dev/sda
  2. Delete the Existing Partition: First, print the partition table:
    Command (m for help): p
    Example output:
    Disk /dev/sda: 160 GiB, 171798691840 bytes, 335544320 sectors
Disk model: Virtual Disk
Units: sectors of 1 * 512 = 512 bytes
Device     Boot    Start       End   Sectors  Size Id Type
/dev/sda1  *        2048    999423    997376  487M 83 Linux
/dev/sda2         999424  20971519  19972096   20G 8e Linux LVM
    Note the Start sector of /dev/sda2. Delete /dev/sda2:
    Command (m for help): d
Partition number (1,2, default 2): 2
  3. Recreate the Partition: Create a new primary partition:
    Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2): 2
First sector (999424-335544319, default 999424): 999424
Last sector, +/-sectors or +/-size{K,M,G,T,P} (999424-335544319, default 335544319):
    Important: Use the same starting sector as noted earlier (999424 in this example). If prompted about removing the signature, choose No:
    Partition #2 contains a LVM2_member signature.
Do you want to remove the signature? [Y]es/[N]o: N
  4. Set the Partition Type: Change the partition type to Linux LVM:
    Command (m for help): t
Partition number (1,2, default 2): 2
Hex code (type L to list all codes): 8e
  5. Write Changes and Exit:
    Command (m for help): w

Step 3: Reboot the System

After modifying the partition table, reboot your system:

sudo reboot

This step is necessary for the kernel to recognise the new partition table.

Step 4: Resize the Physical Volume

Once the system has rebooted, notify LVM of the partition’s new size:

sudo pvresize /dev/sda2

This command adjusts the physical volume to use the newly expanded partition space.

Step 5: Extend the Logical Volume

Increase the size of the logical volume to use all available space:

sudo lvextend -l +100%FREE /dev/vg00/lv01

This command allocates all available free space in the volume group to the logical volume.

Step 6: Resize the Filesystem

Finally, resize the filesystem on the logical volume to use the newly allocated space.

For ext4 filesystem:

sudo resize2fs /dev/vg00/lv01

For xfs filesystem:

sudo xfs_growfs /dev/vg00/lv01

Verification

After completing all steps, verify that your root partition has been successfully expanded:

df -h /

This will show the new size of your root filesystem.

Troubleshooting Common Issues

Issue: Unable to Delete Partition

If you receive errors when trying to delete the partition, ensure no process is using it:

lsof | grep sda2

Issue: LVM Not Recognising New Size

If pvresize doesn’t recognise the new size, try rescanning the SCSI bus:

echo 1 > /sys/class/block/sda/device/rescan

Issue: Filesystem Not Expanding

If the filesystem doesn’t expand despite successful LV extension, check the filesystem type:

df -T /

And use the appropriate resize command based on the filesystem type.

Conclusion

You’ve now successfully expanded your root partition to utilise additional disk space, enhancing your Debian server’s capacity without significant downtime. Remember, the key to a smooth operation is careful planning and ensuring you have backups before starting.

This technique is particularly useful for virtual machines where disk space can be allocated dynamically, allowing you to adjust storage resources as your requirements grow.

Block WordPress XML-RPC Requests on Linux Servers

By in Blog Tags: , , , , , ,

The XML-RPC Security Challenge

If you run a server hosting multiple websites, you may encounter issues with WordPress installations due to a feature called XML-RPC. This remote admin functionality allows third-party applications to connect to WordPress sites and post new articles or perform other administrative actions.

While XML-RPC is excellent in theory, in practice it has become a significant avenue for attack by bot networks. Server administrators often observe hundreds or thousands of malicious connection attempts per minute across multi-domain hosting environments. On one of our servers hosting 40 sites (nearly all WordPress), these attacks are a constant concern.

Impact on Server Performance

These malicious XML-RPC requests consume valuable server resources even when they fail to compromise a site. With multiple WordPress installations managed by people with varying technical skills, some sites may become vulnerable due to outdated software or poor maintenance practices.

The cumulative impact of these requests can significantly degrade server performance, causing:

  • Increased response times for legitimate visitors
  • Higher CPU and memory usage
  • Potential service disruptions during peak attack periods
  • Premature need for hardware upgrades

Implementing a Solution with mod_security

If you’re running mod_security (which is strongly recommended for protecting your sites and your clients’ sites), you can add the following rule to block malicious XML-RPC requests:

#Block requests to xmlrpc.php with no referring URL
SecRule REQUEST_METHOD "POST" "deny,status:401,id:5000900,chain,msg:'xmlrpc.php request blocked, no referer'"
SecRule &HTTP_REFERER "@eq 0" "chain"
SecRule REQUEST_URI "xmlrpc.php"

Adapted from a post on cPanel forums by ‘dalem’ (https://forums.cpanel.net/threads/xmlrpc-spam.646765/)

How This Solution Works

Once implemented, any request to xmlrpc.php that doesn’t originate from a link or trigger from a legitimate website (i.e., has no referrer) will be blocked with a 401 (Forbidden) response. This effectively stops the vast majority of automated attacks whilst allowing legitimate XML-RPC usage.

Enhanced Protection with Firewalls

For additional protection, if you have a firewall or IP banning software such as CSF Firewall or Fail2ban, you can configure these tools to scan your log files and automatically ban repeat offenders that trigger this mod_security rule.

This layered security approach provides both immediate request blocking and longer-term protection against persistent attackers.

When This Approach May Not Be Suitable

This solution may not be appropriate in all scenarios, particularly:

  • If you or your customers actively use third-party services that utilise the XML-RPC functionality
  • When legitimate applications need programmatic access to WordPress without a referrer

If you have specific IPs that need to access XML-RPC (such as from a content management system or publishing tool), you could modify the rule to exclude those IP addresses. However, this approach may become difficult to maintain over time, especially with services that don’t use fixed IP addresses.

Alternative Approaches

If the solution above doesn’t suit your needs, consider these alternatives:

  1. Selective enabling: Only enable XML-RPC on specific WordPress sites that genuinely need it
  2. Plugin solutions: Use security plugins that provide more granular control over XML-RPC access
  3. IP whitelisting: Configure your web server to only allow XML-RPC requests from trusted IP addresses
  4. Request throttling: Implement rate limiting for XML-RPC endpoints

Conclusion

By implementing this mod_security rule, you can significantly reduce server load and improve security across multiple WordPress installations. The approach is particularly valuable for shared hosting environments where WordPress sites of varying ages and maintenance levels coexist.

Monitor your server logs after implementation to ensure legitimate services aren’t being inadvertently blocked, and adjust your rules as needed to balance security and functionality.

cPanel: How to Combine Apache Logs for Easier Troubleshooting

By in Blog Tags: , , , , , , ,

Default Location of Logs

By default, cPanel places its log files in /usr/local/apache/domlogs/DOMAIN. These logs are split up by domain, which keeps things organised but can make troubleshooting difficult when you need to identify patterns across multiple domains or determine where a specific problem might lie.

The Challenge with Separate Domain Logs

When managing a server with multiple websites, having logs separated by domain creates several challenges:

  1. You need to check multiple files when troubleshooting server-wide issues
  2. It’s difficult to identify patterns that might affect several domains
  3. When investigating suspicious activity, you can’t easily see if it’s targeting multiple sites
  4. Resource usage analysis becomes more complex

Creating Combined Logs

While combining logs can be done with a bash script that concatenates existing files, this approach is time-consuming and doesn’t provide real-time insight. A more efficient solution is to modify the Apache configuration to create a new combined log file that captures all domain activity in a single location.

Edit Your vhost.default File

For Apache 2.4 (the current version at the time of writing), you’ll need to edit the vhost template file:

nano /var/cpanel/templates/apache2_4/vhost.default

Add the following lines just before the </VirtualHost> tag at the bottom of the file:

LogFormat "%V %a %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" vcommon
CustomLog /usr/local/apache/logs/vhost-access_log vcommon

Understanding the Log Format

This configuration creates a custom log format called “vcommon” with the following components:

  • %V – The server name according to the UseCanonicalName setting
  • %a – Client IP address
  • %l – Remote logname (usually ‘-‘)
  • %u – Remote user if authenticated
  • %t – Time the request was received
  • \"%r\" – First line of request (method, path, protocol)
  • %s – Status code
  • %b – Size of response in bytes
  • \"%{Referer}i\" – Referer header
  • \"%{User-agent}i\" – User-agent header

The %V at the beginning is particularly important as it identifies which domain the log entry is associated with, allowing you to still filter by domain when needed while keeping everything in one file.

Rebuilding the Apache Configuration

To implement these changes, you need to rebuild the Apache configuration and restart the service:

/scripts/verify_vhosts_includes
/scripts/rebuildhtpdconf
/scripts/restartsrv_httpd

After completing these steps, a new combined log file will be created at /usr/local/apache/logs/vhost-access_log that contains entries from all your domains.

SSL Considerations

If you want to track SSL accesses in the combined log as well, you’ll need to make a similar modification to the SSL vhost template. However, as the author notes, this may not be necessary if you have a limited number of SSL sites, as they tend to stand out more in normal monitoring.

If you do wish to include SSL logs, edit:

nano /var/cpanel/templates/apache2_4/ssl_vhost.default

And add the same LogFormat and CustomLog lines as above.

Log Rotation Considerations

Remember that creating a new log file requires proper log rotation configuration to prevent it from growing too large. This topic will be covered in a future post, but it’s important to implement log rotation for any new log files you create.

A basic approach would be to add an entry to /etc/logrotate.d/apache to ensure your new combined log is rotated alongside the standard Apache logs.

Benefits of Combined Logs

With this configuration in place, troubleshooting becomes significantly easier:

  1. One file to tail or grep instead of dozens
  2. Ability to see patterns across multiple domains
  3. Easier identification of distributed attacks targeting multiple sites
  4. Simplified log analysis with tools like AWStats or GoAccess

This simple configuration change can save considerable time and effort when managing a multi-domain cPanel server, making it a valuable addition to your server administration toolkit.

How to Enable mod_deflate for All Sites (Entire Server) in cPanel/WHM

By in Blog Tags: , , , , , , , ,

Mod_deflate is essential nowadays for websites – it speeds up the transfer of HTML pages and other files (such as stylesheets) by compressing them first. With today’s powerful servers, there is very little reason not to use this functionality, especially since many “speed up” tests will not pass without it being enabled.

While you can enable mod_deflate site-by-site, for shared hosts with many websites this approach is inefficient. Fortunately, you can enable it globally for all sites on your server.

Step-by-Step Implementation

1. Ensure mod_deflate is Installed

First, make sure the module is installed on your server:

  • Use EasyApache and verify that mod_deflate is selected (it may already be)
  • Once Apache has been compiled, deflate functionality will be enabled

2. Site-by-Site Option

If you want to enable the functionality site-by-site:

  • In each site’s control panel, a new option is now available
  • Under Software/Services, users can choose to optimize their website
  • This essentially allows deflate to be turned on and off per site

3. Global Implementation (Recommended)

To enable mod_deflate for all websites on your server:

  1. In WHM, navigate to: Services Configuration → Apache Configuration → Include Editor → Post Virtual Include
  2. Select All Versions (there’s no reason not to apply it to all Apache versions)
  3. Paste the following code into the file and click Update:
<IfModule mod_deflate.c>
SetOutputFilter DEFLATE
<IfModule mod_setenvif.c>
# Don't compress images as it's generally pointless
SetEnvIfNoCase Request_URI .(?:gif|jpe?g|png)$ no-gzip dont-vary
</IfModule>
<IfModule mod_headers.c>
# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary
</IfModule>
#maximum compression - why not, we have spare CPU
DeflateCompressionLevel 9
</IfModule>

Benefits of Global mod_deflate Implementation

Enabling mod_deflate across your entire server provides several advantages:

  1. Reduced Bandwidth Usage: Compressed files are significantly smaller, reducing overall bandwidth consumption
  2. Faster Page Loading: Smaller file sizes mean quicker transfers and faster loading times for visitors
  3. Improved User Experience: Faster websites mean happier users and higher engagement
  4. Better SEO Performance: Page speed is a ranking factor for search engines
  5. Consistent Configuration: No need to remember to enable it for each new site
  6. Minimal Server Impact: Modern CPUs can handle the compression with negligible performance impact

Technical Details

The configuration above sets the compression level to 9 (maximum), which provides the highest level of compression. While this requires slightly more CPU resources, modern servers generally have plenty of processing power to spare, making the trade-off worthwhile.

The configuration also excludes image files (GIF, JPEG, PNG) from compression since these formats are already compressed, and attempting to compress them further would waste CPU resources with little to no benefit.

The “Vary” header is included to ensure proper handling by proxies and caches, preventing them from serving compressed content to clients that can’t handle it.

By implementing this server-wide setting, you’ll ensure all websites hosted on your server benefit from compression without requiring individual configuration.

Copying Files Between Linux Servers – Using SCP

By in Blog Tags: , , , , , , , ,

For a few years now I keep catching myself transferring files between Linux servers in a silly way…

What I mean is using something like FTP, or copying to my desktop and then up to the other server – all hassle (and in the case of FTP – inherently insecure). Why do I do this? Well not because there’s no alternative, but because I can never easily remember how to do it the “proper” way.

The Proper Way: SCP

So how to do it – simply use SCP. This copies a file using SSH and so is more secure (we trust SSH right?), even better I’ve never had a server which doesn’t support it out of the box.

Basic File Transfer

Right so how to transfer one file:

scp /sourcedir/filename.extension user@serverip:/destinationdir/

It’s quite simple really – if you run SSH on an abnormal port (you really should it cuts down the attacks) then you can specify the port like this:

scp -P 2100 /sourcedir/filename.extension user@serverip:/destinationdir/

Transferring Multiple Files

Also, if you want to transfer multiple files (e.g. all the images in a directory) then you can just use wildcards – like below:

scp -P 2100 /sourcedir/*.extension user@serverip:/destinationdir/

Additional Useful Options

Here are some other helpful SCP options that can make file transfers even more efficient:

  • -r: Recursively copy entire directories
    scp -r /sourcedir/ user@serverip:/destinationdir/
  • -C: Enable compression during the transfer
    scp -C /sourcedir/largefiles.tar user@serverip:/destinationdir/
  • -v: Verbose mode to see detailed transfer information
    scp -v /sourcedir/filename.extension user@serverip:/destinationdir/
  • -q: Quiet mode to suppress progress meter and warning messages

Why SCP is Better

Using SCP provides several advantages over FTP or manual transfers:

  1. Security: All transfers are encrypted through SSH
  2. Simplicity: No need to install additional software
  3. Efficiency: Direct server-to-server transfers save bandwidth and time
  4. Authentication: Uses your existing SSH keys/credentials

So next time you need to move files between Linux servers, skip the FTP client and use SCP directly. Your data will be more secure, and you’ll save valuable time.

Troubleshooting Webmail Connection Issues in cPanel: The CSF Firewall Configuration Solution

By in Blog Tags: , , , , , , , , ,

If you administer cPanel servers, you may eventually encounter a frustrating issue: webmail clients like Horde, Roundcube, or SquirrelMail suddenly stop sending emails, despite mail services appearing to function normally otherwise. This issue can be particularly perplexing because the problem often isn’t immediately apparent from typical mail server logs or standard troubleshooting procedures.

The Symptoms

Recently, I faced this exact problem on one of my managed servers. The symptoms were clear but deceptively simple:

  • Webmail interfaces would load correctly
  • Users could authenticate and view their emails
  • Attempts to send emails through webmail would fail with the error:
SMTP Error: SMTP error: Connection failed: Failed to connect socket: Connection timed out.
  • Mail sent through other methods (desktop clients, server scripts) worked fine
  • Mail server logs showed no obvious issues

Initial Troubleshooting Steps

My first instinct was to look at the SMTP configuration in cPanel. cPanel includes an “SMTP Tweak” option that can help resolve some connection issues. After enabling this option, the system appeared to operate correctly for a few hours, but then the error returned.

Checking Basic Connectivity

A common troubleshooting step for any mail-related issue is to test SMTP connectivity directly. I used the standard telnet approach:

telnet localhost 25

This test worked perfectly, establishing a connection to the mail server. This suggested that the mail server itself was functioning correctly and accepting connections, which made the webmail failure even more mysterious.

The Root Cause: ConfigServer Firewall (CSF) Settings

After more extensive research and forum crawling, I discovered that the issue was related to the ConfigServer Firewall (CSF) settings. CSF is a popular firewall solution for cPanel servers that provides robust security features, but its comprehensive nature means some settings can have unexpected effects on system functionality.

The specific issue was related to a configuration parameter in the CSF settings:

# If SMTP_BLOCK is enabled but you want to allow local connections to port 25
# on the server (e.g. for webmail or web scripts) then enable this option to
# allow outgoing SMTP connections to 127.0.0.1
SMTP_ALLOWLOCAL = 0

When SMTP_ALLOWLOCAL is set to 0, it prevents local connections to the SMTP server on port 25, which directly impacts webmail functionality. The webmail clients attempt to connect to the mail server locally (on 127.0.0.1), and the firewall blocks these connections despite the fact that they’re originating from the same server.

The Solution

The fix for this issue is straightforward once identified:

  1. Edit the CSF configuration file:
    vi /etc/csf/csf.conf
  2. Find the SMTP_ALLOWLOCAL parameter and change it from 0 to 1:
    SMTP_ALLOWLOCAL = 1
  3. Restart CSF to apply the changes:
    csf -r

After making this change, webmail services immediately began working correctly again, successfully sending emails without any timeouts or connection errors.

Why This Issue Occurs

There are several reasons why you might encounter this issue:

  1. CSF Updates: Sometimes CSF updates can reset or modify configuration parameters, including SMTP_ALLOWLOCAL.
  2. Security Hardening: An overzealous security hardening process might have changed this setting without considering the impact on webmail functionality.
  3. Default Settings: Depending on your CSF installation method or version, this setting might be disabled by default.
  4. Manual Changes: It’s possible that during routine security configurations, this setting was inadvertently changed.

In my case, the server had been working correctly for approximately 4 months before this issue appeared, suggesting that a change or update had modified this setting from its previously functional state.

Preventive Measures

To avoid similar issues in the future:

  1. Document Firewall Configurations: Keep detailed documentation of all firewall settings, especially after a successful initial setup.
  2. Test After Updates: Always test critical functionalities like webmail after any system or security updates.
  3. Create Configuration Backups: Before making changes to CSF, create a backup of the configuration file:
    cp /etc/csf/csf.conf /etc/csf/csf.conf.bak
  4. Periodic Functionality Checks: Implement routine checks of key server functions, including webmail send/receive capabilities.

Conclusion

The SMTP_ALLOWLOCAL setting in CSF is a perfect example of how security measures can sometimes inadvertently impact system functionality. When troubleshooting webmail issues in a cPanel environment, it’s worth checking this setting early in your process, especially if you’re seeing timeout errors when attempting to send mail.

Remember, robust security and full functionality don’t have to be mutually exclusive. With proper configuration, you can maintain strong security measures while ensuring all your server’s services operate correctly. This small configuration change allows your webmail clients to communicate with the local mail server while still maintaining protection against external threats.

Linux Disk Space Management: Essential Commands and Monitoring Techniques

By in Blog Tags: , , , , , , , ,

Managing disk space effectively is a critical skill for any Linux system administrator. Unexpected disk space issues can lead to system failures, service outages, and data loss. This guide covers essential commands and techniques for monitoring and managing disk space on Linux servers.

Common Causes of Unexpected Disk Space Consumption

Several factors can lead to unexpected disk space depletion on Linux servers:

  1. Log File Growth: System and application logs can rapidly consume disk space, especially when verbose logging is enabled or when an application is experiencing errors.
  2. Suboptimal Partition Layouts: Rented or managed servers often come with predefined partition schemes that might not align with your specific needs. Some partitions (like /var or /tmp) might be assigned insufficient space.
  3. Temporary Files: Applications may create temporary files that aren’t properly cleaned up.
  4. Database Growth: Databases can expand rapidly, especially when handling high transaction volumes.
  5. Backup Files: Automatic backups or system snapshots may accumulate without proper rotation policies.

Essential Disk Space Monitoring Commands

Basic Disk Usage Reporting with df

The simplest way to check disk space utilization is with the df (disk free) command:

df

This command displays a table showing all mounted filesystems, their sizes, used space, available space, usage percentage, and mount points. The output looks something like this:

Filesystem     1K-blocks      Used Available Use% Mounted on
/dev/sda1       41251136  30950700   8246844  79% /
/dev/sda2      103081248  83059076  14758808  85% /home
tmpfs            2048000         0   2048000   0% /dev/shm

Making Output More Human-Readable

For easier interpretation, especially on systems with large storage capacities, use the -h (human-readable) flag:

df -h

This modifies the output to use more intuitive size units (KB, MB, GB, TB):

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        40G   30G  7.9G  79% /
/dev/sda2        99G   80G   14G  85% /home
tmpfs           2.0G     0  2.0G   0% /dev/shm

Additional Useful Options

  • df -T: Shows filesystem types
  • df -i: Displays inode information instead of block usage
  • df --total: Adds a total row at the bottom
  • df -x tmpfs: Excludes specific filesystem types (useful for focusing on physical drives)

Real-Time Disk Space Monitoring

To observe disk space changes in real-time, particularly useful when troubleshooting rapid space consumption or during cleanup operations, you can use the watch command:

watch -n 1 "df -h"

This will execute the df -h command every second (-n 1 sets the interval), refreshing the output on your terminal. This creates a live dashboard that helps you:

  • Monitor the impact of file operations in real-time
  • Observe log rotation effects
  • Track cleanup operations
  • Identify rapidly growing filesystems

You can adjust the refresh interval based on your needs. For less critical monitoring, a longer interval might be appropriate:

watch -n 5 "df -h"   # Updates every 5 seconds

Finding Space-Consuming Files and Directories

While df shows overall usage, you often need to identify exactly what’s consuming space. For this, use the du (disk usage) command:

# Find the largest directories in the current location
du -h --max-depth=1 | sort -hr

# Find the largest files in a specific directory
find /var/log -type f -exec ls -lh {} \; | sort -k5hr | head -10

Proactive Disk Space Management Tips

  1. Implement Log Rotation: Ensure all log files are properly rotated and compressed.
  2. Set Up Monitoring Alerts: Configure alerts to notify you when filesystems reach critical thresholds (e.g., 80% usage).
  3. Regular Maintenance: Schedule routine cleanup operations for temporary directories, cache files, and old backups.
  4. Review Partition Schemes: If possible, adjust partition layouts based on observed usage patterns.
  5. Use LVM: Logical Volume Management provides flexibility for resizing filesystems as needs change.

Conclusion

Effective disk space management is essential for maintaining system stability and performance. With the commands and techniques outlined in this guide, you can monitor disk usage, identify potential issues, and implement solutions before they impact your services. Regular monitoring and proactive management will help ensure your Linux systems operate smoothly even as data volumes grow.

Fighting the Email Battle: ASSP Anti-Spam Success and Limitations

By in Blog Tags: , , , , ,

The Search for the Perfect Anti-Spam Solution

After years of testing various anti-spam solutions across different server environments, I’ve finally found what I consider to be the gold standard: ASSP (Anti-Spam SMTP Proxy).

This powerful tool has consistently outperformed other anti-spam solutions I’ve implemented over the years. Its effectiveness in catching unwanted emails while minimizing false positives has made it my go-to recommendation for server administrators dealing with email spam.

The Challenge of Diminishing Returns

Despite ASSP’s impressive performance, I’ve noticed a concerning pattern. Over time, the effectiveness tends to diminish as spammers evolve their tactics. What starts as a rock-solid defense gradually becomes more porous.

This isn’t a criticism of ASSP specifically – it’s the reality of the never-ending cat-and-mouse game between spam filters and spammers. As spam techniques evolve, even the best solutions need reinforcement.

Looking for Complementary Solutions

Rather than replacing ASSP entirely, I’m searching for complementary solutions that can work alongside it, creating multiple layers of defense against unwanted emails.

My question to fellow IT professionals: What anti-spam systems have you successfully implemented that work well in conjunction with ASSP? I’m specifically looking for solutions that:

  • Can run in parallel with ASSP
  • Target different aspects of spam detection
  • Don’t conflict with ASSP’s functionality
  • Provide additional layers of protection

If you’ve found a winning combination that strengthens your email security beyond what ASSP can provide alone, I’d love to hear about your experience.

Copyright Michael Helm 2026.
The earliest version of this site available is from October 2003 - possibly viewable here: https://web.archive.org/web/20031228013629/http://www.ihelm.org.uk:80/

↑ Back to top