Linux commands

Keeping Your Raspberry Pi Install Tidy After Upgrades and Changes

By in Blog Tags: , , , , , , , ,

Whilst Linux in general is very good at being neat and tidy, devices such as the Raspberry Pi, where resources can be much more limited than other computers, can sometimes suffer—usually from lack of space (performance will be dealt with in another post).

The Problem with Updates and Package Changes

If you’ve recently made changes—removed some packages, or manually updated some components—you may find your disk space getting a bit lower. This is because Linux package managers don’t always clean up perfectly after themselves. Some common scenarios that lead to wasted space include:

  • Cached package files that were downloaded during updates
  • Dependencies that are no longer needed after removing applications
  • Older versions of packages kept after upgrades
  • Temporary files created during installations

Simple Cleanup Commands

Fortunately, one of the tools you can use is built into apt-get and allows it to scan and remove dependencies no longer required. To reclaim your disk space, simply run the following commands:

1. Clean Up Package Archives

sudo apt-get autoclean

This removes old archive files that might have been left behind. Note you can use clean as well—however, this can cause slowness as it removes all archive files, and some may be needed again soon if you’re frequently installing and updating packages.

2. Remove Unused Packages

sudo apt-get autoremove

This removes any unused packages—often these can be left behind if you remove a programme which had optional dependencies, as these dependencies are not always removed by the system. Other system changes may have now marked certain packages as unused as well.

Results You Can Expect

The output of these commands will look similar to this:

Screen-Shot-2016-02-03-at-00.37.23 Claiming back some disk space using apt-get autoremove

In this particular example, I gained back 300MB of space. However, I’ve previously recovered as much as 1.2 gigabytes from a Raspberry Pi that had gone through multiple update cycles and application changes.

When to Run These Commands

For optimal system maintenance, consider running these cleanup commands:

  1. After removing large applications
  2. Following significant system updates
  3. When you receive low disk space warnings
  4. As part of a regular monthly maintenance routine
  5. Before creating a backup or disk image

Additional Space-Saving Tips

Beyond these basic commands, here are some other ways to keep your Raspberry Pi’s storage optimised:

Clear Logs

Log files can accumulate over time. To safely truncate them:

sudo journalctl --vacuum-time=7d

This keeps only the last seven days of logs.

Check for Large Files

You can find unexpected space-hogging files with:

sudo find / -type f -size +100M

This will list all files larger than 100MB, helping you identify potential candidates for removal.

Minimise Installed Packages

Consider running a minimal installation if your Raspberry Pi is dedicated to a specific task. Many Raspberry Pi OS installations include software you might not need.

Conclusion

Regular maintenance using these simple commands can help ensure your Raspberry Pi runs efficiently within its storage constraints. This is particularly important for models with smaller SD cards or for projects that generate or download significant amounts of data.

By integrating these cleanup steps into your routine maintenance, you can avoid the frustration of running out of space unexpectedly and extend the useful life of your SD card by reducing unnecessary write cycles.

Copying Files Between Linux Servers – Using SCP

By in Blog Tags: , , , , , , , ,

For a few years now I keep catching myself transferring files between Linux servers in a silly way…

What I mean is using something like FTP, or copying to my desktop and then up to the other server – all hassle (and in the case of FTP – inherently insecure). Why do I do this? Well not because there’s no alternative, but because I can never easily remember how to do it the “proper” way.

The Proper Way: SCP

So how to do it – simply use SCP. This copies a file using SSH and so is more secure (we trust SSH right?), even better I’ve never had a server which doesn’t support it out of the box.

Basic File Transfer

Right so how to transfer one file:

scp /sourcedir/filename.extension user@serverip:/destinationdir/

It’s quite simple really – if you run SSH on an abnormal port (you really should it cuts down the attacks) then you can specify the port like this:

scp -P 2100 /sourcedir/filename.extension user@serverip:/destinationdir/

Transferring Multiple Files

Also, if you want to transfer multiple files (e.g. all the images in a directory) then you can just use wildcards – like below:

scp -P 2100 /sourcedir/*.extension user@serverip:/destinationdir/

Additional Useful Options

Here are some other helpful SCP options that can make file transfers even more efficient:

  • -r: Recursively copy entire directories
    scp -r /sourcedir/ user@serverip:/destinationdir/
  • -C: Enable compression during the transfer
    scp -C /sourcedir/largefiles.tar user@serverip:/destinationdir/
  • -v: Verbose mode to see detailed transfer information
    scp -v /sourcedir/filename.extension user@serverip:/destinationdir/
  • -q: Quiet mode to suppress progress meter and warning messages

Why SCP is Better

Using SCP provides several advantages over FTP or manual transfers:

  1. Security: All transfers are encrypted through SSH
  2. Simplicity: No need to install additional software
  3. Efficiency: Direct server-to-server transfers save bandwidth and time
  4. Authentication: Uses your existing SSH keys/credentials

So next time you need to move files between Linux servers, skip the FTP client and use SCP directly. Your data will be more secure, and you’ll save valuable time.

Remounting Linux Filesystems on the Fly: A Quick Reference Guide

By in Blog Tags: , , , , , , , , ,

One of the many advantages of Linux over other operating systems is its flexibility when it comes to filesystem management. A particularly useful capability is the ability to unmount and remount filesystems without requiring a system reboot. This can be invaluable when you need to make configuration changes or perform maintenance tasks while minimizing downtime.

Basic Unmounting and Remounting Procedure

The process of remounting a filesystem “on the fly” is quite straightforward. For example, to work with a tmpfs filesystem:

  1. First, unmount the filesystem:
    umount tmpfs
    This command tells the operating system to unmount the filesystem identified as “tmpfs” in the /etc/fstab file.
  2. Then, remount the filesystem:
    mount tmpfs
    This command will mount the filesystem again, applying any configuration changes you’ve made in the /etc/fstab file.

Potential Challenges and Solutions

When attempting to unmount a filesystem, you might encounter error messages if the filesystem is currently in use. Common scenarios that can prevent unmounting include:

  • Having an SSH session with your current directory inside the filesystem
  • Files being open and in use by running processes
  • System processes accessing the filesystem

If you receive an error indicating that the filesystem is busy, you’ll need to:

  1. Exit any directories within the filesystem
  2. Close any open files on the filesystem
  3. Stop any processes that might be using files on the filesystem

Once you’ve addressed these dependencies, you can try the unmount command again, and it should succeed.

Real-World Applications

There are several practical scenarios where you might need to remount a filesystem:

1. Changing Filesystem Parameters

After modifying filesystem parameters in /etc/fstab, such as adjusting the size of a tmpfs partition (as described in my previous post about resizing RAM disks), you’ll need to remount the filesystem to apply the changes.

2. Changing Mount Options

You might need to add or modify mount options such as:

  • noexec (prevent execution of binaries)
  • nosuid (ignore SUID/SGID bits)
  • ro or rw (read-only or read-write access)
  • noatime (don’t update access times)

3. Filesystem Maintenance

Some maintenance tasks require remounting with specific options, such as remounting a filesystem as read-only to run certain types of filesystem checks.

Advanced Remounting Options

For more complex situations, you can use the mount command with the -o remount option to change mount options without fully unmounting first:

mount -o remount,option1,option2 /mount/point

This approach is particularly useful for system directories that are difficult to fully unmount during operation.

Monitoring Filesystem Status

To verify the current mount status of your filesystems, use:

df -h

Or for more detailed information including mount options:

mount | grep filesystem_name

Conclusion

The ability to remount filesystems without a system restart is one of many features that make Linux an excellent choice for servers and systems requiring high availability. By understanding these basic commands, you can make configuration changes to your filesystems with minimal disruption to running services.

Remember that while this functionality is powerful, it should be used carefully, especially on production systems. Always ensure you understand the dependencies and potential impacts before unmounting any critical filesystem.

How to Use Crontab to Schedule Tasks in Linux: A Complete Guide

By in Blog Tags: , , , , , , , , ,

One of the most essential tools in a system administrator’s toolkit is crontab. This powerful utility allows you to schedule jobs (programs or scripts) to run automatically at specified intervals—a capability that can significantly enhance server management and automation.

What Can You Schedule with Crontab?

The versatility of crontab is remarkable. You can schedule a wide variety of tasks, including:

  • Restarting servers or individual services
  • Creating, modifying, or deleting files
  • Changing file permissions
  • Running batch programs or scripts
  • Performing regular backups
  • Executing maintenance tasks
  • Sending automated emails or reports

Understanding Cron: Key Facts

Before diving into usage, here are some important facts about cron:

  1. Evolution with Consistency: While cron has evolved since its inception, the basic principles remain the same—making it a timeless skill for system administrators.
  2. One-Minute Resolution: Cron can run jobs at most once per minute—this is the smallest time interval available.
  3. Efficient Execution: Modern cron implementations don’t continuously check for jobs every minute. Instead, they load crontab files into memory and only execute when necessary.
  4. Dynamic Reloading: When you edit a crontab file, the cron service automatically reloads it, eliminating the need to restart the service.
  5. Concurrent Execution: Cron will start new jobs according to schedule even if previous instances haven’t finished. This can potentially overload poorly configured systems, so careful planning is essential.

Editing Your Crontab

To edit your personal crontab file:

crontab -e

This command opens your crontab file in the default system editor (typically vim, nano, or another configured editor).

Crontab Syntax and Format

A typical crontab entry looks like this:

1 0 * * * shutdown -r now

This example would reboot the server (shutdown -r now) at one minute past midnight every day.

Understanding the Fields

Each crontab entry consists of five time fields followed by the command to execute:

# .---------------- minute (0 - 59) 
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ... 
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat 
# |  |  |  |  |
# *  *  *  *  * command_to_execute

The asterisk (*) is a wildcard that matches all possible values. When used in a field, it means “every” (every minute, every hour, etc.).

Common Crontab Examples

Here are some practical examples to help you get started:

Run a Script Every Day at 3:30 AM

30 3 * * * /path/to/your/script.sh

Run a Backup Every Monday at 2:15 AM

15 2 * * 1 /usr/local/bin/backup.sh

Run a Database Cleanup on the First of Every Month

0 0 1 * * /usr/local/bin/db_cleanup.sh

Run a Script Every Hour

0 * * * * /path/to/hourly_script.sh

Run a Script Every 15 Minutes

*/15 * * * * /path/to/frequent_script.sh

Capturing Output

When a cron job executes, any output it generates can be captured and redirected to a file. Simply add a redirection operator to your command:

0 0 * * * /path/to/script.sh > /var/log/script_output.log

This will save the standard output to the specified log file. For both standard output and error messages, use:

0 0 * * * /path/to/script.sh > /var/log/script_output.log 2>&1

Best Practices for Crontab

  1. Document Your Crontab: Add comments (lines starting with #) to explain what each job does.
  2. Use Absolute Paths: Always use full paths for both the commands and any files they reference.
  3. Test Commands First: Before adding a command to crontab, test it manually from the command line.
  4. Monitor Job Output: Redirect output to log files and check them regularly.
  5. Consider Alternatives: For very frequent tasks or those requiring more sophisticated scheduling, consider alternatives like systemd timers.

Conclusion

Crontab is a powerful tool that allows for robust task automation in Linux systems. By mastering its syntax and capabilities, you can significantly enhance your system administration efficiency, ensuring that routine tasks are handled automatically and consistently.

In future posts, I’ll cover alternative scheduling methods for situations where cron might not be the optimal solution, including writing custom scheduling services.

Using Grep to Find Text in Files: An Essential Linux Command

By in Blog Tags: , , , , , , , , ,

As a system administrator, some tools become part of your daily workflow, and grep is definitely one of them. Despite using it regularly, I often find myself double-checking the syntax, which inspired me to create this quick reference guide—both for myself and for others who might find it useful.

What is Grep?

grep (Global Regular Expression Print) is a powerful command-line utility that searches text files for lines matching a specified pattern. It’s one of the most versatile search tools available in Linux and Unix-like operating systems.

Basic Usage

For the most effective use of grep, I recommend changing to the directory where you suspect the file containing your target text is located. This simplifies your commands and makes the output easier to interpret.

Finding a String in Files Within the Current Directory

To find a specific string in any file in the current directory:

grep "search_term" *

For example, to find my name in any file:

grep "michael" *

This will return the filename and the matching line for each occurrence found within the current directory.

Recursive Searching Through Subdirectories

To search not just the current directory but all subdirectories as well:

grep -R "search_term" *

For example:

grep -R "michael" *

The -R (or --recursive) flag tells grep to follow directory structures recursively, searching through all files in all subdirectories.

Beyond the Basics

While the commands above cover the most common use cases, grep offers many more powerful features:

  • Case-insensitive searching: Add the -i flag to ignore case distinctions
    grep -i "Michael" *
  • Display line numbers: Use the -n flag to show the line number of each match
    grep -n "michael" *
  • Count occurrences: The -c flag returns only a count of matching lines
    grep -c "error" log.txt
  • Match whole words only: The -w flag ensures the pattern matches whole words
    grep -w "log" *

Real-World Applications

Grep is particularly valuable for examining log files and identifying patterns—essential tasks in system administration. For instance:

  • Find all failed login attempts:
    grep "Failed password" /var/log/auth.log
  • Check for specific error messages across multiple logs:
    grep -R "Connection refused" /var/log/
  • Identify requests from a specific IP address in web server logs:
    grep "192.168.1.1" /var/log/apache2/access.log

Conclusion

The grep command is one of the most powerful tools in a Linux administrator’s toolkit. While I’ve covered the basics here, exploring its more advanced features can significantly enhance your system administration capabilities. The next time you need to find text in files, remember this handy command—it might just save you hours of manual searching.

Adding a New Swap Partition in Linux: Step-by-Step Guide

By in Blog Tags: , , , , , , , , ,

Following my previous post on creating a swap file, I realized it would be worthwhile to discuss another approach: adding an entirely new swap partition. While swap files are excellent for emergency situations, a dedicated swap partition often provides better performance for long-term use.

Prerequisites

Before we begin, you’ll need:

  • A separate hard drive that is either not currently mounted or can be safely unmounted using the standard umount command
  • Root/sudo access to your system

For this guide, I’ll assume the device is /dev/sdc, but your system might use a different device name (such as /dev/sdb, /dev/sda, /dev/hda1, etc.).

Step-by-Step Process

  1. Ensure the drive is unmounted If the drive already contains a swap partition that’s in use, disable it first:
    swapoff /dev/sdc
    If the drive is clean and doesn’t have any swap partitions, you can skip this step.
  2. Access the partition manager Launch the parted utility to manage the disk partitioning:
    parted /dev/sdc
    This will load the parted prompt, which provides an interface for manipulating disk partitions.
  3. View existing partitions At the parted prompt, enter:
    print
    This will display details of any existing partitions and available free space on the disk.
  4. Create the new swap partition Once you’ve identified how much space you want to allocate (at minimum, I recommend matching your RAM size), create the partition with:
    mkpartfs part-type linux-swap start end
    Replace start and end with appropriate values shown from the print command. For example, if the starting point is 1024MB and you want a 4GB swap partition, your end point would be 5120MB (1024 + 4096).
  5. Exit parted Type quit to exit the parted utility. The new partition will have been assigned a number, such as sdc2 (or higher if other partitions already exist).
  6. Format the new partition for swap Format the newly created partition as swap space:
    mkswap /dev/sdc2
    (Remember to replace sdc2 with your actual partition name)
  7. Activate the swap partition Enable the swap partition with:
    swapon /dev/sdc2

Making the Change Permanent

If you want the system to automatically use this swap partition on every boot, you need to add it to the file system table:

  1. Edit the fstab file
    vi /etc/fstab
  2. Add the swap entry Press Insert to enter edit mode, then add this line to the bottom of the file:
    /dev/sdc2 swap swap defaults 0 0
  3. Save and exit Press Escape, then type :wq or press Shift+Z twice to save and quit.

Verifying the New Swap Space

To confirm that your new swap partition has been successfully added and is active, you can check the system’s swap status:

cat /proc/swaps

This will display all active swap spaces on your system, including your newly created partition.

Final Thoughts

Adding a dedicated swap partition is generally considered a more efficient long-term solution than using a swap file, especially for systems that regularly require swap space. The performance difference is modest on modern systems, but can be noticeable under heavy load.

Remember that while swap space is important for system stability, it’s not a replacement for adequate RAM. If your system is consistently using large amounts of swap space, consider upgrading your physical memory for better performance.

Adding Swap Space Without a Partition in Linux: A Quick Emergency Fix

By in Blog Tags: , , , , , , , , ,

Recently, we encountered a challenging issue on one of the servers I help administer. The server would occasionally max out its RAM usage, forcing it to rely on swap space. When both RAM and swap space were exhausted, the server would grind to a halt, requiring a manual restart.

The Problem: Growing Needs, Limited Resources

The server in question had been in operation for several years, starting with modest traffic but gradually growing as the websites it hosted gained popularity and their databases expanded. Despite this growth, it was still running with only 512MB of RAM—a limitation we’ve since addressed with an upgrade.

However, we needed an immediate solution before the hardware upgrade could be implemented. The server already had a swap partition, but creating another one wasn’t feasible without significant downtime, as all available hard drives were in active use.

The Solution: Adding a Swap File

The answer turned out to be creating a swap file rather than a swap partition. This approach allows you to add additional swap space without reconfiguring your disk partitions—perfect for an emergency fix. Here’s how to do it:

  1. Navigate to a directory with plenty of available space: 

    cd /home
  2. Create a swap file using /dev/zero: 
    bash
    dd if=/dev/zero of=swapfile bs=1024 count=4194304
    This creates an empty file called “swapfile” with a block size of 1024 bytes and 4,194,304 blocks, resulting in a 4GB file. (To calculate the size in GB: 4,194,304 ÷ 1024 ÷ 1024 = 4GB)
  3. Set appropriate permissions on the file: 

    chmod 600 swapfile
  4. Format the file as swap space: 

    mkswap swapfile
  5. Activate the swap file: 

    swapon swapfile

To verify that your new swap space is active, you can run:

cat /proc/swaps

This command displays all active swap spaces on your system, and you should now see your newly created swap file listed alongside any existing swap partitions.

Important Considerations

This solution doesn’t automatically activate on system boot, which was fine for our purposes since we planned to upgrade the RAM soon. If you need a more permanent solution, you would need to add an entry to your /etc/fstab file.

Remember that while this approach can prevent system crashes, it’s only a temporary workaround. Swap space, especially on a traditional hard drive, is significantly slower than RAM. For optimal performance, it’s always better to address the root cause by:

  1. Upgrading physical RAM
  2. Optimizing your application configurations
  3. Considering server upgrades if necessary

That said, when you need a quick fix to keep a server stable until more permanent solutions can be implemented, adding a swap file is a valuable technique to have in your system administration toolkit.

Copyright Michael Helm 2026.
The earliest version of this site available is from October 2003 - possibly viewable here: https://web.archive.org/web/20031228013629/http://www.ihelm.org.uk:80/

↑ Back to top