Minimalist Raspberry Pi Server Image

The following tutorial will help you generate a 367MB Raspbian image (uncompressed).

The standard 2012-12-16 version of Raspbian “Wheezy” fills up 1.5GB (or 88%) of a 2GB SD card, which has a formatted capacity of only 1.8GB. While I have a larger capacity SDHC card for this particular RPi (I have five “pies”), one of the projects I’m considering is potentially quite data intensive, so I want to reserve as much disk space as possible.

My previous article began addressing the removal of unnecessary packages but its primary focus was on reducing I/O usage, however this article is focused on conserving disk space and reducing the operating system fingerprint. This optimizes the RPi for a server-type environment, rather than its use as a desktop workstation.

This minimalist image is different from others since it is significantly smaller than most images, yet it retains sound and touch-screen support.

Remove the GUI Modules

Let’s start by getting rid of over 325 optional packages (about 700MB) used by the graphical user interface:

sudo apt-get purge xserver.* x11.* xarchiver xauth xkb-data console-setup xinit lightdm lxde.* python-tk python3-tk scratch gtk.* libgtk.* openbox libxt.* lxpanel gnome.* libqt.* libxcb.* libxfont.* lxmenu.* gvfs.* xdg-.* desktop.* tcl.* shared-mime-info penguinspuzzle omxplayer gsfonts
sudo apt-get --yes autoremove
sudo apt-get upgrade

Remove Development Packages (Optional)

My personal minimalist RPi image retains most of the default installed development packages.

sudo apt-get purge gcc-4\.[0-5].*

However, you can optionally recover almost 150MB of space by removing the majority of compiler and development packages from your image.

sudo apt-get purge `sudo dpkg --get-selections | grep "\-dev" | sed s/install//` gcc-4\.[0-6].* python.*
sudo apt-get --yes autoremove

Disable Memory Swapping

Memory swapping allows part of the usable memory of your computer to be stored on disk, effectively allowing the system to have more RAM than it does physically. Disabling swap files helps free up valuable disk space.

sudo swapoff -a
sudo apt-get purge dphys-swapfile
sudo rm /var/swap

Delete Unnecessary Extras

A few stray files and folders slice up more than 125MB of the Raspberry Pi.

sudo rm -rf /usr/share/doc/* /opt/vc/src/hello_pi/hello_video/test.h264 /home/pi/python_games
find /usr/share/locale/* -maxdepth 0 -type d |grep -v en |xargs sudo rm -rf
find /usr/share/man/* -maxdepth 0 -type d |grep -Pv 'man\d' |xargs sudo rm -rf

Replace SSHd with Dropbear

Dropbear is a lightweight SSH v2 server which lacks backwards compatability support for SSH v1 (translation: smaller code base, uses less memory and fewer resources) and is secure enough for general use — ideal for the Raspberry Pi.

sudo apt-get install dropbear openssh-client
sudo /etc/init.d/ssh stop
sudo sed -i 's/NO_START=1/NO_START=0/' /etc/default/dropbear
sudo /etc/init.d/dropbear restart

After testing that you can still connect to your RPi via SSH, remove the default ssh server.

sudo apt-get purge openssh-server.* openssh-blacklist.*

Eliminate Rarely Used Caches

Prevent aptitude from caching package data by creating /etc/apt/apt.conf.d/02_nocache. This will slow down package updates, but will save about 35MB of disk space in the final image. You could remove the aptitude package altogether, but I find its presence more useful than not, especially when the bulky cache files are removed.

Dir::Cache {
   srcpkgcache "";
   pkgcache "";
}

Since we’re done using apt for package updating, after you reboot your Pi with sudo reboot, you can remove the orphaned cache files to recover another 35MB of disk space:

sudo rm -f /var/cache/apt/*cache.bin
sudo apt-get --yes autoclean
sudo apt-get --yes clean

Prepare for Final Imaging

Numerous files and logs are safely re-created or re-fetched as needed. Removing them will shrink the overall size of the finalized minimalist distro.

sudo find / -type f -name "*-old" |xargs sudo rm -rf
sudo rm -rf /var/backups/* /var/lib/apt/lists/* ~/.bash_history
find /var/log/ -type f |xargs sudo rm -rf
sudo cp /dev/null /etc/resolv.conf

Having also incorporated some drive-I/O-reduction techniques, running df -hlP on my Raspberry Pi shows disk usage of only 367MB, a significant reduction from the original 1.5GB. Your mileage may vary. Shutdown the Raspberry Pi with sudo halt, remove the SD card from the Raspberry Pi board and access it from a Mac or Linux computer to create the final image.

Author’s Note: If you’re just cleaning up your Raspbian installation and have no need to create a distributable image, you can stop now.

To determine which drive designation to use on a Mac, type diskutil list and locate the Identifier of the Linux disk type that ranges around 1.9 GB. On my MacBook Pro, mine is “disk1s2”.

sudo dd if=/dev/disk1s2 of=2012-12-16-wheezy-raspbian-rr-minimalist.img count=3788800

An archive image file is transferred to your Linux PC or Mac, ready for the next steps to generate a zero-byte-filled, compressed final distribution image. Since I’m using a Mac, for which the steps are not very straightforward because the Mac will not natively mount the RPi ext4 partition, I’ll cover that in a future article.

In the meantime, for native Linux users, you can generate a compressed image with the following generic steps.

mkdir mnt
sudo mount -o loop,offset=$((512*122880)) 2012-12-16-wheezy-raspbian-rr-minimalist.img mnt
sudo sfill -z -l -l -f mnt
sudo umount mnt
7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on 2012-12-16-wheezy-raspbian-rr-minimalist.img.7z 2012-12-16-wheezy-raspbian-rr-minimalist.img

If you’ve gotten this far, congratulations! The original Raspbian installation image uncompressed to 1.5GB, while the minimalist version is well under 600MB — or even sub-400MB, depending on the options chosen. Use your image any way you want in-house, however I wouldn’t start distributing your image quite yet; wait for a future article where we cover the final topics of security and key re-generation. And no one, not even the builders of Raspbian, seems to have properly addressed the licensing concerns of the included non-free hardware drivers.



Extend the Life of Your RPi SD Card

Many of the modern single-board or System-on-Chip computers (such as Raspberry Pi or BeagleBone) use standard SD memory cards for drive volumes. Since frequent writes eventually damage flash memory systems, in order to extend the life of the SD card, it is important to reduce as many superfluous writes as practical. Shutting down the Pi improperly while it is writing to the card can also induce errors and corruption to open volumes, so reducing writes can have the added benefit of prolonged stability of the file systems.

Author’s Note: This tutorial assumes you have the 2012-12-16 Raspbian “wheezy” distribution installed on your Raspberry Pi, and you have not begun additional customization. While the techniques described below generally apply to other common Linux distros, the instructions, code, and commands may not apply or function in those other cases. You have been warned.

While I’ll be using one of my Pi’s as a built-in intranet web server installed into my powered structured media cabinet (mainly as a wiki and general file server), the goal of significantly fewer periodic writes should be obtainable because I’m hoping that much of the information on the wiki will be fairly static. Worst case, the Pi could buffer logs in memory and then dump them to a NAS or to an Amazon EC2 instance — which would be another article for another time.

Install Sysstat

In order to measure the success of your efforts, you will need a method of monitoring disk activity — one of the most widely used commands of which is iostat, part of the sysstat package.

sudo apt-get install sysstat

Running iostat with no arguments displays aggregate usage information gathered since the system was last booted. Adding interval and count arguments first displays usage since boot, then displays usage during each time period since the previous report.

In the following example, three iostat reports are displayed; the first report on usage since boot is displayed instantly, followed by a usage report gathered over 300 seconds (or five minutes), followed by another five-minute report. Because the focus of this examination is on device usage and not CPU usage, the “-d” (or “device”) option is added; the opposite would require a “-c” (or “cpu”) option.

Run the command before you go any further in order to test the impact of the optimizations below.

$ iostat -d 300 3
Linux 3.2.27+ (raspberrypi) 	2013/02/07 	_armv6l_	(1 CPU)

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.13         2.87         0.95     138189      45840

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.05         0.00         0.35          0        104

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.06         0.00         0.49          0        144

According to the above initial results, during the first five minutes of examination, 104 kilobytes of data was written to disk and 144 KB was written during the second five minutes, for a total of 248 KB over ten minutes. No reads were performed during the entire test.

Disable Swapping

Memory swapping allows part of the usable memory of your computer to be stored on disk, effectively allowing the system to have more RAM than it does physically. Even on high-power Windows or Mac computers, running applications with extensive memory usage beyond the hardware RAM will result in the computer slowing down while the hard drive light blinks incessantly, indicating that the OS is swapping in and out available virtual RAM.

Raspbian utilizes a swap file (managed by dphys-swapfile) in lieu of a more standard swap partition. Removing the functionality is a better long-term solution that simply turning it off.

sudo apt-get remove dphys-swapfile

Remove Unused Packages

In order to build a distribution that satisifies the needs of the majority, Raspbian includes many packages that many RPi users won’t use, especially those that solely intend their Pi to be headless. Removing packages needed for the X Window System graphical user interface will recover a significant amount of disk space, as well as remove always-on daemons that continually generate log messages and other disk I/O (even if you never use the desktop).

sudo apt-get --yes purge xserver.* x11-.* xarchiver xauth xkb-data console-setup xinit lightdm lxde.* python-tk python3-tk scratch gtk.* libgtk.* openbox libxt.* lxpanel gnome.* libqt.* gvfs.* xdg-.* desktop.*
sudo apt-get --yes autoremove
sudo apt-get --yes autoclean
sudo apt-get --yes clean

Author’s Note: Use the above purge command with caution if you have already begun customization of your Raspberry Pi. You may delete packages that are unintended.

Purely to recover more disk space (and not neceesarily in consideration of the SD card life expectancy), I used sudo ncdu from the root partition to remove unused locales from within /usr/share/locale/ as well as everything under /tmp/.

Also consider removing the following files:

  • /var/swap (100 MB)
  • /opt/vc/src/hello_pi/hello_video/test.h264 (30 MB)

Before I started purging, disk usage on my Pi was 1.5 GB (as determing by running df -hlP). After performing all of above suggestions, total disk usage was reduced to only 609 MB, recovering almost 60% of installed space. Your milege may vary.

Reduce Unnecessary Logging

Every couple of minutes cron jobs tend to add log entries to /var/log/auth.log similar to the following:

pam_unix(cron:session): session opened for user root by pi(uid=0)
pam_unix(cron:session): session closed for user root

Edit the /etc/pam.d/common-session-noninteractive configuration file and add the following line near the bottom of the file just before the pam_unix.so call to prevent cron jobs that complete successfully from adding the authorization logs.

session [success=1 default=ignore] pam_succeed_if.so service in cron quiet use_uid

Comment out the log settings line in the /etc/dpkg/dpkg.cfg configuration file. Set the DHCP lease time on your router to no less than one day.

Rotate Logs Less Frequently

The default logrotate configuration files tend to rotate logs daily or weekly, unnecessary on a low-usage server. Edit the /etc/logrotate.d/rsyslog configuration file and replace the “daily” and “weekly” directives with “monthly”, and update the various “rotate nn” options to “rotate 12”.

For your (and my) reference, here’s a copy of my refactored main syslog logrotate configuration file:

monthly
size 10m
rotate 12
missingok
notifempty
compress
delaycompress

/var/log/syslog {
	postrotate
		invoke-rc.d rsyslog rotate > /dev/null
	endscript
}

/var/log/mail.info /var/log/mail.warn /var/log/mail.err
/var/log/*.log /var/log/debug /var/log/messages {
	sharedscripts
	postrotate
		invoke-rc.d rsyslog rotate > /dev/null
	endscript
}

Use Temporary Filesystems

Older Raspbian distributions once seem to have enabled temporary filesystems for the /tmp/ folder, but the latest “Wheezy” has the feature disabled by default, only mounting /run/ and its subfolders as tmpfs filesystems. Reboot after re-enabling the capability.

sudo sed -i 's/\#\s*RAMTMP\s*=\s*no/RAMTMP=yes/i' /etc/default/tmpfs

Verify Improvements

Run the same iostat command as above again in order to compare before and after results. Hopefully you should see some improvement.

$ iostat -d 300 3
Linux 3.2.27+ (raspberrypi) 	2013/02/08 	_armv6l_	(1 CPU)

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.44         1.74         7.84     177045     795692

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.00         0.00         0.00          0          0

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.03         0.00         0.24          0         72

A small sampling of data isn’t truly sufficient to gauge long-term effectiveness of these procedures. That said, according to the latest results above, only 72 kilobytes of data was written to disk during the entire ten-minute test, a reduction of approximately 70% as compared to the initial usage test results.

Given similar usage over an extended period of time, the longevity of the above-tested SD card could be extended by more than three times simply by eliminating memory swaps, uninstalling unneeded modules, and reducing a few unnecessary logs and their subsequent maintenance.