O'Reilly: Other Administrative Tasks (Running Linux)

After reading the previous three chapters, you now have all the skills you need to start using your system. But don't ignore this chapter for long. Some of the activities, such as making backup tapes, are important habits to develop. You may also find it useful to have access to files and programs on MS-DOS and Windows. Finally, we'll help you handle events that you hope will never happen, but sometimes do--system panics and corruption.

8.1. Making Backups

Making backups of your system is an important way to protect yourself from data corruption or loss in case you have problems with your hardware, or you make a mistake such as deleting important files inadvertently. During your experiences with Linux, you're likely to make quite a few customizations to the system that can't be restored by simply reinstalling from your original installation media. However, if you happen to have your original Linux floppies or CD-ROM handy, it may not be necessary to back up your entire system. Your original installation media already serves as an excellent backup.

Under Linux, as with any Unix-like system, you can make mistakes while logged in as root that would make it impossible to boot the system or log in. Many newcomers approach such a problem by reinstalling the system entirely from backup, or worse, from scratch. This is seldom, if ever, necessary. In the section "Section 8.6, "What to Do in an Emergency"," we'll talk about what to do in these cases.

If you do experience data loss, it is sometimes possible to recover that data using the filesystem maintenance tools described in the section "Section 6.1.5, "Checking and Repairing Filesystems"" in Chapter 6, "Managing Filesystems, Swap, and Devices". Unlike some other operating systems, however, it's generally not possible to "undelete" a file that has been removed by rm or overwritten by a careless cp or mv command (for example, copying one file over another destroys the file you're copying to). In these extreme cases, backups are key to recovering from problems.

Backups are usually made to tape or floppy. Neither medium is 100 percent reliable, although tape is more dependable than floppy in the long term. There are many tools available that help you to make backups. In the simplest case, you can use a combination of gzip and tar to back up files from your hard drive to floppy or tape. This is the best method to use when you make only occasional backups, no more often than, say, once a month.

If you have numerous users on your system or you make frequent changes to the system configuration, it makes more sense to employ an incremental backup scheme. Under such a scheme, you would take a "full backup" of the system only about once a month. Then, every week, you would back up only those files that changed in the last week. Likewise, each night, you could back up just those files that changed over the previous 24 hours. There are several tools to aid you in this type of backup.

The idea behind an incremental backup is that it is more efficient to take backups in small steps; you use fewer floppies or tapes, and the weekly and nightly backups are shorter and easier to run. With this method, you have a backup that is at most a day old. If you were to, say, accidentally delete your entire system, you would restore it from backup in the following manner:

Restore from the most recent monthly backup. Say, if you wiped the system on July 17th, you would restore the July 1 full backup. Your system now reflects the state of files when the July 1 backup was made.
Restore from each of the weekly backups made so far this month. In our case, we could restore from the two weekly backups from July 7th and 14th. Restoring each weekly backup updates all of the files that changed during that week.
Restore from each of the daily backups during the last week, that is, since the last weekly backup. In this case, we would restore the daily backups from July 15th and 16th. The system now looks as it did when the daily backup was taken on July 16th; no more than a day's worth of files have been lost.

Depending on the size of your system, the full monthly backup might require 2 GB or more of backup storage--often not more than one tape using today's tape media, but quite a few ZIP disks. However, the weekly and daily backups would generally require much less storage space. Depending on how your system is used, you might decide to take the weekly backup on Sunday night and not bother with daily backups for the weekend.

One important characteristic that backups should (usually) have is the ability to select individual files from the backup for restoration. This way, if you accidentally delete a single file or group of files, you can simply restore those files without having to do a full system restoration. Depending on how you take backups, however, this task will be either very easy or painfully difficult.

In this section, we're going to talk about the use of tar, gzip, and a few related tools for making backups to floppy and tape. We'll even cover the use of floppy and tape drives in the bargain. These tools allow you to take backups more or less "by hand"; you can automate the process by writing shell scripts and even schedule your backups to run automatically during the night using cron. All you have to do is flip tapes. There are other software packages that provide a nice menu-driven interface for creating backups, restoring specific files from backup, and so forth. Many of these packages are, in fact, nice frontends to tar and gzip. You can decide for yourself what kind of backup system suits you best.

8.1.1. Simple Backups

The simplest means of taking a backup is to use tar to archive all the files on the system or only those files in a set of specific directories. Before you do this, however, you need to decide what files to back up. Do you need to back up every file on the system? This is rarely necessary, especially if you have your original installation disks or CD-ROM. If you have made important changes to the system, but everything else is as just the way it was found on your installation media, you could get by only archiving those files you have made changes to. Over time, however, it is difficult to keep track of such changes.

In general, you will be making changes to the system configuration files in /etc. There are other configuration files as well, and it can't hurt to archive directories, such as /usr/lib, /usr/X11R6/lib/X11 (which contains the XFree86 configuration files, as we'll see in the section "Section 10.3, "Installing XFree86"" in Chapter 10, "Installing the X Window System").

You should also back up your kernel sources (if you have upgraded or built your own kernel); these are found in /usr/src/linux.

During your Linux adventures it's a good idea to keep notes on what features of the system you've made changes to so you can make intelligent choices when taking backups. If you're truly paranoid, go ahead and back up the whole system: that can't hurt, but the cost of backup media might.

Of course, you should also back up the home directories for each user on the system; these are generally found in /home. If you have your system configured to receive electronic mail (see the section "Section 16.2.1, "The smail Mail Transport Agent"" in Chapter 16, "The World Wide Web and Electronic Mail" ), you might want to back up the incoming mail files for each user. Many people tend to keep old and "important" electronic mail in their incoming mail spool, and it's not difficult to accidentally corrupt one of these files through a mailer error or other mistake. These files are usually found in /var/spool/mail.

8.1.1.1. Backing up to tape

Assuming you know what files or directories to back up, you're ready to roll. The tar command can be used directly, as we saw in the section "Section 7.1.2, "Using tar"" in Chapter 7, "Upgrading Software and the Kernel", to make a backup. For example, the command:

tar cvf /dev/rft0 /usr/src /etc /home

archives all of the files from /usr/src, /etc, and /home to /dev/rft0. /dev/rft0 is the first "floppy-tape" device--that is, for the type of tape drive that hangs off of the floppy controller. Many popular tape drives for the PC use this interface. If you have a SCSI tape drive, the device names are /dev/st0, /dev/st1, and so on, based on the drive number. Those tape drives with another type of interface have their own device names; you can determine these by looking at the documentation for the device driver in the kernel.

You can then read the archive back from the tape using a command such as:

tar xvf /dev/rft0

This is exactly as if you were dealing with a tar file on disk, as seen in the section "Section 7.1, "Archive and Compression Utilities"" in Chapter 7, "Upgrading Software and the Kernel".

When you use the tape drive, the tape is seen as a stream that may be read from or written to in one direction only. Once tar is done, the tape device will be closed, and the tape will rewind. You don't create a filesystem on a tape, nor do you mount it or attempt to access the data on it as files. You simply treat the tape device itself as a single "file" to create or extract archives from.

Be sure your tapes are formatted before you use them. This ensures that the beginning-of-tape marker and bad-blocks information has been written to the tape. At the time of this writing, no tools exist for formatting QIC-80 tapes (those used with floppy tape drivers) under Linux; you'll have to format tapes under MS-DOS or use preformatted tapes.

Creating one tar file per tape might be wasteful if the archive requires but a fraction of the capacity of the tape. In order to place more than one file on a tape, you must first prevent the tape from rewinding after each use, and you must have a way to position the tape to the next "file marker," both for tar file creation and for extraction.

The way to do this is to use the nonrewinding tape devices, which are named /dev/nrft0, /dev/nrft1, and so on for floppy-tape drivers, and /dev/nrst0, /dev/nrst1, and so on for SCSI tapes. When this device is used for reading or writing, the tape will not be rewound when the device is closed (that is, once tar has completed). You can then use tar again to add another archive to the tape. The two tar files on the tape won't have anything to do with each other. Of course, if you later overwrite the first tar file, you may overwrite the second file or leave an undesirable gap between the first and second files (which may be interpreted as garbage). In general, don't attempt to replace just one file on a tape that has multiple files on it.

Using the nonrewinding tape device, you can add as many files to the tape as space permits. In order to rewind the tape after use, use the mt command. mt is a general-purpose command that performs a number of functions with the tape drive.

For example, the command:

mt /dev/nrft0 rewind

rewinds the tape in the first floppy-tape device. (In this case, you can use the corresponding rewinding tape device as well; however, the tape will rewind just as a side effect of the tape device being closed.)

Similarly, the command:

mt /dev/nrft0 reten

retensions the tape by winding it to the end and then rewinding it.

When reading files on a multiple-file tape, you must use the nonrewinding tape device with tar and the mt command to position the tape to the appropriate file.

For example, to skip to the next file on the tape, use the command:

mt /dev/nrft0 fsf 1

This skips over one file on the tape. Similarly, to skip over two files, use:

mt /dev/nrft0 fsf 2

Be sure to use the appropriate nonrewinding tape device with mt. Note that this command does not move to "file number two" on the tape; it skips over the next two files based on the current tape position. Just use mt to rewind the tape if you're not sure where the tape is currently positioned. You can also skip back; see the mt manual page for a complete list of options.

You need to use mt every time you read a multifile tape. Using tar twice in succession to read two archive files usually won't work; this is because tar doesn't recognize the file marker placed on the tape between files. Once the first tar finishes, the tape is positioned at the beginning of the file marker. Using tar immediately will give you an error message, because tar will attempt to read the file marker. After reading one file from a tape, just use:

mt device fsf 1

to move to the next file.

8.1.1.2. Backing up to floppy

Just as we saw in the last section, the command:

tar cvf /dev/fd0 /usr/src /etc /home

makes a backup of /usr/src, /etc, and /home to /dev/fd0, the first floppy device. You can then read the backup using a command such as:

tar xvf /dev/fd0

Because floppies have a rather limited storage capacity, GNU tar allows you to create a "multivolume" archive. (This feature applies to tapes, as well, but is far more useful in the case of floppies.) With this feature, tar prompts you to insert a new volume after reading or writing each floppy. To use this feature, simply provide the M option to tar, as in:

tar cvMf /dev/fd0 /usr/src /etc /home

Be sure to label your floppies well, and don't get them out of order when attempting to restore the archive.

One caveat of this feature is that it doesn't support the automatic gzip compression provided by the z option (as seen in the previous section). However, there are various reasons why you may not want to compress your backups created with tar, as discussed later. At any rate, you can create your own multivolume backups using tar and gzip in conjunction with a program that reads and writes data to a sequence of floppies (or tapes), prompting for each in succession. One such program is backflops, available on several Linux distributions and on the FTP archive sites. A do-it-yourself way to accomplish the same thing would be to write the backup archive to a disk file and use dd or a similar command to write the archive as individual chunks to each floppy. If you're brave enough to try this, you can figure it out for yourself.

8.1.1.3. To gzip, or not to gzip?

There are good arguments both for and against compression of tar archives when making backups. The overall problem is that neither tar nor gzip is particularly fault-tolerant, no matter how convenient they are. Although compression using gzip can greatly reduce the amount of backup media required to store an archive, compressing entire tar files as they are written to floppy or tape makes the backup prone to complete loss if one block of the archive is corrupted, say, through a media error (not uncommon in the case of floppies and tapes). Most compression algorithms, gzip included, depend on the coherency of data across many bytes in order to achieve compression. If any data within a compressed archive is corrupt, gunzip may not be able to uncompress the file at all, making it completely unreadable to tar.

This is much worse than if the tar file were uncompressed on the tape. Although tar doesn't provide much protection against data corruption within an archive, if there is minimal corruption within a tar file, you can usually recover most of the archived files with little trouble, or at least those files up until the corruption occurs. Although far from perfect, it's better than losing your entire backup.

A better solution would be to use an archiving tool other than tar to make backups. There are several options available. cpio is an archiving utility that packs files together, similar in fashion to tar. However, because of the simpler storage method used by cpio, it recovers cleanly from data corruption in an archive. (It still doesn't handle errors well on gzipped files.)

The best solution may be to use a tool such as afio. afio supports multivolume backups and is similar in some respects to cpio. However, afio includes compression and is more reliable because each individual file is compressed. This means that if data on an archive is corrupted, the damage can be isolated to individual files, instead of to the entire backup.

These tools should be available with your Linux distribution, as well as from all of the Internet-based Linux archives. A number of other backup utilities, with varying degrees of popularity and usability, have been developed or ported for Linux. If you're serious about backups, you should look into them.[32] Among those programs are the freely available are taper and tob, as well as commercial programs like ARKEIA and Perfect Backup.

[32]Of course, this section was written after the author took the first backup of his Linux system in nearly four years of use!

8.1.2. Incremental Backups

Incremental backups, as described earlier in this chapter, are a good way to keep your system backups up to date. For example, you can take nightly backups of only those files that changed in the last 24 hours, weekly backups of all files that changed in the last week, and monthly backups of the entire system.

You can create incremental backups using the tools mentioned above: tar, gzip, cpio, and so on. The first step in creating an incremental backup is to produce a list of files that changed since a certain amount of time ago. This is easily done with the find command.[33] If you use a special backup program, you will most likely not have to do this, but set some option somewhere that you want to do an incremental backup.

[33]If you're not familiar with find, become so soon. find is a great way to locate files across many directories that have certain filenames, permissions, or modification times. find can even execute a program for each file that it locates. In short, find is your friend, and all good system administrators know how to use it well.

For example, to produce a list of all files that were modified in the last 24 hours, we can use the command:

find / -mtime -1 \! -type d -print > /tmp/filelist.daily

The first argument to find is the directory to start from--here, /, the root directory. The -mtime -1 option tells find to locate all files that changed in the last 24 hours.

The \! -type d is complicated (and optional), but it cuts some unnecessary stuff from your output. It tells find to exclude directories from the resulting file list. The ! is a negation operator (meaning here, "exclude files of type d"), but put a backslash in front of it because otherwise the shell interprets it as a special character.

The -print causes all filenames matching the search to be printed to standard output. We redirect standard output to a file for later use. Likewise, to locate all files that changed in the last week, use:

find / -mtime -7 -print > /tmp/filelist.weekly

Note that if you use find in this way, it traverses all mounted filesystems. If you have a CD-ROM mounted, for example, find attempts to locate all files on the CD-ROM as well (which you probably do not wish to backup). The -prune option can be used to exclude certain directories from the walk performed by find across the system; or, you can use find multiple times with a first argument other than /. See the manual page for find for details.

Now you have produced a list of files to back up. Previously, when using tar, we have specified the files to archive on the command line. However, this list of files may be too long for a single command line (which is usually limited to around 2048 characters), and the list itself is contained within a file.

You can use the -T option with tar to specify a file containing a list of files for tar to back up. In order to use this option, you have to use an alternate syntax to tar in which all options are specified explicitly with dashes. For example, to back up the files listed in /tmp/filelist.daily to the device /dev/rft0, use the command:

tar -cv -T /tmp/filelist.daily -f /dev/rft0

You can now write a short shell script that automatically produces the list of files and backs them up using tar. You can use cron to execute the script nightly at a certain time; all you have to do is make sure there's a tape in the drive. You can write similar scripts for your weekly and monthly backups. cron is covered in the next section.