# scdbackup-0.9.0 http://scdbackup.webframe.org scdbackup@gmx.net Thomas Schmitt http://scdbackup.sourceforge.net Purpose: backup large amounts of data on one or more CDs or DVDs, simple (therefore no excuse not to do the backup), no special tool needed for reading the backup Formats: ISO9660 file systems (readable after : mount ) afio archives (readable by : afio ... /dev/cdrom ) star archives (readable by : star ... = 0.1.2 this means you need a split directory. The buffer file should not be necessary if you got modern hardware. (It is useful for advanced purposes, nevertheless.) After you decided to have a split directory, find a suitable filesystem and create a new empty directory there. For a buffer file you would just have to choose a name in an existing directory on a suitable filesystem. Suitable here means : with enough free space. You must have separate directories resp. files for CD and DVD unless you only got one single burner which is in charge for both media types. I strongly recommend to use different names in any case. Set the access permissions of the newly created split directories to rwx for the interested user(s). (In most frigthening case: chmod a+rwx ) Make sure the directory of an eventual bufferfile offers rwx-permissions to all those users. Each single scdbackup installation (i.e. for security reasons : each user) should use own addresses for their buffer directories and buffer files. ------------------------------------------------------------------------------ Installation To achieve a clear security situation each user should have an own separate scdbackup installation. Steps 1 and 2 must be done by the particular user who shall use that installation. This is not necessarily the superuser but rather the owner of the backup data. The superuser may follow these steps for the installation of an own, very private backup system. But superuser authority will elsewise only be needed for a few optional activities. If programs cdrecord and/or growisofs are installed properly on the system and may be used by the installing user for burning, then there is no urgent need for superuser actions at all. Each of the steps may be repeated to restore original software and permission settings or to edit the configuration. --- Step 1 : Copy the archive to a directory where you can create a sub directory scdbackup-0.9.0, go there, unpack the archive and go to the sub directory inst newly created by this. For example use $HOME/scdbackup : mkdir $HOME/scdbackup cd $HOME/scdbackup tar xzf $HOME/downloaded_stuff/scdbackup-0.9.0.tar.gz cd scdbackup-0.9.0/inst If the superuser is doing this, it is also absolutely necessary to do chown root ../../scdbackup-0.9.0/*/* ../../scdbackup-0.9.0/* \ ../../scdbackup-0.9.0 and best to issue an appropriate chgrp command, too. --- Step 1b (only needed when upgrading from an older version) : If you already had installed another version of scdbackup, you may now copy some history data and the old configuration as a template into the new directory : ./FETCH_CONFIGURATION This refuses to overwrite any files so it only makes sense in a freshly unpacked directory. Any users afterwards are still configured to use their old installation. If you maintain incremental backup configurations and suffered from long planning times you may be interested in reading the first paragraph of appendix INODE below. Only if you programmed own applications which make use of inner parts of scdbackup previous to version 0.8.5, only then you should now read appendix DIRECTORIES. The mere use of commands scdbackup* or sdvdbackup* within your programs will continue to work without ./COMPATIBILITY_LINKS, of course. Unless the user id number is 0 the following scripts write the files .scdbackup_dir and .scdbackup_lang into the user's $HOME-Directory. They eventually write file .scdbackup_userid into the user's $HOME-Directory. --- Step 2a if you got a CD writer (for DVD writer only, see Step 2b below) : While the CD-Recorder is not busy, run : ./CONFIGURE_CD Several questions have to be answered: What ISO 9660 formatter program to use ? (... found programs ...) What CD burn program to use ? (... found programs ...) Address of CD recorder ? (... found addresses ...) Speed ? (e.g. 2) Automatically erase CD before writing ? (yj1/n0) Large file split directory ? ( "-" = exclude large files) Buffer file name ? ( or "-" or "-afio_compress" ) Estimated capacity for a compressed afio CD ? (eg. 1024m) Exclusion list of directories for system backup ? ( With other shells than bash : Your user id number ? ) If no disk buffer (see above) is needed then answer the question "Large file split directory" by a single "-" or if advised by the message text: "-xorriso_cut_out". "Buffer file name" should be "-afio_compress" for true burnproof hardware and media, else enter "-" or a filename. The "Exclusion list of directories for system backup" is only of interest if you are superuser right now. As normal user just press the Enter key. --- Step 2b if you got a DVD writer (works with or without Step 2a) : If neither command growisofs nor cdrskin nor wodim nor cdrecord is not found by your shell then you will have to install dvd+rw-tools as described in Appendix DVD. While the DVD writer is not busy, run : ./CONFIGURE_DVD Several questions have to be answered: What ISO 9660 formatter program to use ? (... found programs ...) What DVD burn program to use ? (... found programs ...) Address of DVD recorder ? (... found addresses ...) Speed ? (e.g. 2) Automatically erase DVD before writing ? (yj1/n0) Large file split directory ? ( "-" = exclude large files) Buffer file name ? ( or "-" or "-afio_compress" ) Estimated capacity for a compressed afio DVD ? (e.g. 8000m) Exclusion list of directories for system backup ? ( Non-bash users also may be prompted for a unique user id number ) If no disk buffer (see above) is needed then answer the question "Large file split directory" by a single "-" or if advised by the message test: "-xorriso_cut_out". "Buffer file name" should be "-afio_compress" for true burnproof hardware and media, else enter "-" or a filename. The "Exclusion list of directories for system backup" is only of interest if you are superuser right now. As normal user just press the Enter key. --- Step 2c. (Only with private installation for superuser) For his own private installation of scdbackup, the superuser has to run : ./ADD_USER Make no mistake : only if you are already superuser now and only with the installation which shall do the superuser's backup jobs and where no other user is permitted. After Configuration Permission settings after a first configuration will not allow others to use your scdbackup installation. This is for your security. Stay with it. The commands of scdbackup are now accessible in subdirectory ./cmd of $HOME/scdbackup/scdbackup-0.9.0 . Make a first test : $HOME/scdbackup/scdbackup-0.9.0/cmd/scdbackup_home -version should reply "scdbackup 0.9.0 2008.06.29.143124" (timestamp varies). You may now add $HOME/scdbackup/scdbackup-0.9.0/cmd to your PATH variable : export PATH="$PATH":"$HOME/scdbackup/scdbackup-0.9.0/cmd" and then execute commands like scdbackup_home -version There are startup scripts like .profile or .bashrc where you may add this directory path permanently. Ask your sysadmin. If this is an upgrade from scdbackup-0.8 or older then the superuser will have to remove the old commands in /usr/bin (or wherever they are) or to overwrite them as described in the next paragraph. If this is your first encounter with scdbackup then i advise you to next read "Usage" or just try $HOME/scdbackup/scdbackup-0.9.0/cmd/scdbackup_home (without "-version"). The eventual superuser activities can wait for an hour - if cdrecord and DVD writing device are in proper state, that is. Superuser Part of Installation The superuser may decide to copy the scdbackup commands to a public command directory rather than to teach his users how to permanently set their PATH. These command scripts are quite small and merely direct the users to their own scdbackup installations. Their only hazard is to get tricked into a fake scdbackup installation by the user's own environment variables SCDBACKUP_DIR or HOME or by the user's own configuration file $HOME/.scdbackup_dir . Regular installations of other users should repel such an attempt, anyway. With upgrades from scdbackup-0.8 or older, the superuser has to make sure that the old installed commands get removed or overwritten by the new ones. Commands "which scdbackup" and "which scdbackup_sys" should tell where they are located. A mistrusting superuser may decide to install the commands manually following this template (copy commands to /usr/local/bin and /usr/sbin ): target=/usr/local/bin chmod a-w ../scdbackup-0.9.0/cmd/* cp ../scdbackup-0.9.0/cmd/* "$target" chown root "$target"/scdbackup* "$target"/sdvdbackup* chmod a+rx,u+w,go-w "$target"/scdbackup* "$target"/sdvdbackup* chmod go-x "$target"/scdbackup_sys "$target"/sdvdbackup_sys (have a look into /usr/local/sbin whether there is s*dbackup_sys to be removed) But there is a script which does this more comfortably, reliably and avoids any "*" in the commands. If the superuser intends to use scdbackup for own backup runs, then there is no reason to particularly mistrust the following script. (Bug reports welcome) --- Step 3 (optional, see possible reasons above) Become superuser now and run (within directory scdbackup-0.9.0/inst) ./SUPERUSER_CONFIGURE This script copies the command scripts scdbackup* to a public command directory like /usr/bin or /usr/local/bin if CD writing is actually configured. If DVD writing has been configured, then the commands sdvdbackup* are copied to that directory. You will be asked for the public command directory where to install the scripts Install directory ? (e.g. /usr/bin , "-" = do not install) The user should afterwards check whether scdbackup_home -version yields the same result as in the first test : $HOME/scdbackup/scdbackup-0.9.0/cmd/scdbackup_home -version There is no need to repeat step 3 with further user's installations. Permission to use the recorder: Depending on the burn program used, the permission to use a CD/DVD recorder has to be achieved in different ways: cdrecord (package cdrtools) Needs to be run by the superuser or to be owned by the superuser and treated with chmod u+s growisofs (package dvd+rw-tools) The user needs rw-permission for /dev/srN resp. /dev/hdX. Uses device addresses but no SCSI numbers. chmod u+s is deprecated. wodim (package cdrkit) Either to be used like cdrecord or to be used under normal user id with rw-permissions on appropriate /dev/scdN resp. /dev/hdX files. cdrskin (packages cdrskin The user needs rw-permission for /dev/srN resp. or libburn) /dev/hdX. The superuser should perform cdrskin --devices to learn about the device files for chmod +rw Application of chmod u+s is strongly deprecated. xorriso (packages xorriso Same preconditions as with cdrskin. Uses device or libisoburn) addresses but no SCSI numbers. xorriso is a burn program as well as an ISO 9660 formatter. If the superuser does not have an own scdbackup installation yet, then this would nearly be the time to start -as superuser- with step 1 in order to get one. But first read this : Superuser Planning The superuser will most likely use format afio unless there are user data to take care of, which clearly benefit from ISO 9660 format. By help of script star_as_afio_wrapper one may alternatively employ the archive format of program star. (See example in appendix WORKERS.) The idea is to have a rather slim system backup, which is made with scdbackup_sys Be aware that scdbackup_sys does not create a recovery system but just records the files (even quite special ones) in an afio archive. This conserves the system configuration, but for re-installing you need to do a base installation of Linux first. The base installation has to be compatible with the system you made the archive from. So make copies of your Linux CDs to have a compatible system at hand when needed. You may also use program star as intended by its author to make a full backup of the system partitions. This will not need copies of the originally installed Linux distribution media. See appendix RECOVERY for details of disaster precautions and recovery. Alternatively you may choose to use a disaster recovery system like mkCDrec ( http://mkcdrec.ota.be/project/index.html ) which tries to save you as quick as possible from the consequences of big accidents. Nevertheless, if you cannot get the old hardware running again it might be impossible to use system backups of any kind flatly. Then you have to install a completely new operating system and may use the old system backup only for as source of configuration files. I myself never lost a complete system but had occasions when an older reference copy of the system files came in very handy. User data which rely on exact file attributes should not be stored as ISO9660. For such data, one should better use scdbackup_afio . Usually there are several kinds of data on a Linux disk, which should be backuped: Persistent system objects, often with special attributes and permissions Installed application packages (often below /opt) User created or collected data (often below /home) Other kinds of data do not need backup or would even make problems : Volatile system objects (e.g. /proc) Temporary files (e.g. /tmp or $HOME/.netscape/cache) Removable filesystems (/mnt /floppy /cdrom) During installation, you will be asked for "Exclusion list of directories for system backup". This list should contain all directories under / , which do not contain persistent system data. Example (have a look in your own / directory for more candidates) : /opt /home /mnt /floppy /cdrom /cdrw /tmp /proc /sys All other subdirectories of the root directory will be covered by the system backup. Among the excluded directories, only /opt and /home need backup. If there is only one real user who has not got a giant amount of data, a combined backup of /opt and /home is appropriate. Since my Netscape cache is quite large, i exclude it. scdbackup_afio /home /opt -not /home/thomas/.netscape/cache If there are very large amounts of data, it will be convenient to have separate backups. These backups can be run with different frequency. /opt will not need to be backuped as often as /home/projects . scdbackup_afio /opt scdbackup_afio /home -not /home/projects /home/thomas/.netscape/cache scdbackup /home/projects Real users (like "thomas", "guest1", "guest2") should do separate backups with scdbackup_home or scdbackup_afio $HOME Exclude their $HOME directories from the general data backup scdbackup_afio /home \ -not /home/thomas /home/guest1 /home/guest2 Next you should make a time schedule, when to do which backup. It is generally a good idea to have at least three older backups. From time to time a permanent backup could be made on CD-R in order to be kept forever. On the other hand ... be aware what traces you leave for future archeologists. Permission settings after an installation will not allow others to use a scdbackup installation, unless these restrictions are lifted by ./SET_PERMISSIONS (see below "Access Permissions") or unless laxer settings have been fetched during ./FETCH_CONFIGURATION. Interested and authorized users then go to the installation directory and perform ./ADD_USER . Now go and -as superuser- begin with step 1 of the installation procedure. If the superuser did not use scdbackup before , execute as step 1b : ./SUPERUSER_FETCH_CONF /home/thomas/scdbackup/scdbackup-0.9.0 to get the previously made settings of user "thomas" as templates for own ./CONFIGURE_CD resp ./CONFIGURE_DVD . Else ./FETCH_CONFIGURATION is ok. In any case : check what you confirm with ./CONFIGURE_CD and ./CONFIGURE_DVD . Take care not to allow others to access your eventual disk buffer objects. Best is to have none, if one can live without them. ----------------------------------------------------------------------- Usage scdbackup offers different formats for the data on media. The data format preferable for user data files is in most cases ISO 9660. The media are mountable and the files may be used directly by any program that can handle files which offer no write access. This format is also most likely to be accessible on other operating systems. In the most simple case, command scdbackup_home is sufficient. For arbitrary data locations use command scdbackup . As said, this is ok for directories trees and data files of a single user, where write permissions are general and simple. mkisofs with version numbers below 2.0 will not reliably cope with symbolic links. Independently of eventual weaknesses of the ISO formatter program there may also arise problems with the filesystem drivers. Not all of them show the w-permissions of directories correctly. Another disadvantage is that scdbackup does not offer data compression together with ISO format. For such purposes, scdbackup offers the format of the reliable archiver afio or the format of Joerg Schilling's backup program star. The commands scdbackup_afio and scdbackup_sys produce these formats. Backup Commands The backup creating command scripts of the scdbackup-system all accept the options described by cd_backup_planer -help | less but usually it is only necessary to provide some directory or file names. See also doc/examples.html for commented usage examples. ISO9660 Filesystems These filesystems can be mounted on devices like /dev/cdrom or /dev/dvd . They also can be exchanged with other systems like Windows or Mac. scdbackup_home a command that backups the user's $HOME directory on CD except the Netscape cache. No arguments needed. sdvdbackup_home DVD version of above. scdbackup a command that backups the files and directories given as arguments by the user. These objects may be renamed by giving the target name before '=' and the source name. Example: rename /usr/home (on disk) to /home2 (on CD) /home2=/usr/home Subdirecories and files may be excluded by writing them behind option -not. Example: exclude three directories -not /home/guest /home/kindergarden /usr/home/guest Complete example: scdbackup /home /home2=/usr/home -not /home/guest Backup /home and /usr/home . But use /home2 on CD as name for /usr/home and omit subdir /home/guest sdvdbackup DVD version of above. Directory access permissions may be restored by the compressed shell script /added_by_scdbackup/DIR_PERMS.sh.gz on the last volume of such a backup. (See also examples.html#restore_incr ) This may be necessary because directory access permissions on a mounted CD may not be the same as they were during the backup. afio Archives afio archives can be read with the CD-ROM or DVD drive while the media is inserted but not mounted. Check Archive: afio -tv /dev/cdrom Re-install (*beware*, only if you need): cd / afio -ivZ /dev/cdrom Example : Extract X11 configuration files to a harmless place No need to be superuser for that ! mkdir $HOME/system_reference_copy cd $HOME/system_reference_copy afio -ivZ -y 'etc/X11/XF86Config*' /dev/cdrom Now you may compare them with those of your current system diff etc/X11/XF86Config /etc/X11/XF86Config | less Read for details : man afio . By help of script star_as_afio_wrapper one may alternatively create archives in the format of program star, provided that program is installed. For an example see appendix WORKERS. scdbackup_afio like scdbackup but with afio archives rather than ISO9660 file systems. If there is a buffer file or if compression was set during configuration then the archives contain compressed files. scdbackup_afio /home -not /home/guest sdvdbackup_afio DVD version of above. scdbackup_sys a command that backups the root directory as afio archives. Some well known non-system directories and some trash directories are excluded. (My apologies to all those who work hard on programs that fill in the "trash" :) Arguments can be given to include or exclude other directories. The exclusion list is set by CONFIGURE . Example: /cdrom /cdrw /floppy /home /mnt /opt /proc /tmp Example (to be executed as superuser): include /opt (which usually is excluded) but omit /home2 and /usr/trash scdbackup_sys /opt -not /home2 /usr/trash sdvdbackup_sys DVD version of above. If the size of the resulting afio archives exceeds 650 MB then they are split into pieces of 650 MB and stored on several CDs which form a multi volume archive. Such archives have to be read from their first CD up to the one which contains the desired data. If a CD is missing then all following ones may be unreadable. DVD split size is 4480 MB. The main advantage of compressed multi volume archives is their most effective use of CD space. Single volume compressed archives usually need quite a lot of spare space. The ability to create multi volume archives allows you to risc larger single volume ones. Worst case is an additional CD with a few MB on it. (You may adjust the input size by -max_size number ) It is convenient to have a copy of the program bin/raedchen (e.g. on floppy) to be able to concatenate the parts of such an archive. Check: raedchen -quiet -pipe /dev/cdrom 650m "cat" \ -post_pipe_command "eject /dev/cdrom" | afio -tv - Restore: raedchen -quiet -pipe /dev/cdrom 650m "cat" \ -post_pipe_command "eject /dev/cdrom" | afio -ivZ - This is necessary because the CD returns some more bytes than have been written. The false extra bytes spoil at least the file which is stored at the end of the first and the beginning of the second CD. With DVD use : raedchen -quiet -pipe /dev/dvd 4480m "cat" \ -post_pipe_command "eject /dev/dvd" | afio ... If you are completely without scdbackup, then you will have to use a shell loop for concatenation. Like this ( 332800 = 650 MB / 2 kB ) : ( while test -n 1 do dd if=/dev/cdrom bs=2048 count=332800 || exit 0 eject /dev/cdrom echo 'Next media please' >&2 read dummy done ) | afio ... There is an appropriate option to make the verify commands handle such overflown media. (see below) The actual CD overflow size for afio is set by configuration file scdbackup_media_cap_value or shell variable SCDBACKUP_MEDIA_CAP . For DVD it is set in sdvdbackup_media_cap_value resp. SDVDBACKUP_MEDIA_CAP. If you change the defaults 650m / 4480m take care to record the new size on paper or on the raedchen floppy. Restartability All the above backup commands can be executed with first argument -resume and an optional number as second argument. They then restart an existing backup script at the backup piece with the given number. A dash (-) preceding the number will cause only this particular volume to be created and the script to end afterwards. If the number is missing (or the second argument is "AUTO") then the script resumes at the piece depicted by the status file ({skriptname}.pieceno). scdbackup , scdbackup_sys and scdbackup_afio additionaly recognize these options if one of them is given as first argument : -prepare_only prepares for -resume but does not create CDs -last_volume_count prints number of volumes of most recent backup See script outer_loop for an example how to use them. Verifying CDs and DVDs First of all : the verification command described here does *not* ensure that the backup contains what you *want* to backup. Wether you and scdbackup did it right has to be proven by looking at the meaning of the content i.e. whether the byte size is plausible and whether expected filenames are on CD (locatable by ASKME). This verification nevertheless ensures that the data stream has been transfered to the CD successfully and can be read without alteration. (There is only a very very remote chance of 1 to 3e38 that an alteration will be unnoticed.) Therefore it checks the CD recorder, the CD media and the CD reading device. In order to be verificable, a CD must have been created by version 0.8 -or more modern- of scdbackup with standard content of file scdbackup_make_checksum_value . With these settings the commands scdbackup, scdbackup_home, scdbackup_sys and scdbackup_afio record the length and a 128 Bit MD5 checksum of every backup volume that is written. This record is stored in a recordfile on your disk and may be used later to checkread the CD. Beginning with version 0.8.3 the record is also stored as a checksum tag at the end of the volume itself. scdbackup_verify Reads raw data from a CD and checks whether these are the same as the ones handed over to cdrecord. It reads CD data from /dev/cdrom and compares them with the most recent volume record. scdbackup_verify -auto tries to gather records from any source and checks whether one of them matches or obviously indicates a mismatch. Other data sources (CD devices) or older records may be used by options to scdbackup_verify. E.g.: scdbackup_verify /dev/cdrom -auto For backups made with scdbackup 0.8 to 0.8.2 you need to keep the record list. Backup it. See output of scdbackup_verify -help resp. file scdbackup_verify_help for a description of options, the recordfile's name and more examples. The command reads the whole CD and reports its result by a text on stderr as well as by its exit value: * "OK ..." and an exit value of 0 if the verification process was successful. * Various complaints in CAPITAL LETTERS and exit value 1 if verification fails (i.e. damaged or confused CD). sdvdbackup_verify DVD version of above. Uses a different recordfile ! The record list usually is stored in file logs/backup_log resp. logs/backup_dvd_log below scdbackup's installation directory. With normal installation settings you can only read that list if you are the same user as the one who wrote the backup. The list's address may be permanently changed in files scdbackup_checksum_list_value resp. sdvdbackup_checksum_list_value and temporarily overriden by exporting shell variable SCDBACKUP_CHECKSUM_LIST resp. SDVDBACKUP_CHECKSUM_LIST . Checksum computation and checksum verification are based on cd_backup_planer options -filter_md5 , -compare_checksum and -search_md5 . The format of checksum records and tags is described in appendix VERIFY below. For multi volume afio archives (see above) you will have to give scdbackup_verify the media capacity as it was set when the backup was written. Like : scdbackup_verify /dev/cdrom -auto - - 650m sdvdbackup_verify /dev/dvd -auto - - 4480m If verification fails directly after recording then you may redo the affected volume by a -resume run with the appropriate volume number. Like sdvdbackup -resume 3 One should verify stored backups in regular intervals to get aware of damage or deterioration. scdbackup_verify -auto If verification fails with such a check then you should consider to replace all volumes of the whole backup by a new backup. If this is not possible then you may make several tries to copy the media image to a reliable disk and check it there in the hope that the copy attempt delivered the correct image by incident (this does happen with small damages). dd if=/dev/cdrom of=/tmp/hopefully_sound_image_file bs=2048 scdbackup_verify /tmp/hopefully_sound_image_file -auto If successful, copy the image to new media: cat /tmp/hopefully_sound_image_file | scdbackup -pipe_to_media With DVD one should eventuellay truncate the media image to the known size plus three extra MB for eventual checksum tags or blocklists. Like: dd if=/dev/cdrom of=/tmp/hopefully_sound_image_file bs=1M count=1718 after sdvdbackup_verify /dev/cdrom -auto_end reported a match or mismatch after "1715m bytes". If the payload size is not known from the previously failed attempts to verify the media, then one should curb the output: dd if=/tmp/hopefully_sound_image_file bs=1M count=1718 | \ sdvdbackup -pipe_to_media Be aware that some old filesystems may be unable to take a file of 4200 MB. If you expect your backups to be not reproducible during the intended storage period (i.e. if they are an outsourced data archive rather than a backup for a life disk), then you should consider to record them with block checksum lists and as several identical copies. See appendix REDUNDANCY. Blanking CD-RW or DVD-RW , Erasing data from media If a CD-RW has been used and shall be re-used, it is necessary to blank it. The same is true with DVD-RW under certain circumstances. If blanking comes into effect it damages the data on the media, of course. If automatic blanking is configured as "1" (or "y") then with each backup the media get treated according to their automatically recognized type and state. Usually classification and recognition of media types is reliable. If nevertheless the type cannot be recognized or if it is not suitable for the backup command, then the backup aborts. It may be resumed after the media is brought to writeable state manually and automatic blanking is temporarily disabled: export SCDBACKUP_BLANKEN=0 scdbackup -resume Not only in case of emergency but also with recognizable media it may be worthwile to do blanking separately from backup writing. Depending on the blank options, on type of the burner, and on the writer software blanking lasts some time. So it is convenient to have some writeable media ready. Also one might sometimes want to apply unusual blanking options. With DVD+RW there is no blanking required for making media writeable again. Nevertheless, the "shred" option can be very helpful to ensure your privacy when reusing media. DVD-RW media for use with programs growisofs or cdrskin should be blanked "force" once to bring them into a mode with the misleading name "Restricted Overwrite". After that, growisofs and cdrskin will behave with them like with DVD+RW. DVD-RW for use with cdrecord need to be blanked before re-use just like CD-RW. The same is true for growisofs DVD-RW media which were written unformatted from unused state or have been de-formatted to mode "Sequential". "shred" is a blanking option which does not necessarily make media re-usable. It rather causes the media to be filled with random data. After that, the previously stored data are unreadable by common means. scdbackup_blank asks you to insert the CD-RW and press 'Enter'. If an argument is given, this is used as blanking option. Usual options are "fast" or "all". See output of option blank=help of the burn program (cdrecord, wodim, or cdrskin). Alternatively, there is option "shred" (see above). Without argument blanking is set to the option in file scdbackup_blanken_value. "1" or "y" mean option "fast". If no other clue is found option "fast" will be used. Note: If a blanking directive blank=... is part of file scdbackup_cdrecord_value then no other blanking option will work. sdvdbackup_blank DVD version of above. Its behavior depends on the media type and on the writer program (growisofs or cdrskin vs. cdrecord or wodim). For DVD+RW only option "shred" does anything senseful. For cdrecord, wodim and DVD-RW : see scdbackup_blank For growisofs, cdrskin and DVD-RW : Options "all", "full" and "sequential" lead to mode "Sequential". All other options lead to convenient mode "Restricted Overwrite". Both commands try to find out the type of the media in the drive. This depends much on output of programs dvd+rw-mediainfo, cdrecord, wodim or cdrskin which is not explicitely mentioned in their documentation. If this classification fails to yield a usable media type, the user is asked to confirm one of the types "CD-RW", "DVD-RW", "DVD+RW". The classification attempt as well as the eventual user inquiry can be avoided by the third argument to the program. Like : sdvdbackup_blank all - DVD-RW CAUTION : Best is not to use these commands with one-time writable media ! E.g. if a DVD-R is not recognized properly or declared by the user to be a DVD+RW, then option "shred" will use it up. After a small backup on DVD it is easy to read old remains from larger backups until they really get overwritten. Therefore before re-use of confidential media apply sdvdbackup_blank shred . Do it multiple times if privacy is a big issue. The "shred" option tells what will happen next and require another Enter keypress as confirmation. Then it eventually applies appropriate blanking. It writes the amount of random data given by the second argument to the program. E.g.: sdvdbackup_blank shred 4.7e9 If that second argument is missing then the configuration settings in file scdbackup_media_cap_value resp. file sdvdbackup_media_cap_value apply. Usually the preset size of 4480m bytes should be sufficient to hit all databytes on a DVD. Nevertheless you may derive a size from the maximum size given by growisofs during write. Subtract 200 (e.g.: shred 4700372792 ). Remains of backups made with version 0.8.3 or newer will get reported by command scdbackup_verify -scan_print . Like : - 1_1 A40907.115444 4699824167 7ec3be0efef3f27dd7ec90216d2ee3b4 which tells that backup data may be stored up to byte 4699824167. Blanking option "shred" produces a checksum record with name "shredded" and therefore together with the appropriate s*backup_verify command is suitable as a quality check for the media. A large organisation, which presumes to have many enemies, prescribes additional runs with a character and its complement. It also demands verification of the random write result. The said organisation does not allow any erasing of "TOP SECRET" media but demands their complete physical destruction. Although i am in doubt whether the complement achieves the desired effect with CD/DVD media, there are two blanking options "character" and "complement" which perform the prescribed runs. A compliant procedure would be : export SCDBACKUP_BLANK_CHARACTER="U" sdvdbackup_blank character 4.7e9 sdvdbackup_blank complement 4.7e9 sdvdbackup_blank shred 4.7e9 sdvdbackup_verify The character written is a capital "U" (= 01010101). By variable SCDBACKUP_BLANK_CHARACTER a different normal letter or digit may be chosen. Better do not use any special characters. Incremental Backups With larger amounts of data it becomes annoying or even impossible to backup the old data at each new run. To avoid this (within certain limitations) one may use an incremental backup scheme. Such a backup consists of a base backup which might be quite old but contains a full snapshot of all files without regarding their age. On top of the base backup there are update levels which provide more and more short-term updates. This saves a lot of time and CDs but provides similar security as a set of full backups. Invest some of the savings into increased backup frequency and also keep outdated backup CDs a bit longer. This is to compensate the media redundancy which is reduced with incremental backups. It has to be emphasized that correct restoring of a multi-level incremental backup is much more complicated than with a full backup. Consider your potential future situation, when you might really need the full content of your backup. Weigh whether avoiding the repeated workload of a full backup really justifies the additional workload in the future crisis situation. Begin with incremental backups if you have to leave out full backups because you cannot afford them. Initially a backup configuration needs to be created. It records all parameters of the backup, keeps track of the times and states of the backup area when backups have been made. Also the creation process starts the initial base backup which is called -level 0 . Choose a configuration name that can be used to create a new directory. You will need some disk space there for the state records (usually below 1% of the backup area size). Let us assume you choose $HOME/my_backup : $ scdbackup ...the usual arguments for your backup ... \ -conf_dir $HOME/my_backup -level -create_configuration After that creation, only the name of the configuration and the desired update level have to be given with the scdbackup command. Next day one could execute this command to get a backup of the files that are new or changed since the -level 0 backup : $ scdbackup -conf_dir $HOME/my_backup -level 1 Options -conf_dir and -level are explained in output of cd_backup_planer -help . There also is an example of a scheme. Several independant configurations may be defined. Distinct configurations do not interfere even if they share data sources. Not only scdbackup but also scdbackup_home and scdbackup_afio may be used for defining and running incremental backups. The state records in the configuration directory consume some disk space although they get compressed by gzip. You should watch the size of file $HOME/my_backup/level_0/content_file_list.gz during the -level 0 run and abort it if you cannot afford the space needed. While watching the size do not get worried by the output delay of the gzip pipe. Any substantial size change will get visible in time. If you cannot afford the disk space, you may disable state recording by using option -disable_safe_leveling together with -level -create_configuration. But be aware that file movements and deletions cannot be detected that way and will therefore not be included in the higher backup levels. Such a backup is still a good source for recovering from limited data accidents but it may be unable to accurately reconstruct the most recent state of your backup area. The default settings of scdbackup configure a method which depends on the traditional behavior of inode numbers (POSIX "file serial numbers"). If your filesystem turns out to be non-compliant with tradition then you will get suspiciously large backups at higher levels. Please inform me about the type of the affected filesystem and refer to appendix INODE for usage of the old checksum based method. If you ever need to re-install an incremental backup with multiple levels you will begin with copying level 0 first (e.g. by use of cp -dR ). Then you will remove the files which had vanished when level 1 was made. Then you will copy level 1. This is repeated with all higher level backups. For the removal job, there is a compressed shell script /added_by_scdbackup/VANISHED.sh.gz on the last volume of each backup level. Don't run it for fun ! (It is interesting to have a look at its content, nevertheless.) See examples.html for an example restore procedure. More detail information about state recording can be found in the output of cd_backup_planer -help . Look for "re-install an incremental backup" and also refer to the program options mentioned there. The decision whether to put a file object into an incremental backup depends on the following properties : mtime (display by ls -l), ctime (by ls -lc), eventually file content checksum (if enabled and some ancestor directories have young time stamps), eventually the inode and device numbers (if enabled), an economy consideration which eventually includes whole directories if there are many small files to be included. Backup Information Scripts Each backup CD contains a shell script named ASKME which can tell on which one a given fileaddress may have ended up in the backup. Since it only knows the fileaddresses given to the formatter program, this script cannot always tell for sure whether a file was actually present at the time of the backup. But it can tell the CD where one has to look. ASKME only needs an bash or ksh shell but no installation of scdbackup. See text askme_help for usage information. In case of incremental backups the information scripts of all older backup levels are also present on each CD. Because they can become very large and hardly usable, they are by default truncated to stubs which can just identify backup date, level and volume. To be usable for searching, the backup configuration has to be created with option -info_script -full or all level runs have to be done using this option. For incremental backups with default stubs, refer to scdbackup_askme in the next paragraph. With ISO9660 formatted backups, the script is located in the root directory of the backup volume. With archive formatted backups (afio, star) it is the first item of a backup volume. Its name and address vary with the installation settings at backup time. One may obtain the address within the afio archive by afio -t /dev/cdrom 2>/dev/null | head -1 which should reply something like u/thomas/scdbackup/scdbackup-0.9.0/tmp/ASKME_1000.1.z (omit 2>/dev/null in insatisfying reply.) The script should run on nearly any shell that is around but it has the disadvantage to be relatively slow. With multilevel backups of large disks one may have to wait minutes for an answer. Backup Information+Restore Tool Command scdbackup_askme provides a binary program which can read the ASKME scripts and perform the searches much faster. Also it can search for text pieces (like fgrep) or for regular expressions (like grep). The main advantage of scdbackup_askme, nevertheless, is its ability to read content lists and vanish lists from incremental configuration directories or from the last volume of each backup level. Beginning with scdbackup version 0.8.7, non-incremental ISO-9660 backups too have a content list on the lasti volume. These lists hold more detailed information about the backup content than the ASKME scripts. They are able to surely tell whether a particular file was backuped (whereas ASKME can often only tell about a directory above the file's address). The content lists are able to tell the address of a file on the backup media (whereas ASKME can only tell the original address on disk). Based on content lists it is possible to have scdbackup_askme perform restore operations from ISO-9660 backup volumes. Both commands scdbackup_askme and sdvdbackup_askme may be pre-configured by startup files. See below "Startup Files for Information+Restore Tool". Examples (see text doc/scdbackup_askme_help for options and more information) Using the level ASKME scripts from backup media (ISO format): scdbackup_askme /cdrom/ASKME -levelall -search_grep '/trees/.*oak' scdbackup_askme /cdrom/ASKME -levelall -search_sh '/home/*/trees/*oak*' Using an incremental backup configuration on disk scdbackup_askme $HOME/my_backup -levelall -dialog After the backup state is read, the user will get prompted for options or search texts. Caused by option -search_sh those search texts will be compared as shell parser patterns against the source addresses. In this mode there is a current working directory which is addressed by search texts which do not begin with '/'. The special file names '.' and '..' are recognized. -pwd -cd:/home/*/trees *oak* */*oak* ../*oak* -search_shpat *trees/*oak* -search_shname *oak* -page:20 *oak* -search_shpat makes '/' match *,? or []. This allows to freely search for file name snippets. -search_shname looks into any directory and compares the single file names with the search text. Like this shell command: find / -name '*oak*' -page:20 causes output to pause after each 20 lines. The user gets asked for pressing Enter. Input @ suppresses this paging temporarily. input @@ suppresses output, Input @@@ aborts. Any other input is taken as option or search text and aborts the current search. Independend of the search mode there are options -ls and -ls_d which allow a shell pattern search with abbreviated output. They are similar to shell commands ls and ls -d : -ls:.. -cd:.. -ls_d:*tree* All searches may be performed on the target addresses, too. -search_target For the following examples switch back to shell parser search mode -search_sh To see the current settings -status To see all possible settings -status:long To see the help text -help To restore single files or whole directory trees from ISO-9660 backups, restoring first has to enabled by setting a -restore_mode . This one protects existing files on the disk and allows to continue trying after restore problems -restore_mode:overwrite_off:on_error_ignore The program further needs to know from where to read the data and it is convenient to have it operate the according drive via programs eject and mount -restore_from:/cdrom -media_mode:load_eject:mount_umount The original data are protected. So, in order to achieve a full restore result, the restore gets directed to a copy location -restore_skip:/home/thomas -restore_to:/home/test/thomas Now directory tree /home/thomas/gifs gets restored to that copy location -restore:/home/thomas/gifs The user will get prompted for the needed media. It is possible to skip some if they are missing, but of course that will result in an incomplete restore. The result will be a directory tree /home/test/thomas/gifs . Above operations may as well be performed using incremental information from backup media (ISO format). They are stored on the last volume of each level. scdbackup_askme /cdrom/added_by_scdbackup -levelall \ -media_mode:load_eject:mount_unount -dialog The user will get prompted for those last volumes. User input "@@@" confirmed by user input "y" indicates that no more levels are available. Then dialog starts as with the previous example. Restore runs which have been aborted for some reason may be resumed at the last restore position by: scdbackup_askme -resume One may also depict an explicit start level scdbackup_askme -resume:1 and additionally an explicit start volume scdbackup_askme -resume:1:4 Option -resume does also work in dialog but only for restore operations which have been performed within the same dialog session, or for those which were skillfully prepared by options -resume_last_* and -resume_opt_text. Aborted -resume runs may also be resumed by -resume. If the abort left a file object on disk which is neither in its pre-restore state nor restored correctly, then option -resume refuses work and demands the affected fileobject to be handled first. In dialog mode one may enter "go on" after the file is checked, and eventually repaired or deleted. In non-dialog mode the program aborts until the offending command -resume_last_dirty in file $(scdbackup -where tmp)/askme_resume_state_"$UID"_dirty is removed or set to an empty file address. File objects recorded as "dirty" are questionable in any way. The directories in the path of such a file may been left with too generous permissions for the owner or too sparse permissions for anybody. One should check and correct. (Owners usually have rwx on their directories so that is rarely a problem.) scdbackup_askme has the appeal of a command line shell but it should be not too hard to create a graphical frontend for it. The file doc/saskme_frontend.c contains example code covering the task of forking scdbackup_askme and to perform dialog with this forked process. (No GUI code included.) Access Permissions Initially only the installing user and the superuser have permission to run backups by help of a scdbackup installation. In hostile or even just weakly controlled environments one should unconditionally stay with these settings and eventually install independend copies of scdbackup for additional users. In the case of a strictly personal computer it may seem tolerable to have the superuser using the scdbackup installation of the desktop user. But one should be aware, that entertainment components of web browsers and other programs impose a risk that the desktop user might get overtaken by intruders. Nevertheless it may be desirable to have a single scdbackup installation which can be shared by several users. In this case the superuser should install but *absolutely not use* that installation. In case of permission scheme "group_only", the superuser has to set the desired group membership of the installation (e.g. by chgrp -R ). The script ./SET_PERMISSIONS allows the installing user to loosen the access restrictions. The script expects as only argument one of the following permission scheme names : owner_only this is the default described above group_only grants group members the same rights as the installing user anybody allows any user to run backups but tries to keep them from spoiling the configuration. Not really safe. custom prepared name for a custom permission scheme: edit SET_PERMISSIONS and search "Change this according to" unprotected as the name says : totally unprotected and totally unsafe Those settings are permanent and get restored whenever ./CONFIGURE_CD or ./CONFIGURE_DVD are run. Restore the most secure settings by entering the scdbackup installation directory and executing : ./SET_PERMISSIONS owner_only ----------------------------------------------------------------------------- Inner details - Appendice See table of content at the beginning ot this text. ----------------------------------------------------------------------------- Appendix DIRECTORIES : Since version 0.8.5 the components of the scdbackup system are stored in several sub directories of the installation directory. Especially those files which need to be written during daily operation of scdbackup or during configuration are now separated from those files which usually need not to be changed. After unpacking the installation directory scdbackup-0.9.0 contains these data files : TIMESTAMP the timestamp of the tar archive cd_get_variables reads the configuration for CD dvd_get_variables reads the configuration for DVD scdbackup_switcher dispatcher script for user commands and language choice A few symbolic links refer to files in the the ./doc directory : COPYING LIESMICH README These directories are created below scdbackup-0.9.0 : aux a few parts which do not fit into any other category bin binary programs, resulting from C sources cmd user accessible commands conf configuration files and custom intermediate scripts doc documentation inst configuration scripts logs list files with checksum records scripts worker scripts of the scdbackup system (for bash or ksh) src C sources which lead to the programs in ./bin tmp files created during daily usage For using scdbackup at least the directories ./logs and ./tmp have to provide write permissions. There are configuration files which allow to redirect them to other addresses outside of scdbackup's installation directory. Therefore it is generally advised not to address subdirectories of scdbackup directly but to determine their actual address by $(scdbackup -where xyz) . See also the examples throughout this text. Older applications which directly use inner parts of scdbackup will not find those programs and scripts any more. Applications which restricted themselves to commands scdbackup* and sdvdbackup* and did not access configuration files directly are not affected. To ease the transition of affected applications there is a script ./COMPATIBILITY_LINKS in directory ./inst which creates a lot of symbolic links within directory scdbackup-0.9.0 . This makes available all programs which in older documentation have been advertised for use. Eventually missing links may be created by hand like this : cd $(scdbackup -where) ln -s $(scdbackup -where scripts)/cd_afio_command cd_afio_command Nevertheless i advise you to migrate eventual applications to the new directory structure. For mere safety reasons, a normal user's scdbackup installation should *not* be FHS compliant. FHS mainly addresses system-global software. To get a scdbackup installation of the superuser conformant to FHS, you would have to scatter it over your system so it matches the following settings Configuration file : Content : $HOME/.scdbackup_dir /opt/scdbackup-0.9.0 /opt/scdbackup-0.9.0/scdbackup_conf_dir /etc/opt/scdbackup-0.9.0 /etc/opt/scdbackup-0.9.0/scdbackup_tmp_dir_value /var/opt/scdbackup-0.9.0/tmp /etc/opt/scdbackup-0.9.0/scdbackup_logs_dir_value /var/opt/scdbackup-0.9.0/logs See below appendix CONFIGURATION for the meaning of these files. In order to obey the FHS imperative "Programs to be invoked by users must be located in the directory /opt//bin" one may also move the content of the ./cmd directory to ./bin . ----------------------------------------------------------------------------- Appendix CONFIGURATION : These three configuration files are used by both CD and DVD subsystem : $HOME/.scdbackup_dir the address of the scdbackup-directory (such a file is created by ./ADD_USER) May be overridden by shell variable SCDBACKUP_DIR $HOME/.scdbackup_lang the language to be used in dialog. Currently only en (english) and de (german) are supported. Any other content will default to en . May be overridden by shell variable SCDBACKUP_LANG $HOME/.scdbackup_userid only needed if shell variable EUID is not set automatically by the shell. This file may contain a unique decimal number to distinguish this user from any others of the system. The configuration scripts will prompt for a number and create such a file if they detect $EUID empty. Override this file or EUID by SCDBACKUP_USERID . (Note that this does not give you any other user's permissions or privileges.) The following configuration files get created by ./CONFIGURE_CD in directory ./conf (DVD configuration files are listed further below) This address may be overridden by the content of file $(scdbackup -where)/scdbackup_conf_dir resp. variable SCDBACKUP_CONF_DIR. Its actual address may be queried by scdbackup -where conf Each of the files *_value may be overridden by a shell variable. This variable has the same name as the filename's beginning but is written in capital letters. But note that an empty variable does not override anything. Example: Override files scdbackup_scsi_adr_value , scdbackup_speed_value and scdbackup_buffer_value to use the cruelly fast IDE writer which chokes my system if i use a file buffer at speed 12 : export SCDBACKUP_SCSI_ADR="1,0,0" export SCDBACKUP_SPEED="12" export SCDBACKUP_BUFFER="-" scdbackup_scsi_adr_value the address of the recorder (e.g. 3,0 , ATA:1,0,0 , /dev/hdc) scdbackup_speed_value the recording speed (e.g. 12) scdbackup_blanken_value whether to blank the CD before writing: "0"=no , "1"=yes (mode will be "fast") or the blank= option as for cdrecord "all" , "fast" or to perform blanking during the write run rather than in a separate blank run : "inline_all" , "inline_fast" The text may contain all three arguments for scdbackup_blank. Like: all - CD-RW scdbackup_use_prog_value for CONFIGURE_CD to remember the answer to the question which creates the next four files: scdbackup_mkisofs_value name and options of the ISO 9660 formatter program (e.g. mkisofs -J -l -L -R) scdbackup_path_list_value eventually contains cd_backup_planer options for the use of the -path-list feature of ISO formatters. Among others there is the address of the temporary list file which will buffer the path-list. Note that the text "mkisofs_path_list" gets expanded to "mkisofs_path_list"_$EUID and that "=EUID=" gets converted to $EUID. (This does only apply to reading the file but not to evaluating SCDBACKUP_PATH_LIST) scdbackup_iso_filetypes_value eventually contains options for cd_backup_planer which may allow other file types than "data" and "dir" to be handed over to the ISO formatter. Older mkisofs aborts on symbolic links, though. scdbackup_cdrecord_value name and fixed options of the CD recording program (e.g. cdrecord -v ) scdbackup_cdrecord_opt_value additional options of the CD recording program to be used with runs that write data but not with the separate blanking run. (e.g. -fs=8m -eject) scdbackup_split_dir_value for CONFIGURE_CD to remember the answer to the question which creates the following two files : scdbackup_large_file_iso_value options for cd_backup_planer concerning handling of large files when planning an ISO format backup. Note that the text "=EUID=" gets converted to the userid number. (This does only apply to reading the file but not to $SCDBACKUP_LARGE_FILE_ISO) scdbackup_large_file_afio_value As scdbackup_large_file_iso_value with afio format. Does not apply to scdbackup_sys. scdbackup_afio_value name of a program call-compatible to afio plus eventual fixed options for writing. The program must recognize option "-Z" for compression and "-" as target name for stdout. It has to expect the addresses of input files at stdin. (see: man afio) Default value is : afio -o scdbackup_buffer_input_value for CONFIGURE_CD to remember the answer to the question which creates the following two files: scdbackup_buffer_value name of the buffer file or - (e.g. /buf/isofs) scdbackup_afio_compress_value whether to enable compression with afio : 0 = no, 1 = yes, - = compress if buffer file is used (default is - ) scdbackup_z_cap_value estimated capacity for compressed afio. If empty or 0 then cd_dir_afiosize is used. scdbackup_sys_excl_value contains a list of directories to be excluded from the system backup. scdbackup_t_lock_value whether to use the locking protocol: 1 = yes, store lockfiles in scbackup directory /path = yes, store lockfiles in path (e.g. /tmp ) anything else means no locking scdbackup_bin_sbin_root_value parent directory for installation of commands scdbackup_is_configured indicates by content "yes" that scdbackup was configured sucessfully for CD. Further configuration files are not adjustable by ./CONFIGURE_CD but may be changed or overridden to customize scdbackup's behavior : scdbackup_use_shell_value contains the path of a bash compatible interpreter. scdbackup_media_cap_value maximum capacity of media used with afio archives. If an afio archive exceeds this size then it is cut on the first CD and continued at a new one. (set initially to "650m" by CONFIGURE_CD) This size is also used for blank option "shred". scdbackup_max_size_value default volume size if option -max_size is omitted. (640m if file is missing and variable not set) scdbackup_blank_speed_value the speed for blanking CD-RWs (e.g. 4) in case it differs from recording speed. (This file is not created by ./CONFIGURE_CD and usually not needed. Before my CRW8424S deceased completely it was able to blank with speed 4 but to write only with 2.) scdbackup_make_checksum_value contains options for generating MD5 checksums. This file is initially created by CONFIGURE_CD but its content may be set to a single "-" to prevent checksum computation. scdbackup_checksum_list_value address of the file containing checksum records. (if missing : .../scdbackup-0.9.0/logs/backup_log ) scdbackup_verify_adr_value default for data_source with scdbackup_verify (if missing : /dev/cdrom ) scdbackup_keep_buffer_value whether to keep the eventual buffer file with ISO image or afio archive to survive until the next volume gets written. Content "1" means yes. scdbackup_options_with_iso_value additional options for cd_backup_planer which shall take effect with any backup that produces ISO9660 file systems. scdbackup_options_with_afio_value additional options for cd_backup_planer which shall take effect with any backup that produces afio archives. The following configuration files get created by ./CONFIGURE_DVD in directory ./conf (CD configuration files are listed above) This address may be overridden by the content of file $(scdbackup -where)/scdbackup_conf_dir Like with CD configuration each of the files *_value may be overridden by a shell variable. E.g.: Override files sdvdbackup_dev_adr_value and sdvdbackup_speed_value : export SDVDBACKUP_SCSI_ADR="/dev/sr2" export SDVDBACKUP_SPEED="1" sdvdbackup_dev_adr_value the device address of the recorder (e.g. /dev/dvd) Eventually used with growisofs. sdvdbackup_scsi_adr_value the SCSI address of the recorder (e.g. 3,0) Eventually used with cdrecord, wodim, cdrskin. sdvdbackup_speed_value the recording speed (e.g. any, 1) sdvdbackup_blanken_value whether to blank the DVD before writing: "0"=no , "1"=yes (mode will be "fast") or the blank= option for sdvdbackup_blank : "all" , "fast" The text may contain all three arguments for scdbackup_blank. Like: all - DVD-RW sdvdbackup_use_prog_value for CONFIGURE_DVD to remember the answer to the question which creates the next two files: sdvdbackup_mkisofs_value name and options of the ISO 9660 formatter program. (e.g. mkisofs -J -l -L -R) sdvdbackup_path_list_value eventually contains cd_backup_planer options for the use of the -path-list feature of ISO formatters. Among others there is the address of the temporary list file which will buffer the path-list. Note that the text "mkisofs_path_list" gets expanded to "mkisofs_path_list"_$EUID and that "=EUID=" gets converted to $EUID. (This does only apply to reading the file but not to evaluating SCDBACKUP_PATH_LIST) sdvdbackup_iso_filetypes_value eventually contains options for cd_backup_planer which may allow other file types than "data" and "dir" to be handed over to the ISO formatter. Older mkisofs aborts on symbolic links, though. Default : -result_filetypes data:dir:arg_redirect sdvdbackup_cdrecord_value name and fixed options of the DVD recording program (e.g. /u/test/scdbackup-0.9.0/scripts/growisofs_wrapper ) sdvdbackup_cdrecord_opt_value additional options of the DVD recording program to be used with runs that write data. (e.g. -final_eject bufsize=64m) sdvdbackup_split_dir_value for CONFIGURE_DVD to remember the answer to the question which creates the following two files: sdvdbackup_large_file_iso_value options for cd_backup_planer concerning handling of large files when planning an ISO format backup. Note that the text "=EUID=" gets converted to the userid number. (This does only apply to reading the file but not to $SDVDBACKUP_LARGE_FILE_ISO) sdvdbackup_large_file_afio_value As sdvdbackup_large_file_iso_value with afio format. Does not apply to sdvdbackup_sys. sdvbackup_afio_value name of a program call-compatible to afio and eventual fixed options for writing. Default value is : afio -o (See above scdbackup_afio_value for more details) sdvdbackup_buffer_input_value for CONFIGURE_DVD to remember the answer to the question which creates the following two files : sdvdbackup_buffer_value name of the buffer file or - (e.g. /buf/isofs_dvd) sdvdbackup_afio_compress_value whether to enable compression with afio : 0 = no, 1 = yes, - = compress if buffer file is used (default is - ) sdvdbackup_z_cap_value estimated capacity for compressed afio. If empty or 0 then cd_dir_afiosize is used. sdvdbackup_sys_excl_value contains a list of directories to be excluded from the system backup. sdvdbackup_t_lock_value whether to use the locking protocol : 1 = yes, store lockfiles in scbackup directory /path = yes, store lockfiles in path (e.g. /tmp ) anything else means no locking sdvdbackup_bin_sbin_root_value parent directory for installation of commands sdvdbackup_is_configured indicates by content "yes" that scdbackup was configured sucessfully for DVD. Further configuration files are not adjustable by ./CONFIGURE_CD but may be changed or overridden to customize scdbackup's behavior : sdvdbackup_use_shell_value contains the path of a bash compatible interpreter. sdvdbackup_media_cap_value maximum capacity of media used with afio archives. If an afio archive exceeds this size then it is cut on the first DVD and continued at a new one. (set initially to "4480m" by CONFIGURE_DVD) This size is also used for erasing (blanking) DVD. sdvdbackup_max_size_value default volume size if option -max_size is omitted. (4250m if file is missing and variable not set) sdvdbackup_make_checksum_value contains options for generating MD5 checksums. This file is initially created by CONFIGURE_DVD but its content may be set to a single "-" or made empty to prevent checksum computation. sdvdbackup_checksum_list_value address of the file containing checksum records (default: .../scdbackup-0.9.0/logs/backup_dvd_log ) sdvdbackup_verify_adr_value default for device address with sdvdbackup_verify (if missing : sdvdbackup_dev_adr_value ) sdvdbackup_keep_buffer_value whether to keep the eventual buffer file with ISO image or afio archive to survive until the next volume gets written. Content "1" means yes. sdvdbackup_options_with_iso_value additional options for cd_backup_planer which shall take effect with any backup that produces ISO9660 file systems. sdvdbackup_options_with_afio_value additional options for cd_backup_planer which shall take effect with any backup that produces afio archives. This file may be created by hand before running ./CONFIGURE_DVD : sdvdbackup_use_prodvd_value whether to configure for cdrecord (file content "1") or for growisofs (any other content). This file preserves the name of the permission scheme for ./SET_PERMISSIONS . It applies to both, CD and DVD permissions : scdbackup_set_permissions_value one of the names mentioned in paragraph "Access Permissions" above. The files to redirect the directories which need w-permission for daily usage apply to both CD and DVD : scdbackup_tmp_dir_value may contain the address of a directory which will be used instead of the ./tmp directory within scdbackup's installation directory. scdbackup_logs_dir_value may contain the address of a directory which will be used instead of the ./logs directory. Redirection of ./conf, ./tmp, or ./logs is only recommended for situations where scdbackup is installed on a read-only filesystem and the writeable directories need to be outsourced to a rw-filesystem. Other (expert) applications are: sharing logs between several scdbackup version installations of the same user, setting up an FHS compliant installation (FHS with scdbackup is safe only if no other than the superuser is allowed to use it !). One may view the effective configuration of a scdbackup environment by scdbackup -show_configuration | less sdvdbackup -show_configuration | less The burn scripts scripts/cd_backup_command , scripts/cd_afio_command , scripts/cd_pipe_command allow the use of three intermediate scripts or programs. If you provide executable code under one of the following file names within the ./conf directory, it will run at the given occasion. Each of the runs gets at least eight arguments : $1 Address of buffer file ("-" or "" if on-the-fly) $2 SCSI address used by cdrecord or device address used by growisofs $3 Caller ("iso9660" for scdbackup, "afio" for scdbackup_afio) $4 Volume number $5 Total number of volumes $6 scdbackup installation directory $7 cdrecord program used (with options like -v) (contains e.g. .../growisofs_wrapper if running for DVD) $8 formatter program used (with several options) These program runs are prepared : conf/volume_prompt_script before the user gets opportunity to press Enter. conf/volume_preburn_script after the eventual recorder lock is refreshed and before the eventual blank run. conf/volume_afterburn_script after the run of the writer program ended and before the eventual buffer file gets removed. This script gets additional arguments: $9 Address of the eventual checksum list ${10} Recordname of the record written to this list ${11} Timestamp of record ${12} Size ${13} Checksum (superstition brings bad luck. U.Eco) Make sure your shell can evaluate ${10} or use shift to access double digit parameters (after copying all single digit parameters needed). conf/volume_abortedburn_script runs instead of volume_afterburn_script if the writer program indicates failure and the whole backup run is to be aborted. There may also be scripts specialized for a particular media type. If both - the general script and the specialized script - are present, then the generalized script gets executed first and the specialized script gets excecuted after that. Both scripts get the same arguments. Specialized scripts for CD are : conf/volume_prompt_script_cd conf/volume_preburn_script_cd conf/volume_afterburn_script_cd conf/volume_abortedburn_script_cd For DVD : conf/volume_prompt_script_dvd conf/volume_preburn_script_dvd conf/volume_afterburn_script_dvd conf/volume_abortedburn_script_dvd Example : Put your backup into a set of hard disk files rather than onto a set of media. For that prepare a writer which directs the volume stream into a disk file : export SCDBACKUP_CDRECORD="$(scdbackup -where scripts)/pipe_data_to /home/archive_dir/next_volume" Create an afterburn script which renames the disk file to a filename like "A50922.132751_volume_1_7" : echo '#!/bin/sh' > $(scdbackup -where conf)/volume_afterburn_script echo 'mv /home/archive_dir/next_volume /home/archive_dir/"${11}"_volume_"$4"_"$5"' >> $(scdbackup -where conf)/volume_afterburn_script chmod u+x $(scdbackup -where conf)/volume_afterburn_script Now run your backup and have "yes" answer any media prompt yes | scdbackup_afio -max_size 1g /opt /home Do not forget to remove the afterburn script rm $(scdbackup -where conf)/volume_afterburn_script unset SCDBACKUP_CDRECORD Startup Files for Information+Restore Tool Commands scdbackup_askme and sdvdbackup_askme may execute the lines of a startup file as options before any arguments or dialog input lines are executed. The files must be readable but should not be executable. Each line may be any option which is allowed for the commands. Because the startup lines are read as first options they eventually will be in effect when the info source is read in. Empty lines and lines which begin with '#' are ignored. There is no way to set the info source within the startup file. It has always to be the first argument of the commands. conf/scdbackup_askme_rc is read by command scdbackup_askme conf/sdvdbackup_askme_rc is read by command sdvdbackup_askme ----------------------------------------------------------------------------- Appendix WORKERS : The commands use a program called bin/cd_backup_planer as central component. cd_backup_planer splits a list of directories into pieces which then each fit on CD-R or other media. It generates a script with backup commands. The lists of files or directories are determined according to maximum capacities in terms of total size and number of items. Oversized directories are split automatically. Oversized files are skipped unless a "Large file split directory" was defined during configuration. Also the program is able to generate a shell script that can print several informations about the backup. This script then is included in each piece of the backup. Read text doc/cd_backup_planer_help or execute $(scdbackup -where bin)/cd_backup_planer -help | less to get a description of the program and its options. To avoid problems while running mkisofs the program performs a collision test of target names and a readability test of source names. Collisions are circumvented by automatical renaming. Problematic filenames are stated explicitely. Unreadable files and directories may be excluded automatically. The command scripts (e.g. /usr/bin/scdbackup) merely determine the location of the scdbackup system and then start a master script there : scdbackup_switcher finds out what language to use in dialog. It reports the command name, the scdbackup version and the language code on stderr. Then it starts one of the worker scripts. If the only argument of the command is one of the following, scdbackup_switcher prints a reply to stdout and ends rather than starting the program : -version "scdbackup" versioncode timestamp -which the address of the effective worker script -where the address of the effective scdbackup installation directory. An optional second argument may request the actual address of a scdbackup subdirectory in case it is redirected from its usual place. Like : bin conf doc logs scripts tmp (see also its usage throughout this text) -show_configuration a list of variable names and contents. See appendix CONFIGURATION. -show_configuration_exported a list of export commands for the shell. Content is quoted. The scripts use two common shell code snippets to read the configuration settings : cd_get_variables reads the variables which where set with CONFIGURE_CD . (to be executed inline via dot) dvd_get_variables reads the variables which where set with CONFIGURE_DVD (and uses them to override CONFIGURE_CD's settings in cd_get_variables). The following files are stored in directory ./scripts . scdbackup_switcher gets ordered to start one of the following scripts : backup_iso_en plans and performs backups in ISO 9660 format. It is used by scdbackup, scdbackup_home, sdvdbackup, sdvdbackup_home backup_afio_en plans and performs backups in archive format. It is used by scdbackup_afio, scdbackup_sys, sdvdbackup_afio, sdvdbackup_sys blank_en blanks a single media and/or overwrites it with random bytes. (scdbackup_blank, sdvdbackup_blank) verify_en verifies a single media against recorded checksums. (scdbackup_verify, sdvdbackup_verify) askme_en starts the binary information program. (scdbackup_askme, sdvdbackup_askme) clear_en releases all possibly open locks of a scdbackup installation. show_confguration performs option -show_configuration (see above). Common code snippets which make the command scripts less redundant : handle_existing_script performs options like -resume and other activities which deal with the backup script that resulted from the previous run. handle_planer_failure is called if the run of cd_backup_planer fails. install_backup_script installs the new resulting backup script and makes it ready for execution. Size determination is crucial for planning the distribution of backup data. Besides the -internal size measure there may be used du -sk or scripts which simulate the intended backup format : cd_dir_isosize Deprecated, is able consume enormous virtual memory : can be used instead of du -sk to estimate more accurately the size which a directory will have in ISO9660 filesystem. cd_dir_afiosize can be used to determine the size of a (possibly compressed) afio archive made from a particular directory. This is very slow since the archive is actually generated, counted and discarded. Several scripts scripts which refer to writing of media and pseudo-media : cd_backup_command is able to write a single piece to a CD or DVD writer as ISO9660 file system. cd_afio_command is able to write a single piece as afio archive to a CD or DVD writer. If a buffer file is used then the archive contains compressed files (afio option Z). For CDs written on the fly this isn't suitable since the output rate of a compressor can be very low (i.e. the compression rate can be very high). cd_pipe_command directs stdin to a single CD or DVD without any further formatting. I.e. data already have to be formatted. This is used with scdbackup -pipe_to_media . If the first argument is -multi_volume then multi volume rules as with scdbackup_afio apply. Use this mode for piping large archives or dumps onto several media. cd_blank_command blanks a CD-RW using the configuration of scdbackup and eventually starts shredder_command. sdvd_blank_command warns of shredder_command and starts it for DVD. shredder_command overwrites a media with random data. afio_sencrypt a script which produces an encrypted and eventually gzip compressed afio archive. Encryption is done by sencrypt. See http://scdbackup.webframe.org/README_sencrypt See also appendix CRYPTO below. One of three modes is selectable by the first argument. Depending on the mode a second argument may be needed : -each_file_by keyphrase_file afio-traditional encoding of each single file yields a rugged archive but is quite slow. The key phrase has to be stored in a file and the second argument has to be its address. In case of compression the first 10 bytes of each file stay unencrypted to avoid attacks via known cleartext. Unpack (uncompressed) : afio -i -P 'sencrypt' -Q '-d' \ -Q '-f' -Q "$keyphrase_file" -Z /dev/cdrom Unpack (compressed) : afio -i -P 'sencrypt' -Q '-d' -Q '-g' \ -Q '-f' -Q "$keyphrase_file" -Z /dev/cdrom -whole_archive_by keyphrase_file encryption of the resulting archive as a whole. Again the key phrase has to be stored in a file. With uncompressed archives the first 322 bytes stay unencrypted. With compressed ones it is 162. Unpack (uncompressed) : sencrypt -d -s 322 -f "$keyphrase_file" \ &2 ; read dummy dd if=/dev/hdc bs=2048 count=2293760 ) | gunzip | star -t -v The number 2293760 is for DVD (4480 MB). With CD use 332800 (650 MB). With a single DVD life is simpler : gunzip