cd_backup_planer version 0.9.2 usage : bin/cd_backup_planer [option | file] ... Usually this command splits a list of directories into volumes which fit on CD-R media. It generates a script with backup commands. The lists of files or directories are determined according to maximum capacities in terms of total size and number of items. A result list can contain only directories and datafiles. Other file types may be included in directories but most CD writer programs will not process them. The result lists consist of target and source addresses suitable for the program mkisofs in its more modern versions: If fileaddress is a directory: /targetaddress/=fileaddress If it is a data file: /targetaddress=fileaddress These lists are appended to the backup command in order to fill one CD medium. If not given explicitely and not redirected by a collision, the targetaddress is the same as the fileaddress. Depending on the -target_escape mode the character "=" either is escaped or changed to "_". Other output forms of the result lists are available too. Since the working directory, the root directory and possibly the -main_dir all are mapped to the root directory collisions of targetadresses may occur. A builtin collision test can detect and avoid them by renaming one of the affected targetadresses. A filecheck can be made to find problematic files: - those which are ignored by mkisofs-1.12b3 if not given explicitely as arguments. - those which cannot be read (files or directories) this can mislead the estimation of resulting directory sizes. The combined options -filecheck -append -filecheck -exclude_unreadable can help to work around these problems. Incremental backups are possible with an arbitrary number of levels -level 0 does a full backup with no time constraints. -level N backups those files that have changed since the last run of -level N-1 . -level N-1 has to exist already. - A backup with -level N makes invalid all levels above N. The data necessary for keeping track of levels are kept in a dedicated directory which is addressed by -conf_dir. It also hosts all parameters of the backup like filelist or maximum sizes. Incremental backups use the option -younger_than which possibly also includes some older files in the backup. Regrettably, mere name changes do not change the file's time stamps in the general Linux filesystem model (other than e.g.: chmod u+r ). Therefore lengthy checksum computations often become unavoidable. Since a large number of single filenames could make problems with older versions of mkisofs, the incremental backup mainly tries to exclude directories which contain only very few young files and to mention only these few exemption files. Also a certain effectivity is required in terms of byte gain versus number of filenames. This required quote is high if -max_items is set to a small value. An additional script can be generated which is able to print several informations about the backup. Special options are: -hilfe, -id, -count, -list, -levelall , -suball, -fast, -help Other arguments are looked up as file names in the lists. If a name or a directory above this name is found, then the part number, the item and the file name are printed. If none of the files can be found this way, then the script's exit value is 1. This script is stored on the disk and included in each part of backup if a name is given with option -info_script (see below). It is estimated to have 100k size (in case that your scripts actually exceed this size reduce -max_size accordingly). If the info script gets included and the output form for mkisofs (default or -form_script) is used then all possible fileadresses which may lead to the same name on the CD are excluded. A number as optional argument tells the backup script to skip all volumes with lower numbers. To continue a backup which has been interrupted while volume 3 was ready to be written: scriptname 3 Then the script skips volumes 1 and 2. If the script has been generated with -status_file then it can find the appropriate volume automatically by reading the status file This automatic is triggered with the argument "AUTO". Arguments which do not begin with '-' are file addresses. Fileadresses may be prefixed by target adresses. This is useful for composing a CD from various scattered sources. The usual separating character is "=" but another one may be chosen with the option -target_separator. Example: -target_separator '=' /images=/home/private/images ... All fileaddresses together define the "backup area" from which some sub trees may be excluded by options -no, -not, -xdev, -localdev . This set of file objects may get examined and reduced further (e.g. by -younger_than) and so becomes the "backup content". Options: Several options (like -max_size) expect size arguments. This may be integer numbers (like 1000000000) or floating point numbers (like 1.25e9). They may have appended a binary unit letter out of {b,k,m,g,t,p,e} which multiplies the number by {1,1024,1024*1024,1024*1024*1024,...,pow(1024.0,6.0)}. Example: 640m = 640*1024*1024 = 671088640 -help print this text and exit program -hilfe diesen Text ausgeben und Programm beenden (like -help , use german language) -version print version identification on stdout, if not only argument: print warning message on stderr, in any case : end program with exit value 4. -- the next argument is a file, not an option. -accept_all_filetypes Is equivalent to : -result_filetypes any -alldev do not exclude mounted filesystems. (default) -backup_cmd text the given text is used as command in the script generated by this program. text is subject to special -bdir substitution. (default: cd_backup_kommando) -bd set size paramters suitable for single layer Blu-ray media of nominal "25 GB" with medium size formatting. -bdir directory sets the scdbackup installation directory. Currently the only use of this information is -bdir substitution. It is performed when reading file specific_arguments of an incremental backup configuration. If such an argument starts with @SCDBACKUP_DIR@ then this start piece gets replaced by the -bdir directory. This works too if a quotation mark is prepended. With some very few arguments the start piece is replaced also if it matches grep expression .*/scdbackup-[0-9]\.[0-9]\.[0-9]/.* This is called special -bdir substitution. -bd_dl set size paramters suitable for double layer Blu-ray media of nominal "50 GB" with medium size formatting. -blocklist_fileadr address_prefix if address_prefix is not empty and not "-" then it will be used to generate addresses of block checksum lists which are eventually to be stored. Wether there are such lists depends on: -filter_checksum_blocksize with -filter_md5 -scan_for_checksum -blocklist with -search_md5 The effective file address will consist of address_prefix, record name, date stamp and size value of the checksum record. For example : "/tmp/x_" may lead to: /tmp/x_1_1_A41123.194219_229208064 -changetest_options code:code:... chooses properties to be compared during check -younger_than. Compared are the file objects on the disk with those obtained by -load_old_tree. Multiple codes are separated by ":". type e.g. directory, data file, softlink size bytecount crc CRC-32 or MD5 content checksum inode internal ID number of filesystem object off disable comparison with old tree adr enable comparison with old tree (default) mistrust avoid a shortcut with CRC depending on directory timestamps. dir_trust allows an even faster shortcut with CRC but is not safe if directories get renamed in order to replace existing directories. assume_old assume all timestamps as old enough ignore_ctime inspect only mtime but not ctime. ignore_device with inode: do not compare device numbers no_split_economy no check for uneconomical directory splittings by small, old files. NOTE: -load_old_tree with nonempty addresses sets -changetest_options to "type:size:crc:adr" Set own -changetest_options after -load_old_tree If both :inode: and :crc: are chosen, and both information is available, then :inode: will be used for decision. -collision_test mode controls the collision test for target names Modes can be (default is -redirect ) : -off do not perform collision test -on warn about collisions -redirect change target names if necessary -abort abort if a collision is detected -compare_checksum recordlist searchname old_limit young_limit the program ends immediately when stdin closes. It passes stdin to stdout without changes. Reads from file recordlist a record matching the other three arguments. The record may have been written by -filter_crc32 or -filter_md5. In order to match, searchname has to be "-" or exactly the same as the name of the record. If old_limit is not "-" all older records do not match. If young_limit is not "-" then all younger ones do not match. (see -younger_than for time formats) The last matching record in the list is read and its recorded data amount is read from stdin to compute a checksum which is compared with the recorded checksum. If stdin delivers enough bytes and the checksums are equal, then the return value of the program is 0. In case of not enough input or wrong checksum exit is 1. Any surplus bytes at stdin are ignored. -complete_list_adr fileaddress writes a complete file list of the backup area to the given fileaddress. If fileaddress contains no "/" and a -level is given then the file is created in the level_# directory. (See below : -conf_dir) If no -level is given then such a file goes to -tmp_dir. If the given fileaddress ends with .gz then the output gets piped through program gzip. If a target is given in fileaddress then the resulting file will be appended to the last backup volume. (e.g. /complete.gz=complete.gz) The list is written in a special format to save disk space while still being human readable. The first and the last characters of an address determine its type /x/x/x/ absolute directory address /x/x/x absolute file address ./x/ descent into directory x ./x file x in current directory .. one directory up ../ to start directory #=# xxxx additional information The first directory address given is absolute and sets the start directory for relative addresses. By ./x/ and /x/x/x the directory cursor can be moved away from the start directory. As long as the cursor is in the start directory, addresses get prepended a target address which also is valid for all relative addresses below. Both parts are separated by a "="-character. If the target address itself contains "=" or "\" then they are prepended by a "\" each. The resulting list is usable with -vanish_list_adr -composition_memory depth width memory_limit defines size and rules for an acceleration buffer which may help with dividing very large and deep directory trees into digestible pieces. Parameter depth controls how many levels of sub directories shall be recorded for the case that their sizes is requested again after a directory split. If they are recorded then there is no to examine the subtrees on disk again. Thus the buffer can save a lot of time but can also need lots of memory. Parameter width controls how many files a directory may contain to be recorded in the buffer. The memory consumption per file depends mainly on the lenght of the filenames. Plus about 20 Bytes. Total memory consumption is curbed by parameter memory_limit. If memory_limit is preceeded by "-" then freed buffer is not regarded as reusable. Default : 3 1000 4m (0 0 0 would disable the buffer) -composition_mode criterion defines the sequence of equal ranked file objects when backup volumes are composed. All modes try to keep directory trees together. When arranging objects from the start list or objects within split directories there may be used one of several criteria to choose the next file object : bestfit the largest object that will fit target the next higher target address (alphabetically) source the next higher source address time the next younger file object crawler the next one found by tree crawler default currently like bestfit alphabetical currently like target chronological currently like time fast currently like crawler With criteria other than bestfit there is an increased chance to get larger directories split over two adjacent backup volumes. This probability is mainly controled by -min_size_percent where a low percentage causes waste of media capacity and high percentage causes more directory splits. Waste will be prevented by splitting only if the number of -max_items is large enough to accept nearly all Fileobjects in the split directory. Criterion crawler yields results like criterion source as long as the directories have a moderate size. At volume borders it always splits directories. It avoids one or two passes over the backup area which may need substantial time if RAM cannot buffer the whole backup area tree information. -content_list_adr fileaddress writes a file list of the backup content. With the same rules and formats as -complete_list_adr this option lists only those file objects which are part of a backup volume. The content lists and vanish lists of all levels together contain the same information as a single complete list. Additionally a content list tells the volume numbers by marking volume borders. -conf_dir directoryaddress sets the location of a backup configuration. It contains files which are created and used by incremental backups. (see -level) specific_arguments : full backup's options list each line is one argument. highest_level : highest valid level number ASKME_level_# : info script of -level # level_#/ : level directory, contains last_run_date : time stamp of last run number_of_volumes : as it says content_file_list.gz : see -content_list_adr and -load_old_tree vanished_file_list.gz: see -vanish_list_adr VANISHED.sh.gz : see -vanish_script_adr -cut_file source_address start_byte byte_count target_address The program ends immediately after this option is done. It copies an interval of bytes out of a file into a new one. Intended to be used by -split_large_files . If source_address is '-' then data are read from stdin and a number of start_byte bytes is skipped, else the source_address is opened as file and the read pointer is positioned at start_byte. target_address may be '-' for stdout or a suitable address for a new file. The intermediate messages are controlled by -pacifier . -debug_filelist list_adr directory_prefix The program ends immediately after this option is done. It prints the list lines together with eventual resulting absolute file addresses. directory_prefix is prepended to any of these addresses. List lines get prepended three blanks and the count of already listed file items. So they may get filtered away by grep -v '^ ' -debug_stderr_log The program ends immediately after stdin closes. It changes carriage returns into newlines and tries to spare empty lines. Intended for making more readable the results of scdbackup ... 2>&1 | tee -i ... -debug_targetlist list_adr directory_prefix like -debug_filelist but showing the target addresses of the files. -delete_filelist list_adr directory_prefix NOTE: for removing obsolete files during the restore of a backup you should better use -delete_targetlist. The program ends immediately after this option is done. It reads file addresses from list_adr together with type, size, checksum properties. These informations get compared with the current state of the file whose address is composed from directory_prefix and the address read from the list. The file list has to be generated with -vanish_list_adr and must contain type, size and checksum informations. If the addressed file exists and if the other properties match, then the file gets deleted. Directories get deleted if they are empty. If everything existed and matched, the program exits with value 0. Else this value is 1. -delete_targetlist list_adr directory_prefix like -delete_filelist but processing the target addresses of the vanished files. If there have been target=source redirections when a backup was made, then this can make a big difference. -dir_list_adr fileaddress writes a complete directory list of the backup area to the given fileaddress. The output format is similar to -complete_list_adr but the following fields are fixely enabled: "type:chmod:chown:chgrp:date:size:crc:adr" with chmod:chown:chgrp:date defining access permissions, ownership and time stamp. If MD5 checksums have been enabled by -list_output_format then they are used here rather than CRC32 checksums. -dir_script_adr fileaddress writes a shell script for restoring the access permissions of the directories of the backup area. The script will use target addresses relative to the CDs' root directory. -dir_size_cmd text the given text is used as command for determination of a directory's size. The command has to expect a single argument (the adress of the directory) and give back as first word on standard output the size in terms of kilobytes (like du -sk does). Default: "-internal" does not run an external command but recursivly sums up the sizes of file objects and the length of their addresses. Added is an overhead of 256 bytes per file. It may be changed by "-internal+$num" . For example "-internal+10" adds 10 extra bytes. "-internal-$num" disables adding of address length. -disable_safe_leveling is equivalent to -load_old_tree "" "" -content_list_adr "" -complete_list_adr "" -vanish_script_adr "" "" -vanish_list_adr "" "" -dvd set size paramters suitable for single layer DVD media of nominal "4.7 GB". Not suitable for DVD-RAM. -dvd_dl set size paramters suitable for double layer DVD media of nominal "8.5 GB". -dvd_ram set size paramters suitable for DVD-RAM media of nominal "4.7 GB" with medium size formatting. -exclude_size sizelimit exclude any single file which contains at least sizelimit bytes. A sizelimit of 0 disables this test. If sizelimit begins with "-" then with symbolic links the size of the link target gets tested. 'k' 'm' and 'g' are recognized like with -max_size . -extract_multiplied source skip the program ends immediately when stdin closes. It reads source sequentially and expects its data to stem from option -multiply_volumes. The given number of copies get skipped and the next one gets written to stdout. Eventually a defunct marker tag is appended. -extract_tag_tail maximum_size the program ends immediately when stdin closes. It consumes stdin until the header of a potentially valid checksum tag is encountered, which is preceeded by at least one byte. Beginning with this tag header, the rest of the stdin stream is passed to stdout. Different from -search_md5 even damaged tags will be accepted. The passed amount of data will consist of the header plus at most the amount of data given by maximum_size. Use a generous maximum_size to catch all interesting data. The exit value is 0 if data have been extracted. It is non 0 else. -extract_tag_tail_dislocated maximum_size like -extract_tag_tail but rather accepting the first occurence of a checksum tag identifier text. It is not necessary that the position information of the tag matches the position of the identifier text within the data stream. Therefore it is possible to retrieve tags from data streams where bytes are missing. On the other hand this command accepts checksum tags which are part of the payload data (which is not too uncommon). Only tags which start at position 0 in the data stream get skipped. -facl_list_adr fileaddress applies program getfacl to the backup area and writes its output to the given fileaddress. This will record standard file permissions as well as eventual ACL settings. The format is suitable for program setfacl gunzip the prefix directory for loading a directory list with -load_old_tree. Only needed if is not source. tree_source with -load_old_tree read source part of addresses. tree_target with -load_old_tree read target part. Options may be concatenated by a colon. The empty option list "-" is the same as : "overwrite:move:chmod_original:tree_target" . -min_size_percent number percentage of -max_size which has to be used of a volume before a new one may be opened. If this percentage isn't reached then a large directory may be split to fill the volume. (default is 80.0) -multiply_volume source repeat_faktor the program ends immediately when stdin closes. It reads source several times and puts out the copies to stdout. Markers are inserted between the copies in order to enable option -extract_multiplied to fetch single copies out of the resulting data. -no all following fileaddresses are to be omitted from the result lists. This option can cause the result lists to fragment since omitting a address within a directory requires to put to the list all other addresses within this directory. -not same as -no -not_inode works like -no, but additionally tries to recognize the excluded file objects by their device and inode numbers. This test is more expensive and may put extra workload on the disk. -not_pattern shellpattern adds shellpattern to the list of search expressions which get tested against file leafnames and in case of a match cause the exclusion of the file. shellpattern may use the wildcards *,?,[...]. -not_regex regex adds the regular expression regex to the list of search expressions which get tested against file leafnames and in case of a match cause the exclusion of the file. regex is an expression as known from programs grep, sed or vi. -pacifier mode|rythm some options print intermediate messages about their progress to stderr. If you are logging stderr this may be annoying. -off disables these message -on enables a rythm of 1 message per second A number sets the rythm to the a value in seconds -timestamp prepends the messages by the number of elapsed seconds. -composition_memory displays the bufferstate of option -composition_memory -log_mode non-truncated messages in distinct lines All modes besides -off are combinable. E.g.: -pacifier 3 -pacifier -timestamp -pacifier -log_mode -permute_blocks key source blocksize blockcount datastart the program ends immediately when stdin closes. It reads file source via random access and writes the result to stdout. The output data are permuted blockwise. The output data are preceded by a header part which tells how the data have been swapped and moved : scdbackup block permutation header v0.1 blocksize=... Number of bytes per block complete_blocks=... Number of complete blocks remainder=... Number of bytes in last block permutation_key=... Permutation scheme (0 to 7) datastart=... Number of first data block key defines the permutation scheme : 0= blocks stay in sequence 1= swap both halves of the file 2= mirror sequence of blocks 3= swap 1st and 2nd quarter, sawp 3rd and 4th plus 4 = additionally interlace 1st and 2nd half (e.g. 7 = 3+4 = swap quarters, then interlace halves) key "restore" will revert the permution and put out the original file state. If key is preceded by "-" then reversion is performed, too. But in this case, the fileheader is skipped an the parameters are taken from arguments : key, blocksize, blockcount, and datastart. When converting to permuted state, blockcount can be used to set the size of a block device. Else the size is taken from file size or by counting block device bytes. Under normal circumstances, blockcount and datastart should be 0, though. -pre_checksum_size number sets the amount of data from the start of the stream which is used to compute a preview checksum. If this number greater than 0 then -filter_md5 and -filter_crc32 write two records. The first with name appendix "_pre" may be used by option -search_md5 and -search_crc32 to locate the second one which describes the full data stream. Number must be greater or equal -1. Default is 0. If number is larger than 0 then -filter_checksum_blocksize gets disabled. -prevent_doubles on|off if set to "on" the emerging backup plan is checked wether a newly appended item already is and eventually the new item is discarded. This test may need lots of time and is therefore disabled by default. -problemlist_limit number Set the maximum item number for eventual lists of problem files from -filecheck, -exclude_size, -xdev. If the number is exceeded with on of these lists then the planning run will finally abort. (Default: 10000) -process_blocklist action listfile skip source target provides an overview over eventual data damages and also recovery facilities. The program ends after this option is done. It depends on block checksum lists. See -filter_checksum_blocksize . The block checksum list is read from listfile where it has to be appended to a valid checksum tag. That may be a backup volume written with -filter_checksum_blocksize or a -blocklist_fileadr result. The skip value is the number of tags to be skipped until finally one gets read and used for processing. Normally 0 is ok. source is a file address ("-"=stdin) of a potentially damaged backup volume. target will be a file address ("-"=stdin/stdout) where undamaged blocks get assembled. Depending on action both may have to be random access files or may be allowed to be single pass byte streams. The following keywords for action are defined: "report" reads a stream from source and prints -one per line- a list of block numbers to stdout. The end of this bad block list is marked by the line "@". target is not used. "collect" reads a stream from source and writes the undamaged blocks to target via random access. "mend" reads source with random access and eventually reads copies of damaged target blocks from source by random access. If this succeeds then the undamaged block copy gets written into target. mend supports an optional start block number and an even more optional end block number. Numbering starts with 0. Keyword "end" depicts the last block. If start block number is larger than end block number then the blocks are processed in that reversed order. Examples: mend mend:1200 mend:10k:20k mend:end:0 "merge" reads source and target sequentially an writes either an undamaged block, or the damaged one from target or even a block of zeros to stdout. actions which write to target do also append the block checksum list to target's payload data. A short description of the list parameters and a summary of the result are printed to stderr. If stderr is to be interpreted by a program then you should set option -pacifier -off before option -process_blocklist . The exit value is 0 if the resulting data are supposed to be completely undamaged. It is non 0 else. -result_filetypes typelist defines what file types may appear in the result list resp result script. typelist consists of one or more of the following words separated by ":". data ordinary data files dir directories symlink symbolic links block_device device files of hard disks, CD, ... unclassified anything different from above list any everything together arg_redirect redirect explicitely given symlinks File types which are not mentioned in typelist get discarded when they are to be added to the list of a backup piece. If pseudo-type arg_redirect is given, then the arguments of this program get checked for symbolic links. Those are replaced by their link targets (data sources). -scan_for_checksum mode controls -search_crc32 and -search_md5 scanning for checksum tags (appended by -filter_checksum_padsize). Following modes are defined : -on Search enabled. Search properties unchanged. -off Search disabled. Search properties unchanged. -first Search on. End after first valid tag. -whole Search on. Search up to end of byte stream. -standalone Search on. Do not read recordlist. -withlist Search on. Read and use recordlist. -print Search on. List valid tags at end of run. -noprint Search on. Do not list. -anytag Accept any valid checksum tag. -blocklist Accept only tags with block checksum list. -pacifier Print progress messages to stderr. -nopacifier Do not print progress messages. -source:text Use file address standard input -skip:zahl Number of bytes which got skipped at input. With -source: this skipping is done after opening the file address. Combined modes are permitted. Default : -scan_for_checksum -withlist -scan_for_checksum -off -scan_for_checksum -anytag -scan_for_checksum -nopacifier -scattering -on|-off mode -on allows moving large files to CDs different than those which host their directory siblings. This helps to fulfill -min_size_percent but can lead to scattered directory contents. This option is in effect only with -composition_mode default . -search_crc32 recordlist searchname old_limit young_limit like -compare_checksum but with automatic volume recognition and only for records with CRC32 checksums. The search method depends on the setting of -pre_checksum_size . If it is 0 then all records are read, sorted by size and compared with intermediate checksums of the data stream. The stream is read up to its end and more than one match may occur if streams with identical data were recorded. With -pre_checksum_size -1 reading ends at the first match. If -pre_checksum_size is >0 then at first a matching preview record is searched to determine the record with the full stream's checksum. The preview checksums are computed by -filter_md5 and -filter_crc32 if pre_checksum_size was not set to 0. The sizes of record and search have to match in this case. -search_md5 recordlist searchname old_limit young_limit like -search_crc32 but with all records which have MD5 checksums. -set_shell_in_scdbackup_scripts shellname filename ... filename a very special option which is used only for configuration. The given files get tested wether they start with "#!". If true, the rest of the first line gets replaced by shellname. No other option may follow since the rest of arguments is used as a list of filenames. -shell_to_use progname the progname is written after "#!" into the first line of scripts generated by this program. Default: /bin/sh -special_file fileaddress size type defines a file as a special data source. Currently only type "block_device" is permitted. Additionally option -split_large_files has to be given with a split_directory and a split_command. The content of the block device will be included in the backup in form of one or more parts. If size is <=0, then the device is read up to EOF and the resulting count is used. Fileaddress may be of form target=source. -split_file_prefix text defines a text which is put in front of the names of file pieces which emerge by -split_large_files. -split_file_suffix text like -split_file_prefix . The text is put between name and counting numbers. See also -split_list_adr. -split_large_files split_directory split_size split_command sc_option sets the size limit for regular files together with a directory address and a shell command which is able to cut a piece out of a file. This program itself can act as such a command with sc_option -cut_file . Argument split_command is subject to special -bdir substitution. Files which are larger than split_size are processed as several separate parts. Shell scripts resulting from this program will use split_command to temporarily store the parts in split_directory. All partial files of a volume have to fit into this directory. The directory has to contain a file named scdbackup_split_file_directory_mark . If split_directory is '-' then this feature is disabled. split_directory '-xorriso_cut_out' uses option -cut_out of ISO 9660 formatter program xorriso. In this case no directory is needed, split_command and sc_option get ignored then. If split_size is '-' then the effective -max_size - 1m is used. Keep in mind that -exclude_size can prevent large files from being split and processed at all. If split_command is '-' then the previously set command stays in effect. Use '-' as sc_option in this case. -split_list_adr fileaddress writes a list of file parts caused by -split_large_files. The information in this file may be helpful if a script output format is chosen which does not contain target address (e.g. -form_path_list_afio). When restoring such a backup, the file parts will appear in split_directory and need to be put to their original positions. General list format, working directory, compression and backup inclusion are as described at -complete_list_adr. Each data line describes a file part by the form partprefix part_address=buffer_address partprefix has the form **partnumber/total:start_byte+byte_count** part_address consists of original_dir"/"marked_name"_"partno"_"total marked_name consists of prefix_text""original_name""suffix_text buffer_address is composed of split_dir"/"basename_of_part_address"-"hexcount where hexcount is unique for the whole backup. -status_file filename If the filename is not empty and not "-" then the result script will write the current volume number into this file before it calls the -backup_cmd . Also the script will be able to resume at the start of an interrupted volume when given the argument "AUTO" rather than a volume number. -tmp_dir directory_address sets a directory for storing temporary files. One of its uses is with list file addresses without '/' which would go to the level directories if there was any -level set (see -complete_list_adr). directory_address is subject to special -bdir substitution. -target_escape mode controls the handling of "="-characters in target addresses. mode "on" causes "=" and "\" to be preceeded by an additional "\"-character. Such target addresses are recognized by modern versions of mkisofs which know option -graft-points. mode "off" exchanges in target addresses each "=" by "_". "\" is not changed. Use with mkisofs 1.12b3. Note: Option -form_path_list sets mode to "on" -form_script sets it to "off". Afterwards it may be set explicitely by -target_escape. -target_separator text the first character in text is used as separating character between target address and file address. This does only apply to input but not to output where eventually "=" is used. An empty text disables explicit target addresses. Default setting is "=". Within the target address eventual "=" and "\" characters have to be escaped by preceeding "\" characters. Example: map /home/office to the / directory of the CD: -target_separator ':' /:/home/office -target_separator '' Example: map /home/test/=\ to /=\ -target_separator '=' '/\=\\'='/home/test/=\' -tree_hashsize number option for balancing speed of file search in the directories of old_tree against the memory consumption of -old_tree. Each directory in old_tree occupies by its hashvector number*sizeof(int) bytes. The search in large directories is accelerated by at about the factor number. The default of 20 is well suited for normal sized directories of 20 to 1000 files. Directories with several ten thousand files may slow down the program substantially whereas trees with very many directories may consume too much memory. -vanish_list_adr old_list_adr result_address checks the file objects which have been recorded some time ago in old_list_adr. Those objects which have vanished since then are reported to file result_address. If a -level is given and result_address contains no "/" then the file is created in the level_# directory. (See -conf_dir) If result_address ends with .gz then the output gets piped through program gzip. If a target is given in result_address then the resulting file will be appended to the last backup volume. (e.g. /vanished.gz=vanished.gz) old_list_adr may be output of -complete_list_adr Alternatively a plain list of file addresses may be used. If a tree gets loaded by -load_old_tree then it is used for comparison. If there is no old tree then the old_list_adr gets read directly but it has to be a complete list in a single file. NOTE : old_list_adr from this option is identical with old_list_adr from -load_old_tree. If both options are given, the last one will finally set the effective address. CAUTION: If -load_old_tree adresses are empty or -changetest_options is "off" then the result list may contain old file adresses that have been moved but the backup plan will not contain the new addresses. Best is to also test for CRCs (see -changetest_options) -vanish_script_adr old_content_list_adr result_address CAUTION: Running the result script is dangerous ! Especially if generated without CRC test Same as -vanish_list_adr but reports shell commands to form a script that removes all vanished objects. Such a removal is necessary before applying an incremental level during re-install of a multilevel backup. But this script does not check against cruel accidents. NOTE: the script contains target addresses only. -verify_filelist list_adr directory_prefix NOTE: If you want to verify the backuped files on CD or during restore then you should better use -verify_targetlist. The program ends immediately after this option is done. It reads file addresses from list_adr together with eventual informations about type, size and checksum. The informations get compared with the current state of the file. Its address is composed from directory_prefix and the address read from the list. If the addressed file exists and if the other properties match, then the file's address is printed on stdout. Else there will be an error message at stderr. If everything existed and matched, the program exits with value 0. Else this value is 1. Currently this is mainly a harmless version of -delete_filelist . But that might change. -verify_sources listfile the program ends immediately after this option is done. This option is similar to -verify_filelist but it expects addresses of form target=source and only checks the source parts. No directory prefix gets applied. -verify_source_filter shell_filter With output format -form_path_list_mkisofs the filelist can be cleaned from undesired addresses immediately before the prompt to insert a CD. The decision is made by shell_filter which has to put out any valid file to stdout and omit any invalid file. Use this filter in order to omit all files which have been removed since the backup was planned : ..../cd_backup_planer -verify_sources - An empty text as shell_filter disables filtering. shell_filter is subject to special -bdir substitution. -verify_targetlist list_adr directory_prefix like -verify_filelist but processing the target addresses of the listed files. If there have been target=source redirections when a backup was made, then this can make a big difference. -volume_info pass additional arguments to the backup command (before file names) : -v volume -t total_volumes -volume_offset number Adds number to all output of volume numbers and subtracts it from input of volume numbers. -working_directory directory_address set the working directory to the given address. This option should be used only once and only before any fileaddress is given. -write_random target amount randomseed the program ends immediately after this option is done. It writes into the target file ("-" = stdout) the given amount of random bytes or a periodical byte pattern of up to 16 bytes. If randomseed begins with "=" then the rest of it is the periodical pattern. If the "=" is followed by "~" then that pattern is inverted into its binary complement. Without "=", randomseed is the key for a pseudo random number generator based on MD5 and EXOR. If this key starts with "-" then it is used for a slow method which is free of EXOR. The number sequence of a particular randomseed is reproducible. Large amounts like 100g or 2t are permitted if the OS can handle them. -write_to_file targetfile byteaddress bytecount bytecode [... bytecode] -write_to_file targetfile byteaddress "string" piece_of_text -write_to_file targetfile byteaddress "file" sourcefile the program ends immediately after this option is done. It opens the existing targetfile for writing and positions the write pointer at byteaddress (file start = 0). If the third argument is a number between 1 and 4096 then it is the count of the following bytecode arguments. Permissible are decimal bytecodes between 0 and 255. With "string" the forth argument is written into targetfile. With "file" the forth argument gives the path of a file of which the content will be copied into targetfile. targetfile "-" means stdout. It is only permitted with byteaddress 0. sourcefile "-" means stdin. -xdev exclude all filesystems mounted below the items of the backuplist. Opposite of -alldev . -yes all following fileaddresses are appended to the backuplist (i.e. these files are to be backuped). This option ends the range of option "-no". It is the default mode for fileaddresses. If the option -yes is given at least once, then the option -no does not apply to the backuplist. ( -yes x -not x : file x is in backup x -not x : file x is not in backup) -younger_than time_definition try to backup only files where the content or inode changed since the given time. Moves of a file within the same file system cannot be detected, regrettably. Time definitions can have the following forms: Absolute timepoints can be defined by the input format MMDDhhmm[[CC]YY][.ss]] like with UNIX command 'date'. For example: 020616122000 is 6 Feb 2000 16:12:00 local time. Also the format YYMMDD[.hhmm[ss]] is recognized: 11 May 2001 23:45 = A10511.2345 1 Mar 2010 = B00301 , 14 Jul 1998 = 980714 Relative times are preceded by '-' or '+'. They are added to the current time. '-' therefore points to the past, '+' points to the future. The basic unit is the second. It can be modified by a letter at the end of the number: seconds 1s=1 , hours 1h=3600s , days 1d=24h , weeks 1w=7d , months 1m=31d , years y : Xy=X*365.25d+1d Please note : 'm' and 'y' are possibly slightly larger than the actual calendar time range. Example: -3w is 3 weeks in the past = -1814400s . Examples: Divide the content of the working directory into volumes of 625 MB and maximum 200 independend items. Create info script ASKME in working directory and include it in each backup part. bin/cd_backup_planer \ -max_items 200 -max_size 625m -info_script ASKME . >bscript chmod u+x bscript To actually perfom the backup, execute : bscript With this option the ASKME script is generated at a separate place and mapped to the root directory : bin/cd_backup_planer \ -info_script /ASKME=/home/backup_planer_dir/ASKME Divide the directores "office" and "images" into volumes of default size. Do not include subdirectories "office/private" and "images/girlfriends" Replace the omitted "images/girlfriends" by "images/cats" bin/cd_backup_planer \ office images -target_separator = /images/girlfriends=images/cats \ -not office/private images/girlfriends \ >bskript Fill DC-6250 tapes with compressed tar archives. bin/cd_backup_planer \ -backup_cmd 'read dummy ; tar cvzf /dev/rmt0' \ -form_script_raw -max_size 245m -max_items 100 \ -info_script ASKME \ . \ >bskript Create a configuration for incremental backups and perform planning of the first -level 0 run. By -level -create_configuration a new directory gets created which will be called $HOME/my_backup_conf . bin/cd_backup_planer \ -conf_dir $HOME/my_backup_conf -level -create_configuration \ ... all arguments defining the full backup ... \ >bskript The configuration may be used by simply giving this directory and the desired level. All other parameters of the backup will be read from $HOME/my_backup_conf/specific_arguments (1 line = 1 argument). One should use a fixed backup scheme. Let us assume that we made the configuration and the first -level 0 backup at sunday of week 0. A backup scheme which from monday to saturday records only the daily changes and makes a full backup each sunday. Monday : -conf_dir $HOME/my_backup_conf -level -next ... Saturday: -conf_dir $HOME/my_backup_conf -level -next Sunday : -conf_dir $HOME/my_backup_conf -level 0 Now the media of monday to saturday may be recycled. A scheme which makes quarterly (13 weeks) full backups, four-weekly and weekly updates (-conf_dir same as above). This scheme lets you retrieve old file versions for quite a long time. Week 1 : -level 1 Week 2 : -level 2 Week 3 : -level 2 Week 4 : -level 2 Week 5 : -level 1 Week 6 : -level 2 Week 7 : -level 2 Week 8 : -level 2 Week 9 : -level 1 Week 10 : -level 2 Week 11 : -level 2 Week 12 : -level 2 Week 13 : -level 0 For week 14 hop to week 1. The backup of week 12 also covers weeks 11 and 10. The backup of week 9 also covers week 8 to 1. It is advised to keep levels 0 and 1 for a longer time. To combine both schemes shown, simply replace the sunday run of scheme 1 by the weekly run of scheme 2. If you ever need to re-install an incremental backup with multiple levels you will begin with copying level 0 first. Then you need to remove the files which have vanished in the week between the level 0 backup and the next level. For that purpose you will find on the last backup volume of level 1 a file named /tmp/VANISHED.sh.gz . It does not have x-permission to avoid ugly accidents. You can only trust it if it has been produced reading a content list with CRCs or inode numbers. This is true by default but may be changed with options like -content_list_adr , -list_output_format , -load_old_tree , -changetest_options , -vanish_script_adr . If you want to be very cautious, you may use option "md5" rather than "crc" with -list_output_format. You will have to apply chmod u+x and gunzip after copying or you will have to execute it on-the-fly via a pipe like : gunzip